Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board
<< Previous Next >>

2019, 1(1): 113-120

Published Date:2019-2-20 DOI: 10.3724/SP.J.2096-5796.2018.0002

A review on the application of augmented reality in craniomaxillofacial surgery

Abstract

As a new technology from the development of VR (Virtual Reality), AR (Augmented Reality) has been focused on various industries and research institutions in recent years, and got greater development. Augmented reality with 3d registration has been applied in many areas. This article mainly talks about AR’s application in craniomaxillofacial surgery in recent years, introduces preoperative and intraoperative navigation application of VR and AR, and summarizes the key technology of AR: Display, Registration, Interaction. Then we discusses the currently existing problems in the application of AR technology in surgery and prospect the development trend of AR technology in surgical application.

Keyword

Augmented reality ; Virtual reality ; Medicine ; Cranial and maxillofacial surgery ; Surgical navigation

Cite this article

Yuan GAO, Le XIE. A review on the application of augmented reality in craniomaxillofacial surgery. Virtual Reality & Intelligent Hardware, 2019, 1(1): 113-120 DOI:10.3724/SP.J.2096-5796.2018.0002

References

1. Lee K. Augmented reality in education and training. TechTrends, 2012, 56(2): 13−21 DOI:10.1007/s11528-012-0559-3

2. Azuma R T. A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 1997, 6(4): 355−385 DOI:10.1162/pres.1997.6.4.355

3. Mahvash M, Besharati Tabrizi L. A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochirurgica, 2013, 155(5): 943−947 DOI:10.1007/s00701-013-1668-2

4. Cabrilo I, Sarrafzadeh A, Bijlenga P, Landis B N, Schaller K. Augmented reality-assisted skull base surgery. Neurochirurgie, 60(6): 304−306 DOI:10.1016/j.neuchi.2014.07.001

5. Abhari K, Baxter J S H, Chen E C S, Khan A R, Peters T M, Ribaupierre S D, Eagleson R. Training for Planning Tumour Resection. Augmented reality and human factors. IEEE Transactions on Biomedical Engineering, 2015, 62(6): 1466−1477 DOI:10.1109/TBME.2014.2385874

6. Schendel S, Montgomery K, Sorokin A, Lionetti G. A surgical simulator for planning and performing repair of cleft lips. Journal of Cranio-Maxillofacial Surgery, 2005, 33(4): 223-228 DOI:10.1016/j.jcms.2005.05.002

7. Vankipuram M, Kahol K, McLaren A, Panchanathan S. A virtual reality simulator for orthopedic basic skills: A design and validation study. Journal of Biomedical Informatics, 2010, 43(5): 661−668 DOI:10.1016/j.jbi.2010.05.016

8. Wang Q, Chen H, Wu W, Jin H, Heng P. Real-time mandibular angle reduction surgical simulation with haptic rendering. IEEE Transactions on Information Technology in Biomedicine, 2012, 16(6): 1105−1114 DOI:10.1109/TITB.2012.2218114

9. Olsson P, Nysjö F, J-M Hirsch , Carlbom I B. A haptics-assisted cranio-maxillofacial surgery planning system for restoring skeletal anatomy in complex trauma cases. International Journal of Computer Assisted Radiology and Surgery, 2013, 8(6): 887−894 DOI:10.1007/s11548-013-0827-5

10. Olsson P, Nysjö F, Rodríguez-Lorenzo A, Thor A, Hirsch J M, Carlbom I B. Haptics-assisted virtual planning of bone, soft tissue, and vessels in fibula osteocutaneous free flaps. Plastic and Reconstructive Surgery Global Open, 2015, 3(8): 479−479 DOI:10.1097/GOX.0000000000000447

11. Lin Y, Wang X, Wu F, Chen X, Wang C, Shen G. Development and validation of a surgical training simulator with haptic feedback for learning bone-sawing skill. Journal of Biomedical Informatics, 2014, 48: 122−129 DOI:10.1016/j.jbi.2013.12.010

12. Marmulla R, Hoppe H, Mühling J, Eggers G. An augmented reality system for image-guided surgery: This article is derived from a previous article published in the journal International Congress Series. International Journal of Oral and Maxillofacial Surgery, 2005, 34(6): 594−596 DOI:10.1016/j.ijom.2005.05.004

13. Mischkowski R A, Zinser M J, Kübler A C, Krug B, Seifert U, Zöller J E. Application of an augmented reality tool for maxillary positioning in orthognathic surgery—A feasibility study. Journal of Cranio-Maxillofacial Surgery, 2006, 34(8): 478−483 DOI:10.1016/j.jcms.2006.07.862

14. Zinser M J, Mischkowski R A, Dreiseidler T, Thamm O C, Rothamel D, Zöller J E. Computer-assisted orthognathic surgery: waferless maxillary positioning, versatility, and accuracy of an image-guided visualisation display. British Journal of Oral and Maxillofacial Surgery, 2013, 51(8): 827−833 DOI:10.1016/j.bjoms.2013.06.014

15. Lin L, Shi Y, Tan A, Bogari M, Zhu M, Xin Y, Xu H, Zhang Y, Xie L, Chai G. Mandibular angle split osteotomy based on a novel augmented reality navigation using specialized robot-assisted arms—A feasibility study. Journal of Cranio-Maxillofacial Surgery, 2016, 44(2): 215−223 DOI:10.1016/j.jcms.2015.10.024

16. Zhu M, Chai G, Zhang Y, Ma X, Gan J. Registration strategy using occlusal splint based on augmented reality for mandibular angle oblique split osteotomy. Journal of Craniofacial Surgery, 2011, 22(5): 1806−1809 DOI:10.1097/SCS.0b013e31822e8064

17. Zhu M, Liu F, Chai G, Pan J J, Jiang T, Lin L, Xin Y, Zhang Y, Li Q. A novel augmented reality system for displaying inferior alveolar nerve bundles in maxillofacial surgery. Scientific Reports, 2017(7): 42365 DOI:10.1038/srep42365

18. Shi Y, Lin L, Zhou C, Zhu M, Xie L, Chai G. A study of an assisting robot for mandible plastic surgery based on augmented reality. Minimally Invasive Therapy & Allied Technologies, 2017, 26(1): 23−30 DOI:10.1080/13645706.2016.1216864

19. Qu M, Hou Y, Xu Y, Shen C, Zhu M, Xie L, Wang H, Zhang Y, Chai G. Precise positioning of an intraoral distractor using augmented reality in patients with hemifacial microsomia. Journal of Cranio-Maxillofacial Surgery, 2015, 43(1): 106−112 DOI:10.1016/j.jcms.2014.10.019

20. Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, Anderegg S, Weber S. Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies. International Journal of Computer Assisted Radiology and Surgery, 2012, 7(4): 547−556 DOI:10.1007/s11548-011-0660-7

21. Wang J, Suenaga H, Hoshi K, Yang L, Kobayashi E, Sakuma I, Liao H. Augmented reality navigation with automatic marker-free image registration using 3-d image overlay for dental surgery. IEEE Transactions on Biomedical Engineering, 2014, 61(4): 1295−1304 DOI:10.1109/TBME.2014.2301191

22. Wang J, Suenaga H, Yang L, Kobayashi E, Sakuma I. Video see-through augmented reality for oral and maxillofacial surgery. The International Journal of Medical Robotics and Computer Assisted Surgery, 2016, 13(2): e1754 DOI:10.1002/rcs.1754

23. Lin Y K, Yau H T, Wang I C, Zheng C, K-H Chung . A novel dental implant guided surgery based on integration of surgical template and augmented reality. Clinical Implant Dentistry and Related Research, 2013, 17(3): 543−553 DOI:10.1111/cid.12119

24. Chen X, Xu L, Wang Y, Wang H, Wang F, Zeng X, Wang Q, Egger J. Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. Journal of Biomedical Informatics, 2015, 55: 124−131 DOI:10.1016/j.jbi.2015.04.003

25. Chen X, Lin Y, Wang C, Shen G, Zhang S, Wang X. A surgical navigation system for oral and maxillofacial surgery and its application in the treatment of old zygomatic fractures. The International Journal of Medical Robotics and Computer Assisted Surgery, 2010, 7(1): 42−50 DOI:10.1002/rcs.367

26. Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, Bianchi A, Marchetti C. Augmented reality as an aid in maxillofacial surgery: Validation of a wearable system allowing maxillary repositioning. Journal of Cranio-Maxillofacial Surgery, 2014, 42(8): 1970−1976 DOI:10.1016/j.jcms.2014.09.001

27. Liu Z G, Li S Q, Li Z Q. Development and application of augmented reality. Journal of System Simulation, 2003, 15(2): 222−225 DOI:10.3969/j.issn.1004-731X.2003.02.021
柳祖国, 李世其, 李作清. 增强现实技术的研究进展及应用. 系统仿真学报, 2003, 15(2): 222−225

28. Liu Y, Wang Y T, Yan D Y. Research on display technology of augmented reality system. National Symposium on the Development and Application of Digital Imaging Technology and Related materials, 2004
刘越, 王涌天, 闫达远. 增强现实系统显示技术研究. 全国数字成像技术及相关材料发展与应用学术研讨会, 2004

29. 范利君. 增强现实系统中注册方法的研究与实现. 武汉: 中南民族大学, 2013Fan L J. Research and realization of registration method for augmented reality system. Wuhan: South-Central University for Nationalities, 2013

30. Ming D L, Liu J, Tian W. A study on the registration technology in augmented reality. Journal of Image and Graphics, 2003, 8(5): 557−561 DOI:10.11834/jig.200305182
明德烈, 柳健, 田金文. 增强现实中的虚实注册技术研究. 中国图象图形学报, 2003, 8(5): 557−561

31. Xiong Y J, Li S Q, Liu Z G. Research on augmented reality of tracking registration. Application Research of Computers, 2005, 22(4): 81−83 DOI:10.3969/j.issn.1001-3695.2005.04.027
熊友军, 李世其, 柳祖国. 跟踪注册的增强现实技术研究. 计算机应用研究, 2005, 22(4): 81−83

Related

1. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289

2. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329

3. Mengting XIAO, Zhiquan FENG, Xiaohui YANG, Tao XU, Qingbei GUO, Multimodal interaction design and application in augmented reality for chemical experiment Virtual Reality & Intelligent Hardware 2020, 2(4): 291-304

4. Mingxuan CHEN, Ping ZHANG, Zebo WU, Xiaodan CHEN, A multichannel human-swarm robot interaction system in augmented reality Virtual Reality & Intelligent Hardware 2020, 2(6): 518-533

5. Yonghang TAI, Junsheng SHI, Junjun PAN, Aimin HAO, Victor CHANG, Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems Virtual Reality & Intelligent Hardware 2021, 3(4): 274-286

6. Jinyu LI, Bangbang YANG, Danpeng CHEN, Nan WANG, Guofeng ZHANG, Hujun BAO, Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality Virtual Reality & Intelligent Hardware 2019, 1(4): 386-410

7. Xiaomei ZHAO, Fulin TANG, Yihong WU, Real-time human segmentation by BowtieNet and a SLAM-based human AR system Virtual Reality & Intelligent Hardware 2019, 1(5): 511-524

8. Chan QIU, Shien ZHOU, Zhenyu LIU, Qi GAO, Jianrong TAN, Digital assembly technology based on augmented reality and digital twins: a review Virtual Reality & Intelligent Hardware 2019, 1(6): 597-610

9. Wang LI, Junfeng WANG, Sichen JIAO, Meng WANG, Shiqi LI, Research on the visual elements of augmented reality assembly processes Virtual Reality & Intelligent Hardware 2019, 1(6): 622-634

10. Pengfei HAN, Gang ZHAO, A review of edge-based 3D tracking of rigid objects Virtual Reality & Intelligent Hardware 2019, 1(6): 580-596

11. Shenze WANG, Kaikai DU, Ningfang SONG, Dongfeng ZHAO, Di FENG, Zhengqian TU, Study on the adaptability of augmented reality smartglasses for astigmatism based on holographic waveguide grating Virtual Reality & Intelligent Hardware 2020, 2(1): 79-85

12. Lingfei ZHU, Qi CAO, Yiyu CAI, Development of augmented reality serious games with a vibrotactile feedback jacket Virtual Reality & Intelligent Hardware 2020, 2(5): 454-470

13. Yun-Han LEE, Tao ZHAN, Shin-Tson WU, Prospects and challenges in augmented reality displays Virtual Reality & Intelligent Hardware 2019, 1(1): 10-20

14. Zike YAN, Hongbin ZHA, Flow-based SLAM: From geometry computation to learning Virtual Reality & Intelligent Hardware 2019, 1(5): 435-460

15. Yang LI, Dong WU, Jin HUANG, Feng TIAN, Hong'an WANG, Guozhong DAI, Influence of multi-modality on moving target selection in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(3): 303-315

16. Yang LI, Jin HUANG, Feng TIAN, Hong-An WANG, Guo-Zhong DAI, Gesture interaction in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(1): 84-112

17. Athirah SYAMIMI, Yiwei GONG, Ryan LIEW, VR industrial applicationsA singapore perspective Virtual Reality & Intelligent Hardware 2020, 2(5): 409-420

18. Jie GUO, Dongdong WENG, Yue LIU, Qiyong CHEN, Yongtian WANG, Analysis of teenagers' preferences and concerns regarding HMDs in education Virtual Reality & Intelligent Hardware 2021, 3(5): 369-382

19. Susu HUANG, Daqing QI, Jiabin YUAN, Huawei TU, Review of studies on target acquisition in virtual reality based on the crossing paradigm Virtual Reality & Intelligent Hardware 2019, 1(3): 251-264

20. Yuan CHANG, Guo-Ping WANG, A review on image-based rendering Virtual Reality & Intelligent Hardware 2019, 1(1): 39-54

21. Shiguang QIU, Shuntao LIU, Deshuai KONG, Qichang HE, Three-dimensional virtual-real mapping of aircraft autom-atic spray operation and online simulation monitoring Virtual Reality & Intelligent Hardware 2019, 1(6): 611-621

22. Xu PENG, Zhenyu GAO, Yitong DING, Dongfeng ZHAO, Xiaoyu CHI, Study of ghost image suppression in polarized catadioptric virtual reality optical systems Virtual Reality & Intelligent Hardware 2020, 2(1): 70-78

23. Zhiming HU, Sheng LI, Meng GAI, Temporal continuity of visual attention for future gaze prediction in immersive virtual reality Virtual Reality & Intelligent Hardware 2020, 2(2): 142-152

24. Lihui HUANG, Siti Faatihah Binte Mohd TAIB, Ryan Aung BA, Zhe An GOH, Mengshan XU, Virtual reality research and development in NTU Virtual Reality & Intelligent Hardware 2020, 2(5): 394-408

25. Stéphanie PHILIPPE, Alexis D. SOUCHET, Petros LAMERAS, Panagiotis PETRIDIS, Julien CAPORAL, Gildas COLDEBOEUF, Hadrien DUZAN, Multimodal teaching, learning and training in virtual reality: a review and case study Virtual Reality & Intelligent Hardware 2020, 2(5): 421-442

26. Jia Ming LEE, Xinxing XIA, Clemen OW, Felix CHUA, Yunqing GUAN, VEGO: A novel design towards customizable and adjustable head-mounted display for VR Virtual Reality & Intelligent Hardware 2020, 2(5): 443-453

27. Jingcheng QIAN, Yancong MA, Zhigeng PAN, Xubo YANG, Effects of Virtual-real fusion on immersion, presence, and learning performance in laboratory education Virtual Reality & Intelligent Hardware 2020, 2(6): 569-584

28. Hengwei XU, Siru LI, Wenpeng SONG, Jiajun SUN, Xinli WU, Xiaoqi WANG, Wenzhen YANG, Zhigeng PAN, Abdennour EI RHALIBI, Thermal perception method of virtual chemistry experiments Virtual Reality & Intelligent Hardware 2020, 2(4): 305-315

29. TJ MATTHEWS, Feng TIAN, Tom DOLBY, Interaction design for paediatric emergency VR training Virtual Reality & Intelligent Hardware 2020, 2(4): 330-344

30. Hongxin ZHANG, Jin ZHANG, Xue YIN, Kan ZHOU, Zhigeng PAN, Abdennour EI RHALIBI, Cloud-to-end rendering and storage management for virtual reality in experimental education Virtual Reality & Intelligent Hardware 2020, 2(4): 368-380

31. Haoyu WANG, Jianhuang WU, A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction Virtual Reality & Intelligent Hardware 2021, 3(4): 302-314

32. Na ZHANG, Liwen TAN, Fengying LI, Bing HAN, Yifa XU, Development and application of digital assistive teaching system for anatomy Virtual Reality & Intelligent Hardware 2021, 3(4): 315-335

33. Daniel VANKOV, David JANKOVSZKY, Effects of using headset-delivered virtual reality in road safety research: A systematic review of empirical studies Virtual Reality & Intelligent Hardware 2021, 3(5): 351-368

34. Dangxiao WANG, Yuan GUO, Shiyi LIU, Yuru ZHANG, Weiliang XU, Jing XIAO, Haptic display for virtual reality: progress and challenges Virtual Reality & Intelligent Hardware 2019, 1(2): 136-162

35. Aiguo SONG, Liyue FU, Multi-dimensional force sensor for haptic interaction: a review Virtual Reality & Intelligent Hardware 2019, 1(2): 121-135

36. Wenmin ZHU, Xiumin FAN, Yanxin ZHANG, Applications and research trends of digital human models in the manufacturing industry Virtual Reality & Intelligent Hardware 2019, 1(6): 558-579

37. Mohammad Mahmudul ALAM, S. M. Mahbubur RAHMAN, Affine transformation of virtual 3D object using 2D localization of fingertips Virtual Reality & Intelligent Hardware 2020, 2(6): 534-555

38. Yuan WEI, Dongdong GUAN, Qiuchen WANG, Xiangxian LI, Yulong BIAN, Pu QIN, Yanning XU, Chenglei YANG, Virtual fire drill system supporting co-located collaboration Virtual Reality & Intelligent Hardware 2019, 1(3): 290-302