Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board
<< Previous Next >>

2020, 2(5): 381-393

Published Date:2020-10-20 DOI: 10.1016/j.vrih.2020.07.009

VR and AR in human performance researchAn NUS experience


With the mindset of constant improvement in efficiency and safety in the workspace and training in Singapore, there is a need to explore varying technologies and their capabilities to fulfil this need. The ability of Virtual Reality (VR) and Augmented Reality (AR) to create an immersive experience of tying the virtual and physical environments coupled with information filtering capabilities brings a possibility of introducing this technology into the training process and workspace. This paper surveys current research trends, findings and limitation of VR and AR in its effect on human performance, specifically in Singapore, and our experience in the National University of Singapore (NUS).


Human computer interaction ; Virtual environment ; Human performance

Cite this article

Jun-Hao YIN, Chin-Boon CHNG, Pooi-Mun WONG, Nicholas HO, Matthew CHUA, Chee-Kong CHUI. VR and AR in human performance researchAn NUS experience . Virtual Reality & Intelligent Hardware, 2020, 2(5): 381-393 DOI:10.1016/j.vrih.2020.07.009


1. Krueger M W. Artificial reality. Reading, Massachusetts: Addison-Wesley Publishing, 1991

2. Steuer J. Defining virtual reality: dimensions determining telepresence. Journal of Communication, 1992, 42(4): 73–93 DOI:10.1111/j.1460-2466.1992.tb00812.x

3. Milgram P, Takemura H, Utsumi A, Kishino F. Augmented reality: a class of displays on the reality-virtuality continuum. In: Proc SPIE 2351, Telemanipulator and Telepresence Technologies, 1995, 2351, 282–292 DOI:10.1117/12.197321

4. Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 1994, 77(12): 1321–1329

5. Wickens C D, Hollands J G, Banbury S, Parasuraman R. Engineering psychology and human performance. New York, Psychology Press, 2015 DOI:10.4324/9781315665177

6. Zhai S M, Hunter M, Smith B A. The metropolis keyboard―an exploration of quantitative techniques for virtual keyboard design. In: Proceedings of the 13th annual ACM symposium on User interface software and technology-UIST '00. SanDiego, California, USA, NewYork, Press ACM, 2000, 119–128 DOI:10.1145/354401.354424

7. Bliss J P, Tidwell P D, Guest M A. The effectiveness of virtual reality for administering spatial navigation training to firefighters. Presence: Teleoperators & Virtual Environments, 1997, 6(1): 73–86 DOI:10.1162/pres.1997.6.1.73

8. Lee J H, Ku J, Cho W, Hahn W Y, Kim I Y, Lee S M, Kang Y, Kim D Y, Yu T, Wiederhold B K, Wiederhold M D, Kim S I. A virtual reality system for the assessment and rehabilitation of the activities of daily living. CyberPsychology & Behavior, 2003, 6(4): 383–388 DOI:10.1089/109493103322278763

9. Arora S, Sevdalis N, Aggarwal R, Sirimanna P, Darzi A, Kneebone R. Stress impairs psychomotor performance in novice laparoscopic surgeons. Surgical Endoscopy, 2010, 24(10): 2588–2593 DOI:10.1007/s00464-010-1013-2

10. SYFC–Singapore Youth Flying Club. Available from: https://www.syfc.sg/

11. Seletar Flying Club. Available from: http://www.seletar-flying-club.org/

12. Learn to Fly in Singapore | Flight School Singapore. Available from: http://flightschool.sg/

13. Technology's Role in Training Safer Doctors. Available from: http://nusmedicine.nus.edu.sg/newsletter/issue25/in-vivo/technology-s-role-in-training-safer-doctors

14. Cai Y, Chui C, Ye X Z, Wang Y P, Anderson J H. VR simulated training for less invasive vascular intervention. Computers & Graphics, 2003, 27(2): 215–221 DOI:10.1016/s0097-8493(02)00278-9

15. Anderson J H, Chui C, Cai Y, Wang Y, Li Z, Ma X, Nowinski W L, Solaiyappan M, Murphy K J, Gailloud P, Venbrux A C. Virtual reality training in interventional radiology: the Johns Hopkins and kent ridge digital laboratory experience. Seminars in Interventional Radiology, 2002, 19(2): 179–185 DOI:10.1055/s-2002-32796

16. Lian Z, Chui C K, Teoh S H. A biomechanical model for real-time simulation of PMMA injection with haptics. Computers in Biology and Medicine, 2008, 38(3): 304–312 DOI:10.1016/j.compbiomed.2007.10.009

17. Wang Y P, Chui C, Lim H, Cai Y Y, Mak K. Real-time interactive simulator for percutaneous coronary revascularization procedures. Computer Aided Surgery, 1998, 3(5): 211–227 DOI:10.3109/10929089809149843

18. Chiang P, Zheng J M, Yu Y, Mak K H, Chui C K, Cai Y. A VR simulator for intracardiac intervention. IEEE Computer Graphics and Applications, 2013, 33(1): 44–57 DOI:10.1109/mcg.2012.47

19. Chui C K, Nguyen H T, Wang Y P, Mullick R, Ragahavan R. Potential field of vascular anatomy for real-time computation of catheter navigation. Proceedings of the Visible Human Project Conference, 1996, 113–114

20. Rasool S, Sourin A. Image-driven virtual simulation of arthroscopy. The Visual Computer, 2013, 29(5): 333–344 DOI:10.1007/s00371-012-0736-6

21. Rasool S, Sourin A, Xia P J, Weng B, Kagda F. Towards hand-eye coordination training in virtual knee arthroscopy. In: Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology-VRST '13. Singapore, York New, Press ACM, 2013, 17–26 DOI:10.1145/2503713.2503715

22. Chui C K, Ong J S K, Lian Z Y, Wang Z L, Teo J, Zhang J, Yan C H, Ong S H, Wang S C, Wong H K, Teo C L, Teoh S H. Haptics in computer-mediated simulation: Training in vertebroplasty surgery. Simulation & Gaming, 2006, 37(4): 438–451 DOI:10.1177/1046878106291667

23. Zhang J, Zhou J, Huang W, Qin J, Yang T, Liu J, Su Y, Chui C K, Chang S. GPU-friendly gallbladder modeling in laparoscopic cholecystectomy surgical training system. Computers & Electrical Engineering, 2013, 39(1): 122–129 DOI:10.1016/j.compeleceng.2012.05.012

24. Xiong L F, Chui C K, Teo C L. Reality based modeling and simulation of gallbladder shape deformation using variational methods. International Journal of Computer Assisted Radiology and Surgery, 2013, 8(5): 857–865 DOI:10.1007/s11548-013-0821-y

25. Qin J, Pang W M, Nguyen B P, Ni D, Chui C K. Particle-based simulation of blood flow and vessel wall interactions in virtual surgery. In: Proceedings of the 2010 Symposium on Information and Communication Technology-SoICT '10. Hanoi, Viet nam, New York, ACM Press, 2010, 128–133 DOI:10.1145/1852611.1852636

26. Muller-Wittig W, Bockholt U, Arcos J L L, Vossl G. Enhanced training environment for minimally invasive surgery. In: Proceedings Tenth IEEE International Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises. WET ICE 2001. Cambridge, MA, USA, IEEE, 2001, 269–272 DOI:10.1109/enabl.2001.953426

27. Ho N, Wong P M, Chua M, Chui C K. Virtual reality training for assembly of hybrid medical devices. Multimedia Tools and Applications, 2018, 77(23): 30651–30682 DOI:10.1007/s11042-018-6216-x

28. Kockro R A, Tsai Y T, Ng I, Hwang P, Zhu C G, Agusanto K, Hong L X, Serra L. Dex-rayaugmented reality neurosurgical navigation with a handheld video probe. Neurosurgery, 2009, 65(4): 795–808 DOI:10.1227/01.neu.0000349918.36700.1c

29. Yang L, Chui C, Chang S. Design and development of an augmented reality robotic system for large tumor ablation. International Journal of Virtual Reality, 2009, 8(1): 27–35 DOI:10.20870/ijvr.2009.8.1.2710

30. Wen R, Chui C K, Ong S H, Lim K B, Chang S K Y. Projection-based visual guidance for robot-aided RF needle insertion. International Journal of Computer Assisted Radiology and Surgery, 2013, 8(6): 1015–1025 DOI:10.1007/s11548-013-0897-4

31. Yang T, Yang L, Liu J, Chui C K, Huang W, Zhang J, Zhou J, Lee B H, Tan N M, Wong W K D, Yin F, Chang K Y, Su Y. Robotic device for use in image-guided robot assisted surgical training. Patent, US8764448B2, 2014-07-01

32. Wen R, Tay W L, Nguyen B P, Chng C B, Chui C K. Hand gesture guided robot-assisted surgery based on a direct augmented reality interface. Computer Methods and Programs in Biomedicine, 2014, 116(2): 68–80 DOI:10.1016/j.cmpb.2013.12.018

33. Wen R, Chng C, Chui C. Augmented reality guidance with multimodality imaging data and depth-perceived interaction for robot-assisted surgery. Robotics, 2017, 6(2): 13 DOI:10.3390/robotics6020013

34. Pang Y, Nee A Y C, Khim Ong S, Yuan M L, Youcef-Toumi K. Assembly feature design in an augmented reality environment. Assembly Automation, 2006, 26(1): 34–43 DOI:10.1108/01445150610645648

35. Wang Z, Ng L X, Ong S K, Nee A Y C. Assembly planning and evaluation in an augmented reality environment. International Journal of Production Research, 2013, 51: 7388–7404 DOI:10.1080/00207543.2013.837986

36. Yew A W W, Ong S K, Nee A Y C. Towards a griddable distributed manufacturing system with augmented reality interfaces. Robotics and Computer-Integrated Manufacturing, 2016, 39: 43–55 DOI:10.1016/j.rcim.2015.12.002

37. Wang X, Ong S K, Nee A Y C. Real-virtual components interaction for assembly simulation and planning. Robotics and Computer-Integrated Manufacturing, 2016, 41: 102–114 DOI:10.1016/j.rcim.2016.03.005

38. Yew A W W, Ong S K, Nee A Y C. Immersive augmented reality environment for the teleoperation of maintenance robots. Procedia CIRP, 2017, 61: 305–310 DOI:10.1016/j.procir.2016.11.183

39. Jtc Safety Induction Course-SCAL Academy Pte Ltd. Available from: https://scal-academy.com.sg/courses/jtc-safety-induction-course

40. Tang S L, Kwoh C K, Teo M Y, Sing N W, Ling K V. Augmented reality systems for medical applications. IEEE Engineering in Medicine and Biology Magazine, 1998, 17(3): 49–58 DOI:10.1109/51.677169

41. Tan X, Chng C B, Duan B, Ho Y, Wen R, Chen X, Lim K B, Chui C K. Cognitive engine for robot-assisted radio-frequency ablation system. Acta Polytechnica Hungarica, 2017, 14, 129–145 DOI:10.12700/APH.14.1.2017.1.9

42. Tan X. Cognitive engine and deep reinforcement learning for robot-assisted surgery. Dissertation for the Doctoral Degree, Singapore, National University of Singapore, ScholarBank@NUS Repository, 2019

43. Chng C B. Spherical mechanism design and application for robot-assisted surgery. Dissertation for the Doctoral Degree, Singapore, National University of Singapore, ScholarBank@NUS Repository, 2019

44. Liu X F, Shahriar M R, Al Sunny S M N, Leu M C, Hu L W. Cyber-physical manufacturing cloud: Architecture, virtualization, communication, and testbed. Journal of Manufacturing Systems, 2017, 43, 352–364 DOI:10.1016/j.jmsy.2017.04.004

45. Joerger G, Rambourg J, Gaspard-Boulinc H, Conversy S, Bass B L, Dunkin B J, Garbey M. A cyber-physical system to improve the management of a large suite of operating rooms. ACM Transactions on Cyber-Physical Systems, 2018, 2(4): 34 DOI:10.1145/3140234

46. Li Y T, Jacob M, Akingba G, Wachs J P. A cyber-physical management system for delivering and monitoring surgical instruments in the OR. Surgical Innovation, 2013, 20(4): 377–384 DOI:10.1177/1553350612459109

47. Wang L H, Törngren M, Onori M. Current status and advancement of cyber-physical systems in manufacturing. Journal of Manufacturing Systems, 2015, 37, 517–527 DOI:10.1016/j.jmsy.2015.04.008

48. Okamoto J, Masamune K, Iseki H, Muragaki Y. Development concepts of a Smart Cyber Operating Theater (SCOT) using ORiN technology. Biomedizinische Technik. Biomedical Engineering, 2018, 63(1): 31–37 DOI:10.1515/bmt-2017-0006

49. Chng C B, Wong P M, Ho N, Tan X Y, Chui C K. Towards a cyber-physical systems based operating room of the future. In: OR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging. Cham: Springer International Publishing, 2019: 47–55 DOI:10.1007/978-3-030-32695-1_6

50. Chng C B, Chia D W T, Cao Y, Yo K, Fujie M G, Chui C K. A cyber-physical system approach to immobilization of patient on radiation treatment. In: 2019 IEEE 23rd International Conference on Intelligent Engineering Systems (INES). Gödöllő, Hungary, IEEE, 2019, 153–158 DOI:10.1109/ines46365.2019.9109457

51. Group-Based Upgrading Projects-SME Centre@SCCCI. Available from: https://smecentre-sccci.sg/group-based-upgrading-projects

52. Silversea Media Group|Immersive Media Company, Silversea Media Group. Available from: https://www.silversea-media.com/

53. Lai L. Immersive tech opens new world of opportunities for businesses. SGSME.SG, 2020. Available from: https://www.sgsme.sg/news/immersive-tech-opens-new-world-opportunities-businesses

54. Siong L C. Training and assessment of hand-eye coordination with electroencephalography. Dissertation for the Doctoral Degree, Singapore, National University of Singapore, ScholarBank@NUS Repository, 2015

55. Zhou Z Y, Cheok A D, Qiu Y, Yang X. The role of 3-D sound in human reaction and performance in augmented reality environments. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2007, 37(2): 262–272 DOI:10.1109/tsmca.2006.886376

56. Ranjan R, Gan W S. Natural listening over headphones in augmented reality using adaptive filtering techniques. ACM Transactions on Audio, Speech, and Language Processing, 2015, 23(11): 1988–2002 DOI:10.1109/taslp.2015.2460459

57. Hong J Y, He J, Lam B, Gupta R, Gan W. Spatial audio for soundscape design: recording and reproduction. Applied Sciences, 2017, 7(6): 627 DOI:10.3390/app7060627

58. Zhao M, Ong S K, Nee A Y C. An augmented reality-assisted therapeutic healthcare exercise system based on bare-hand interaction. International Journal of Human-computer Interaction, 2016, 32(9): 708–721 DOI:10.1080/10447318.2016.1191263

59. Shen Y, Gu P W, Ong S K, Nee A Y C. A novel approach in rehabilitation of hand-eye coordination and finger dexterity. Virtual Reality, 2012, 16(2): 161–171 DOI:10.1007/s10055-011-0194-x

60. Shah L B I, Torres S, Kannusamy P, Chng C M L, He H G, Klainin-Yobas P. Efficacy of the virtual reality-based stress management program on stress-related variables in people with mood disorders: the feasibility study. Archives of Psychiatric Nursing, 2015, 29(1): 6–13 DOI:10.1016/j.apnu.2014.09.003

61. Chua S H, Zhang H M, Hammad M, Zhao S, Goyal S, Singh K. ColorBless: augmenting visual information for colorblind people with binocular luster effect. ACM Transactions on Computer-Human Interaction, 2015, 21(6): 32 DOI:10.1145/2687923

62. NUSteam creates interactive, VRgame multisensory. Available from: https://news.nus.edu.sg/research/nus-team-creates-interactive-multisensory-vr-game

63. Shorey S, Ang E, Yap J, Ng E D, Lau S T, Chui C K. A virtual counseling application using artificial intelligence for communication skills training in nursing education: development study. Journal of Medical Internet Research, 2019, 21(10): e14658 DOI:10.2196/14658


1. Rachel HUANG, Carisa HARRIS-ADAMSON, Dan ODELL, David REMPEL, Design of finger gestures for locomotion in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(1): 1-9

2. Carlos ARCE-LOPERA, María José ARIAS, Gustavo CORRALES, Training birdsong recognition using virtual reality Virtual Reality & Intelligent Hardware 2021, 3(5): 397-406

3. Vivian GÓMEZ, Kelly PEÑARANDA, Pablo FIGUEROA, Lessons learned from requirements gathering for virtual reality simulators Virtual Reality & Intelligent Hardware 2021, 3(5): 407-422

4. Jiaxin LIU, Hongxin ZHANG, Chuankang LI, COMTIS: Customizable touchless interaction system for large screen visualization Virtual Reality & Intelligent Hardware 2020, 2(2): 162-174

5. Zixiang ZHAO, Quanwei ZHOU, Xiaoguang HAN, Lili WANG, Dynamic targets searching assistance based on virtual camera priority Virtual Reality & Intelligent Hardware 2021, 3(6): 484-500

6. Ahmed L ALYOUSIFY, Ramadhan J MSTAFA, AR-assisted children book for smart teaching and learning of Turkish alphabets Virtual Reality & Intelligent Hardware 2022, 4(3): 263-277

7. Shiguang QIU, Shuntao LIU, Deshuai KONG, Qichang HE, Three-dimensional virtual-real mapping of aircraft autom-atic spray operation and online simulation monitoring Virtual Reality & Intelligent Hardware 2019, 1(6): 611-621

8. Abhishek MUKHOPADHYAY, G S Rajshekar REDDY, KamalPreet Singh SALUJA, Subhankar GHOSH, Anasol PEÑA-RIOS, Gokul GOPAL, Pradipta BISWAS, Virtual-reality-based digital twin of office spaces with social distance measurement feature Virtual Reality & Intelligent Hardware 2022, 4(1): 55-75