Detection of scene-irrelevant head movements via eye-head coordination information

Abstract

Background

Accurate motion tracking in head-mounted displays (HMDs) has been widely used in immersive VR interaction technologies. However, tracking the head motion of users at all times is not always desirable. During a session of HMD usage, users may make scene-irrelevant head rotations, such as adjusting the head position to avoid neck pain or responding to distractions from the physical world. To the best of our knowledge, this is the first study that addresses the problem of scene-irrelevant head movements.

Methods

We trained a classifier to detect scene-irrelevant motions using temporal eye-head-coordinated information sequences. To investigate the usefulness of the detection results, we propose a technique to suspend motion tracking in HMDs where scene-irrelevant motions are detected.

Results/Conclusions

Experimental results demonstrate that the scene-relevancy of movements can be detected using eye-head coordination information, and that ignoring scene-irrelevant head motions in HMDs improves user continuity without increasing sickness or breaking immersion.

Keyword

Virtual reality ; Human-centered computing ; Human-computer interaction (HCI) ; Interaction paradigms ; HCI design and evaluation methods ; User models

Cite this article

Xiaoxiong FAN, Yun CAI, Yufei YANG, Tianxing XU, Yike Li, Songhai ZHANG, Fanglue ZHANG. Detection of scene-irrelevant head movements via eye-head coordination information. Virtual Reality & Intelligent Hardware, 2021, 3(6): 501-514 DOI:10.1016/j.vrih.2021.08.007

References

1. Kavanagh S, Luxton-Reilly A, Wuensche B, Plimmer B. A systematic review of virtual reality in education. Themes in Science and Technology Education, 2017, 10(2): 85–119

2. Mahrer N E, Gold J I. The use of virtual reality for pain control: a review. Current Pain and Headache Reports, 2009, 13(2): 100–109 DOI:10.1007/s11916-009-0019-8

3. LaValle S M, Yershova A, Katsev M, Antonov M. Head tracking for the oculus rift. In: 2014 IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, China, IEEE, 2014, 187–194 DOI:10.1109/icra.2014.6906608

4. Gandrud J, Interrante V. Predicting destination using head orientation and gaze direction during locomotion in VR. In: Proceedings of the ACM Symposium on Applied Perception. Anaheim California, New York, NY, USA, ACM, 2016, 31–38 DOI:10.1145/2931002.2931010

5. Wille M, Grauel B, Adolph L. Strain caused by head mounted displays. Proceedings of the Human Factors and Ergonomics Society Europe, 2013, 267–277

6. Waterworth E L, Waterworth J A. Focus, locus, and sensus: the three dimensions of virtual experience. Cyberpsychology & Behavior, 2001, 4(2): 203–213 DOI:10.1089/109493101300117893

7. Garau M, Friedman D, Widenfeld H R, Antley A, Brogni A, Slater M. Temporal and spatial variations in presence: qualitative analysis of interviews from an experiment on breaks in presence. Presence, 2008, 17(3): 293–309 DOI:10.1162/pres.17.3.293

8. Oh C, Herrera F, Bailenson J. The effects of immersion and real-world distractions on virtual social interactions. Cyberpsychology, Behavior, and Social Networking, 2019, 22(6): 365–372 DOI:10.1089/cyber.2018.0404

9. Doshi A, Trivedi M M. Head and eye gaze dynamics during visual attention shifts in complex environments. Journal of Vision, 2012, 12(2): 9 DOI:10.1167/12.2.9

10. Pfeil K, Taranta E M II, Kulshreshth A, Wisniewski P, LaViola J J. A comparison of eye-head coordination between virtual and physical realities. In: Proceedings of the 15th ACM Symposium on Applied Perception. Vancouver British Columbia Canada, New York, NY, USA, ACM, 2018, 1–7 DOI:10.1145/3225153.3225157

11. Zank M, Kunz A. Eye tracking for locomotion prediction in redirected walking. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI). Greenville, SC, USA, IEEE, 2016, 49–58 DOI:10.1109/3dui.2016.7460030

12. Freedman E G. Coordination of the eyes and head during visual orienting. Experimental Brain Research, 2008, 190(4): 369–387 DOI:10.1007/s00221-008-1504-8

13. Dodge R. The latent time of compensatory eye-movements. Journal of Experimental Psychology, 1921, 4(4): 247–269 DOI:10.1037/h0075676

14. Clément G, Reschke M F. Compensatory eye movements. Springer Science & Business Media, New York, NY, 2010, 163–188

15. Zangemeister W H, Stark L. Gaze latency: Variable interactions of head and eye latency. Experimental Neurology, 1982, 75(2): 389–406 DOI:10.1016/0014-4886(82)90169-8

16. Bizzi E. The coordination of eye-head movements. Scientific American, 1974, 231(4): 100–106

17. Mourant R R, Grimson C G. Predictive head-movements during automobile mirror-sampling. Perceptual and Motor Skills, 1977, 44(1): 283–286 DOI:10.2466/pms.1977.44.1.283

18. Di Girolamo S, Picciotti P, Sergi B, Di Nardo W, Paludetti G, Ottaviani F. Vestibulo-ocular reflex modification after virtual environment exposure. Acta Oto-Laryngologica, 2001, 121(2): 211–215 DOI:10.1080/000164801300043541

19. Rothe S, Hußmann H, Allary M. Diegetic cues for guiding the viewer in cinematic virtual reality. In: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. Gothenburg Sweden. New York, NY, USA, ACM, 2017 DOI:10.1145/3139131.3143421

1. Yulong BIAN, Chao ZHOU, Motivation effect of animated pedagogical agent's personality and feedback strategy types on learning in virtual training environment Virtual Reality & Intelligent Hardware 2022, 4(2): 153-172

2. Yang LI, Dong WU, Jin HUANG, Feng TIAN, Hong'an WANG, Guozhong DAI, Influence of multi-modality on moving target selection in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(3): 303-315

3. Yang LI, Jin HUANG, Feng TIAN, Hong-An WANG, Guo-Zhong DAI, Gesture interaction in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(1): 84-112

4. Athirah SYAMIMI, Yiwei GONG, Ryan LIEW, VR industrial applications―A singapore perspective Virtual Reality & Intelligent Hardware 2020, 2(5): 409-420

5. Jie GUO, Dongdong WENG, Yue LIU, Qiyong CHEN, Yongtian WANG, Analysis of teenagers' preferences and concerns regarding HMDs in education Virtual Reality & Intelligent Hardware 2021, 3(5): 369-382

6. Yijun LI, Miao WANG, Derong JIN, Frank STEINICKE, Qinping ZHAO, Effects of virtual environment and self-representations on perception and physical performance in redirected jumping Virtual Reality & Intelligent Hardware 2021, 3(6): 451-469

7. Susu HUANG, Daqing QI, Jiabin YUAN, Huawei TU, Review of studies on target acquisition in virtual reality based on the crossing paradigm Virtual Reality & Intelligent Hardware 2019, 1(3): 251-264

8. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289

9. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120

10. Yuan CHANG, Guo-Ping WANG, A review on image-based rendering Virtual Reality & Intelligent Hardware 2019, 1(1): 39-54

11. Shiguang QIU, Shuntao LIU, Deshuai KONG, Qichang HE, Three-dimensional virtual-real mapping of aircraft autom-atic spray operation and online simulation monitoring Virtual Reality & Intelligent Hardware 2019, 1(6): 611-621

12. Xu PENG, Zhenyu GAO, Yitong DING, Dongfeng ZHAO, Xiaoyu CHI, Study of ghost image suppression in polarized catadioptric virtual reality optical systems Virtual Reality & Intelligent Hardware 2020, 2(1): 70-78

13. Zhiming HU, Sheng LI, Meng GAI, Temporal continuity of visual attention for future gaze prediction in immersive virtual reality Virtual Reality & Intelligent Hardware 2020, 2(2): 142-152

14. Lihui HUANG, Siti Faatihah Binte Mohd TAIB, Ryan Aung BA, Zhe An GOH, Mengshan XU, Virtual reality research and development in NTU Virtual Reality & Intelligent Hardware 2020, 2(5): 394-408

15. Stéphanie PHILIPPE, Alexis D. SOUCHET, Petros LAMERAS, Panagiotis PETRIDIS, Julien CAPORAL, Gildas COLDEBOEUF, Hadrien DUZAN, Multimodal teaching, learning and training in virtual reality: a review and case study Virtual Reality & Intelligent Hardware 2020, 2(5): 421-442

16. Jia Ming LEE, Xinxing XIA, Clemen OW, Felix CHUA, Yunqing GUAN, VEGO: A novel design towards customizable and adjustable head-mounted display for VR Virtual Reality & Intelligent Hardware 2020, 2(5): 443-453

17. Jingcheng QIAN, Yancong MA, Zhigeng PAN, Xubo YANG, Effects of Virtual-real fusion on immersion, presence, and learning performance in laboratory education Virtual Reality & Intelligent Hardware 2020, 2(6): 569-584

18. Hengwei XU, Siru LI, Wenpeng SONG, Jiajun SUN, Xinli WU, Xiaoqi WANG, Wenzhen YANG, Zhigeng PAN, Abdennour EI RHALIBI, Thermal perception method of virtual chemistry experiments Virtual Reality & Intelligent Hardware 2020, 2(4): 305-315

19. TJ MATTHEWS, Feng TIAN, Tom DOLBY, Interaction design for paediatric emergency VR training Virtual Reality & Intelligent Hardware 2020, 2(4): 330-344

20. Hongxin ZHANG, Jin ZHANG, Xue YIN, Kan ZHOU, Zhigeng PAN, Abdennour EI RHALIBI, Cloud-to-end rendering and storage management for virtual reality in experimental education Virtual Reality & Intelligent Hardware 2020, 2(4): 368-380

21. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329

22. Haoyu WANG, Jianhuang WU, A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction Virtual Reality & Intelligent Hardware 2021, 3(4): 302-314

23. Na ZHANG, Liwen TAN, Fengying LI, Bing HAN, Yifa XU, Development and application of digital assistive teaching system for anatomy Virtual Reality & Intelligent Hardware 2021, 3(4): 315-335

24. Daniel VANKOV, David JANKOVSZKY, Effects of using headset-delivered virtual reality in road safety research: A systematic review of empirical studies Virtual Reality & Intelligent Hardware 2021, 3(5): 351-368

25. Xiaolong LIU, Lili WANG, Redirected jumping in virtual scenes with alleys Virtual Reality & Intelligent Hardware 2021, 3(6): 470-483

26. Liming WANG, Xianwei CHEN, Tianyang DONG, Jing FAN, Virtual climbing: An immersive upslope walking system using passive haptics Virtual Reality & Intelligent Hardware 2021, 3(6): 435-450

27. Dangxiao WANG, Yuan GUO, Shiyi LIU, Yuru ZHANG, Weiliang XU, Jing XIAO, Haptic display for virtual reality: progress and challenges Virtual Reality & Intelligent Hardware 2019, 1(2): 136-162

28. Aiguo SONG, Liyue FU, Multi-dimensional force sensor for haptic interaction: a review Virtual Reality & Intelligent Hardware 2019, 1(2): 121-135

29. Wenmin ZHU, Xiumin FAN, Yanxin ZHANG, Applications and research trends of digital human models in the manufacturing industry Virtual Reality & Intelligent Hardware 2019, 1(6): 558-579

30. Mohammad Mahmudul ALAM, S. M. Mahbubur RAHMAN, Affine transformation of virtual 3D object using 2D localization of fingertips Virtual Reality & Intelligent Hardware 2020, 2(6): 534-555

31. Yuan WEI, Dongdong GUAN, Qiuchen WANG, Xiangxian LI, Yulong BIAN, Pu QIN, Yanning XU, Chenglei YANG, Virtual fire drill system supporting co-located collaboration Virtual Reality & Intelligent Hardware 2019, 1(3): 290-302

32. Haochen HU, Yue LIU, Kang YUE, Yongtian WANG, Navigation in virtual and real environment using brain computer interface:a progress report Virtual Reality & Intelligent Hardware 2022, 4(2): 89-114