2020, 2(5): 454-470
Published Date:2020-10-20 DOI: 10.1016/j.vrih.2020.05.005
Development of augmented reality serious games with a vibrotactile feedback jacket
Abstract
Keyword
Cite this article
References
1.
Cai Y. 3D Immersive Interactive Learning. Singapore: Springer Verlag, 2013 DOI:10.1007/978-981-4021-90-6
2.
Xie Y, Zhang Y Z, Cai Y Y. Virtual reality engine disassembly simulation with natural hand-based interaction. In: VR, Simulations and Serious Games for Education. Singapore: Springer Singapore, 2018, 121–128 DOI:10.1007/978-981-13-2844-2_11
3.
Orozco M, Silva J, El A, Petriu E. The role of haptics in games. In: Haptics Rendering and Applications. InTech, 2012, 217–234 DOI:10.5772/32809
4.
Barajas A O, Al Osman H, Shirmohammadi S. A Serious Game for children with Autism Spectrum Disorder as a tool for play therapy. In: 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH). Perth, WA, Australia, IEEE, 2017, 1–7 DOI:10.1109/segah.2017.7939266
5.
DiPietro J, Kelemen A, Liang Y L, Sik-Lanyi C. Computer- and robot-assisted therapies to aid social and intellectual functioning of children with autism spectrum disorder. Medicina, 2019, 55(8): 440 DOI:10.3390/medicina55080440
6.
Tsikinas S, Xinogalos S. Studying the effects of computer serious games on people with intellectual disabilities or autism spectrum disorder: a systematic literature review. Journal of Computer Assisted Learning, 2019, 35(1): 61–73 DOI:10.1111/jcal.12311
7.
Lu A, Chan S, Cai Y Y, Huang L H, Nay Z T, Goei S L. Learning through VR gaming with virtual pink dolphins for children with ASD. Interactive Learning Environments, 2018, 26(6): 718–729 DOI:10.1080/10494820.2017.1399149
8.
Cai Y Y, Chiew R, Nay Z T, Indhumathi C, Huang L H. Design and development of VR learning environments for children with ASD. Interactive Learning Environments, 2017, 25(8): 1098–1109 DOI:10.1080/10494820.2017.1282877
9.
Tang J S Y, Falkmer M, Chen N T M, Bӧlte S, Girdler S. Designing a serious game for youth with ASD: perspectives from end-users and professionals. Journal of Autism and Developmental Disorders, 2019, 49(3): 978–995 DOI:10.1007/s10803-018-3801-9
10.
Borresen A, Wolfe C, Lin C, Tian Y, Raghuraman S, Nahrstedt K, Prabhakaran B, Annaswamy T M. Usability of an immersive augmented reality based telerehabilitation system with haptics (ARTESH) for synchronous remote musculoskeletal examination. International Journal of Telerehabilitation, 2019, 11(1): 23–32 DOI:10.5195/ijt.2019.6275
11.
Culbertson H, Kuchenbecker K J. Ungrounded haptic augmented reality system for displaying roughness and friction. IEEE/ASME Transactions on Mechatronics, 2017, 22(4): 1839–1849 DOI:10.1109/tmech.2017.2700467
12.
Maisto M, Pacchierotti C, Chinello F, Salvietti G, de Luca A, Prattichizzo D. Evaluation of wearable haptic systems for the fingers in augmented reality applications. IEEE Transactions on Haptics, 2017, 10(4): 511–522 DOI:10.1109/toh.2017.2691328
13.
Pridhvi Krishna M V, Mehta S, Verma S, Rane S. Mixed reality in smart computing education system. In: 2018 International Conference on Smart Systems and Inventive Technology (ICSSIT). Tirunelveli, India, IEEE, 2018, 72–75 DOI:10.1109/icssit.2018.8748813
14.
Bhargava A, Bertrand J W, Gramopadhye A K, Madathil K C, Babu S V. Evaluating multiple levels of an interaction fidelity continuum on performance and learning in near-field training simulations. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(4): 1418–1427 DOI:10.1109/tvcg.2018.2794639
15.
Johnson C I, Bailey S K T, van Buskirk W L. Designing effective feedback messages in serious games and simulations: A research review. In: Instructional Techniques to Facilitate Learning and Motivation of Serious Games. Cham: Springer International Publishing, 2016, 119–140 DOI:10.1007/978-3-319-39298-1_7
16.
Menelas B A, Benaoudia R. Use of haptics to promote learning outcomes in serious games. Multimodal Technologies and Interaction, 2017, 1(4): 31 DOI:10.3390/mti1040031
17.
Vaquero-Melchor D, Bernardos A M. Enhancing interaction with augmented reality through mid-air haptic feedback: architecture design and user feedback. Applied Sciences, 2019, 9(23): 5123 DOI:10.3390/app9235123
18.
Lee J, Kim Y, Kim G J. Effects of visual feedback on out-of-body illusory tactile sensation when interacting with augmented virtual objects. IEEE Transactions on Human-Machine Systems, 2017, 47(1): 101–112 DOI:10.1109/thms.2016.2599492
19.
de Jesus Oliveira V A, Brayda L, Nedel L, Maciel A. Designing a vibrotactile head-mounted display for spatial awareness in 3D spaces. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(4): 1409–1417 DOI:10.1109/tvcg.2017.2657238
20.
Garciavalle G, Ferre M, Brenosa J, Vargas D. Evaluation of presence in virtual environments: haptic vest and user's haptic skills. IEEE Access, 2018, 6: 7224–7233 DOI:10.1109/access.2017.2782254
21.
Rantala J, Majaranta P, Kangas J, Isokoski P, Akkil D, Špakov O, Raisamo R. Gaze interaction with vibrotactile feedback: review and design guidelines. Human-Computer Interaction, 2020, 35(1): 1–39 DOI:10.1080/07370024.2017.1306444
22.
Shim Y A, Lee J, Lee G. Exploring multimodal watch-back tactile display using wind and vibration. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Montreal QC, Canada, New York, ACM Press, 2018, 1–12 DOI:10.1145/3173574.3173706
23.
Kaul O B, Rohs M. HapticHead: 3D guidance and target acquisition through a vibrotactile grid. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. California, USA, New York, ACM Press, 2016, 2533–2539 DOI:10.1145/2851581.2892355
24.
Young E M, Memar A H, Agarwal P, Colonnese N. Bellowband: a pneumatic wristband for delivering local pressure and vibration. In: 2019 IEEE World Haptics Conference (WHC). Tokyo, Japan, IEEE, 2019, 55–60 DOI:10.1109/whc.2019.8816075
25.
Wolf D, Rietzler M, Hnatek L, Rukzio E. Face/on: multi-modal haptic feedback for head-mounted displays in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(11): 3169–3177 DOI:10.1109/tvcg.2019.2932215
26.
Louison C, Ferlay F, Mestre D R. Spatialized vibrotactile feedback contributes to goal-directed movements in cluttered virtual environments. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI). Los Angeles, CA, USA, IEEE, 2017, 99–102 DOI:10.1109/3dui.2017.7893324
27.
Delazio A, Nakagaki K, Klatzky R L, Hudson S E, Lehman J F, Sample A P. Force jacket: pneumatically-actuated jacket for embodied haptic experiences. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Montreal QC, Canada, New York, ACM Press, 2018 DOI:10.1145/3173574.3173894
28.
Kishishita Y, Das S, Ramirez A V, Thakur C, Tadayon R, Kurita Y. Muscleblazer: force-feedback suit for immersive experience. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Osaka, Japan, IEEE, 2019, 1813–1818 DOI:10.1109/vr.2019.8797962
29.
Rognon C, Koehler M, Duriez C, Floreano D, Okamura A M. Soft haptic device to render the sensation of flying like a drone. IEEE Robotics and Automation Letters, 2019, 4(3): 2524-2531 DOI:10.1109/LRA.2019.2907432
30.
Ouyang Q Q, Wu J, Wu M. Vibrotactile display of flight attitude with combination of multiple coding parameters. Applied Sciences, 2017, 7(12): 1291 DOI:10.3390/app7121291
31.
Günther S, Makhija M, Müller F, Schön D, Mühlhäuser M, Funk M. PneumAct: pneumatic kinesthetic actuation of body joints in virtual reality environments. In: Proceedings of the 2019 on Designing Interactive Systems Conference. San Diego, CA, USA, ACM, 2019, 227–240 DOI:10.1145/3322276.3322302
32.
ImmerzInc, product of KOR-FX [2020-02-20]. http://www.korfx.com/products
33.
bHapticsInc, product of TACTOT, haptic vest for torso [2020-02-20]. https://www.bhaptics.com/tactsuit/
34.
USAInc Woojer, product of Vest Pro [2020-02-20]. https://www.woojer.com/vest/
35.
Ltd VRElectronics, product of TESLASUIT [2020-02-20]. https://teslasuit.io/the-suit/
36.
Zhang Z. A Flexible New Technique for Camera Calibration (Technical report). Microsoft Research. MSR-TR-98-71, 1998
37.
Reaction Time Statistics (2020) [2020-02-20]. http://www.humanbenchmark.com/tests/reactiontime/statistics
38.
Tang Z, Lin Y, Lee K, Hwang J, Chuang J. ESTHER: joint camera self-calibration and automatic radial distortion correction from tracking of walking humans. IEEE Access, 2019, 7: 10754–10766 DOI:10.1109/access.2019.2891224
39.
Zhang Z D, Matsushita Y, Ma Y. Camera calibration with lens distortion from low-rank textures. In: CVPR 2011. Providence, RI, USA, IEEE, 2011, 2321–2328 DOI:10.1109/cvpr.2011.5995548
40.
Tehrani M A, Beeler T, Grundhöfer A. A practical method for fully automatic intrinsic camera calibration using directionally encoded light. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA, IEEE, 2017, 125–133 DOI:10.1109/cvpr.2017.21
41.
Kakani V, Kim H, Lee J, Ryu C, Kumbham M. Automatic distortion rectification of wide-angle images using outlier refinement for streamlining vision tasks. Sensors, 2020, 20(3): 894 DOI:10.3390/s20030894
Related
1. Mengting XIAO, Zhiquan FENG, Xiaohui YANG, Tao XU, Qingbei GUO, Multimodal interaction design and application in augmented reality for chemical experiment Virtual Reality & Intelligent Hardware 2020, 2(4): 291-304
2. Mingxuan CHEN, Ping ZHANG, Zebo WU, Xiaodan CHEN, A multichannel human-swarm robot interaction system in augmented reality Virtual Reality & Intelligent Hardware 2020, 2(6): 518-533
3. Yonghang TAI, Junsheng SHI, Junjun PAN, Aimin HAO, Victor CHANG, Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems Virtual Reality & Intelligent Hardware 2021, 3(4): 274-286
4. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289
5. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120
6. Jinyu LI, Bangbang YANG, Danpeng CHEN, Nan WANG, Guofeng ZHANG, Hujun BAO, Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality Virtual Reality & Intelligent Hardware 2019, 1(4): 386-410
7. Xiaomei ZHAO, Fulin TANG, Yihong WU, Real-time human segmentation by BowtieNet and a SLAM-based human AR system Virtual Reality & Intelligent Hardware 2019, 1(5): 511-524
8. Chan QIU, Shien ZHOU, Zhenyu LIU, Qi GAO, Jianrong TAN, Digital assembly technology based on augmented reality and digital twins: a review Virtual Reality & Intelligent Hardware 2019, 1(6): 597-610
9. Wang LI, Junfeng WANG, Sichen JIAO, Meng WANG, Shiqi LI, Research on the visual elements of augmented reality assembly processes Virtual Reality & Intelligent Hardware 2019, 1(6): 622-634
10. Pengfei HAN, Gang ZHAO, A review of edge-based 3D tracking of rigid objects Virtual Reality & Intelligent Hardware 2019, 1(6): 580-596
11. Shenze WANG, Kaikai DU, Ningfang SONG, Dongfeng ZHAO, Di FENG, Zhengqian TU, Study on the adaptability of augmented reality smartglasses for astigmatism based on holographic waveguide grating Virtual Reality & Intelligent Hardware 2020, 2(1): 79-85
12. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329
13. Jie REN, Chun YU, Yueting WENG, Chengchi ZHOU, Yuanchun SHI, Design and evaluation of window management operations in AR headset+smartphone interface Virtual Reality & Intelligent Hardware 2022, 4(2): 115-131
14. Yun-Han LEE, Tao ZHAN, Shin-Tson WU, Prospects and challenges in augmented reality displays Virtual Reality & Intelligent Hardware 2019, 1(1): 10-20
15. Zike YAN, Hongbin ZHA, Flow-based SLAM: From geometry computation to learning Virtual Reality & Intelligent Hardware 2019, 1(5): 435-460
16. Mohib ULLAH, Sareer Ul AMIN, Muhammad MUNSIF, Muhammad Mudassar YAMIN, Utkurbek SAFAEV, Habib KHAN, Salman KHAN, Habib ULLAH, Serious games in science education: a systematic literature review Virtual Reality & Intelligent Hardware 2022, 4(3): 189-209
17. Rafik HAMZA, Minh-Son DAO, Privacy-preserving deep learning techniques for wearable sensor-based big data applications Virtual Reality & Intelligent Hardware 2022, 4(3): 210-222
18. Hayat ULLAH, Sitara AFZAL, Imran Ullah KHAN, Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey Virtual Reality & Intelligent Hardware 2022, 4(3): 223-246
19. Ahmed L ALYOUSIFY, Ramadhan J MSTAFA, AR-assisted children book for smart teaching and learning of Turkish alphabets Virtual Reality & Intelligent Hardware 2022, 4(3): 263-277