2020, 2(6): 534-555
Published Date:2020-12-20 DOI: 10.1016/j.vrih.2020.10.001
Affine transformation of virtual 3D object using 2D localization of fingertips
Abstract
Keyword
Cite this article
References
1.
Burdea G C, Coiffet P. Virtual reality technology. John Wiley & Sons, 2003
2.
Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B. Recent advances in augmented reality. IEEE Computer Graphics and Applications, 2001, 21(6): 34–47 DOI:10.1109/38.963459
3.
Milgram P, Kishino F. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 1994, 12: 1321–1329
4.
Canessa A, Chessa M, Gibaldi A, Sabatini S P, Solari F. Calibrated depth and color cameras for accurate 3D interaction in a stereoscopic augmented reality environment. Journal of Visual Communication and Image Representation, 2014, 25(1): 227–237 DOI:10.1016/j.jvcir.2013.02.011
5.
Bernardes J. Comparing a mouse and a free hand gesture interaction technique for 3D object manipulation. In: Lecture Notes in Computer Science. Cham: Springer International Publishing, 2020, 19–37 DOI:10.1007/978-3-030-49062-1_2
6.
Feng Z Y, Xu S J, Zhang X, Jin L W, Ye Z C, Yang W X. Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system. In: Proceedings of the 4th International Conference on Internet Multimedia Computing and Service-ICIMCS'12. Wuhan, China, NewYork, Press ACM, 2012, 70–74 DOI:10.1145/2382336.2382356
7.
Han J G, Shao L, Xu D, Shotton J. Enhanced computer vision with microsoft kinect sensor: a review. IEEE Transactions on Cybernetics, 2013, 43(5): 1318–1334 DOI:10.1109/tcyb.2013.2265378
8.
Nai W Z, Liu Y, Rempel D, Wang Y T. Fast hand posture classification using depth features extracted from random line segments. Pattern Recognition, 2017, 65: 1–10 DOI:10.1016/j.patcog.2016.11.022
9.
Kang S K, Nam M Y, Rhee P K. Color based hand and finger detection technology for user interaction. In: 2008 International Conference on Convergence and Hybrid Information Technology. Daejeon, South Korea, IEEE, 2008, 229–236
10.
Gurav R M, Kadbe P K. Real time finger tracking and contour detection for gesture recognition using OpenCV. 2015
11.
Bhuyan M K, MacDorman K F, Kar M K, Neog D R, Lovell B C, Gadde P. Hand pose recognition from monocular images by geometrical and texture analysis. Journal of Visual Languages & Computing, 2015, 28: 39–55 DOI:10.1016/j.jvlc.2014.12.001
12.
Stergiopoulou E, Papamarkos N. Hand gesture recognition using a neural network shape fitting technique. Engineering Applications of Artificial Intelligence, 2009, 22(8): 1141–1158 DOI:10.1016/j.engappai.2009.03.008
13.
RaySarkar A, Sanyal G, Majumder S. Hand gesture recognition systems: a survey. International Journal of Computer Applications, 2013, 71(15): 25–37 DOI:10.5120/12435-9123
14.
Huang Y, Liu X, Jin L, Zhang X. Deepfinger: A cascade convolutional neuron network approach to finger key point detection in egocentric vision with mobile camera. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics. Kowloon, China, IEEE, 2015, 2944–2949
15.
Liu X R, Huang Y C, Zhang X, Jin L W. Fingertip in the eye: an attention-based method for real-time hand tracking and fingertip detection in egocentric videos. In: Communications in Computer and Information Science. Singapore: Springer Singapore, 2016, 145–154 DOI:10.1007/978-981-10-3002-4_12
16.
Wu W, Li C, Cheng Z, Zhang X, Jin L. Yolse: Egocentric fingertip detection from single RGB images. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy, IEEE, 2017, 623–630
17.
Jain V, Hebbalaguppe R. AirPen: a touchless fingertip based gestural interface for smartphones and head-mounted devices. 2019
18.
Huang Y, Liu X, Zhang X, Jin L. A pointing gesture based egocentric interaction system: Dataset, approach and application. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. Las Vegas, NV, USA, IEEE, 2016, 16–23
19.
Poupyrev I, Billinghurst M, Weghorst S, Ichikawa T. The go-go interaction technique: non-linear mapping for direct manipulation in VR. In: Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology. Seattle, Washington, USA, New York, ACM Press, 1996, 79–80 DOI:10.1145/237091.237102
20.
Bowman D A, Hodges L F. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the symposium on Interactive 3D graphics. Rhode Island, USA, 1997 DOI:10.1145/253284.253301
21.
Tomozoe Y, Machida T, Kiyokawa K, Takemura H. Unified gesture-based interaction techniques for object manipulation and navigation in a large-scale virtual environment. In: Proceedings of the IEEE Virtual Reality 2004. Chicago, IL, USA, IEEE, 2004, 259–260
22.
Kiyokawa K, Takemura H. A tunnel window and its variations: Seamless teleportation techniques in a virtual environment. In: Proceedings of the HCI International. Citeseer, Las Vegas, Nevada, USA, 2005
23.
Lee T, Hollerer T. Handy AR: Markerless inspection of augmented reality objects using fingertip tracking. In: 2007 11th IEEE International Symposium on Wearable Computers. Boston, MA, USA, IEEE, 2007, 83–90 DOI:10.1109/iswc.2007.4373785
24.
Rani S S, Dhrisya K, Ahalyadas M. Hand gesture control of virtual object in augmented reality. In: Proceedings of the International Conference Advances in Computing, Communications and Informatics. Udupi, India, IEEE, 2017, 1500–1505
25.
Bai H, Gao L, El-Sana J, Billinghurst M. Markerless 3D gesture-based interaction for handheld augmented reality interfaces. In: 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Adelaide, SA, Australia, IEEE, 2013, 1–6
26.
Song P, Yu H, Winkler S. Vision-based 3D finger interactions for mixed reality games with physics simulation. In: Proceedings of The 7th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. Singapore, New York, ACM Press, 2008, 7 DOI:10.1145/1477862.1477871
27.
Ong S K, Wang Z B. Augmented assembly technologies based on 3D bare-hand interaction. CIRP Annals, 2011, 60(1): 1–4 DOI:10.1016/j.cirp.2011.03.001
28.
Le H Q, Kim J I. An augmented reality application with hand gestures for learning 3D geometry. In: 2017 IEEE International Conference on Big Data and Smart Computing (BigComp). Jeju, South Korea, IEEE, 2017, 34–41 DOI:10.1109/BIGCOMP.2017.7881712
29.
Weichel C, Lau M, Kim D, Villar N, Gellersen H W. MixFab: a mixed-reality environment for personal fabrication. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. Toronto, Ontario, Canada, New York, ACM Press, 2014, 3855–3864 DOI:10.1145/2556288.2557090
30.
Lee J Y, Rhee G W, Seo D W. Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. The International Journal of Advanced Manufacturing Technology, 2010, 51(9/10/11/12): 1069–1082 DOI:10.1007/s00170-010-2671-x
31.
Alam M M, Rahman S M. Detection and tracking of fingertips for geometric transformation of objects in virtual environment. In: 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA). Abu Dhabi, United Arab Emirates, IEEE, 2019, 1–8 DOI:10.1109/AICCSA47632.2019.9035256
32.
Wu M Y, Ting P W, Tang Y H, Chou E T, Fu L C. Hand pose estimation in object-interaction based on deep learning for virtual reality applications. Journal of Visual Communication and Image Representation, 2020, 70: 102802 DOI:10.1016/j.jvcir.2020.102802
33.
Redmon J, Farhadi A. YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA, IEEE, 2017, 7263–7271
34.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014
35.
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, IEEE, 2016, 2818–2826
36.
Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA, IEEE, 2017, 1251–1258
37.
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L C. Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, USA, IEEE, 2018, 4510–4520
38.
Kingma D P, Ba J. Adam: a method for stochastic optimization. 2014
39.
Anton H, Rorres C. Elementary linear algebra: applications version. John Wiley & Sons, 2013
Related
1. Chongyang SUN, Weizhi NAI, Xiaoying SUN, Tactile sensitivity in ultrasonic haptics: Do different parts of hand and different rendering methods have an impact on perceptual threshold? Virtual Reality & Intelligent Hardware 2019, 1(3): 265-275
2. Egemen ERTUGRUL, Ping LI, Bin SHENG, On attaining user-friendly hand gesture interfaces to control existing GUIs Virtual Reality & Intelligent Hardware 2020, 2(2): 153-161
3. Shiwei CHENG, Qiufeng PING, Jialing WANG, Yijian CHEN, EyeGaze: Hybrid eye tracking approach for handheld mobile devices Virtual Reality & Intelligent Hardware 2022, 4(2): 173-188
4. Xiaoxiong FAN, Yun CAI, Yufei YANG, Tianxing XU, Yike Li, Songhai ZHANG, Fanglue ZHANG, Detection of scene-irrelevant head movements via eye-head coordination information Virtual Reality & Intelligent Hardware 2021, 3(6): 501-514
5. Yuanyuan SHI, Yunan LI, Xiaolong FU, Kaibin MIAO, Qiguang MIAO, Review of dynamic gesture recognition Virtual Reality & Intelligent Hardware 2021, 3(3): 183-206
6. Wanlu ZHENG, Wenming ZHENG, Yuan ZONG, Multi-scale discrepancy adversarial network for cross-corpus speech emotion recognition Virtual Reality & Intelligent Hardware 2021, 3(1): 65-75
7. Xuezhi YAN, Qiushuang WU, Xiaoying SUN, Electrostatic tactile representation in multimedia mobile terminal Virtual Reality & Intelligent Hardware 2019, 1(2): 201-218
8. Yang LI, Dong WU, Jin HUANG, Feng TIAN, Hong'an WANG, Guozhong DAI, Influence of multi-modality on moving target selection in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(3): 303-315
9. Yang LI, Jin HUANG, Feng TIAN, Hong-An WANG, Guo-Zhong DAI, Gesture interaction in virtual reality Virtual Reality & Intelligent Hardware 2019, 1(1): 84-112
10. Athirah SYAMIMI, Yiwei GONG, Ryan LIEW, VR industrial applications―A singapore perspective Virtual Reality & Intelligent Hardware 2020, 2(5): 409-420
11. Jie GUO, Dongdong WENG, Yue LIU, Qiyong CHEN, Yongtian WANG, Analysis of teenagers' preferences and concerns regarding HMDs in education Virtual Reality & Intelligent Hardware 2021, 3(5): 369-382
12. Yijun LI, Miao WANG, Derong JIN, Frank STEINICKE, Qinping ZHAO, Effects of virtual environment and self-representations on perception and physical performance in redirected jumping Virtual Reality & Intelligent Hardware 2021, 3(6): 451-469
13. Susu HUANG, Daqing QI, Jiabin YUAN, Huawei TU, Review of studies on target acquisition in virtual reality based on the crossing paradigm Virtual Reality & Intelligent Hardware 2019, 1(3): 251-264
14. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289
15. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120
16. Yuan CHANG, Guo-Ping WANG, A review on image-based rendering Virtual Reality & Intelligent Hardware 2019, 1(1): 39-54
17. Shiguang QIU, Shuntao LIU, Deshuai KONG, Qichang HE, Three-dimensional virtual-real mapping of aircraft autom-atic spray operation and online simulation monitoring Virtual Reality & Intelligent Hardware 2019, 1(6): 611-621
18. Xu PENG, Zhenyu GAO, Yitong DING, Dongfeng ZHAO, Xiaoyu CHI, Study of ghost image suppression in polarized catadioptric virtual reality optical systems Virtual Reality & Intelligent Hardware 2020, 2(1): 70-78
19. Zhiming HU, Sheng LI, Meng GAI, Temporal continuity of visual attention for future gaze prediction in immersive virtual reality Virtual Reality & Intelligent Hardware 2020, 2(2): 142-152
20. Lihui HUANG, Siti Faatihah Binte Mohd TAIB, Ryan Aung BA, Zhe An GOH, Mengshan XU, Virtual reality research and development in NTU Virtual Reality & Intelligent Hardware 2020, 2(5): 394-408
21. Stéphanie PHILIPPE, Alexis D. SOUCHET, Petros LAMERAS, Panagiotis PETRIDIS, Julien CAPORAL, Gildas COLDEBOEUF, Hadrien DUZAN, Multimodal teaching, learning and training in virtual reality: a review and case study Virtual Reality & Intelligent Hardware 2020, 2(5): 421-442
22. Jia Ming LEE, Xinxing XIA, Clemen OW, Felix CHUA, Yunqing GUAN, VEGO: A novel design towards customizable and adjustable head-mounted display for VR Virtual Reality & Intelligent Hardware 2020, 2(5): 443-453
23. Jingcheng QIAN, Yancong MA, Zhigeng PAN, Xubo YANG, Effects of Virtual-real fusion on immersion, presence, and learning performance in laboratory education Virtual Reality & Intelligent Hardware 2020, 2(6): 569-584
24. Hengwei XU, Siru LI, Wenpeng SONG, Jiajun SUN, Xinli WU, Xiaoqi WANG, Wenzhen YANG, Zhigeng PAN, Abdennour EI RHALIBI, Thermal perception method of virtual chemistry experiments Virtual Reality & Intelligent Hardware 2020, 2(4): 305-315
25. TJ MATTHEWS, Feng TIAN, Tom DOLBY, Interaction design for paediatric emergency VR training Virtual Reality & Intelligent Hardware 2020, 2(4): 330-344
26. Hongxin ZHANG, Jin ZHANG, Xue YIN, Kan ZHOU, Zhigeng PAN, Abdennour EI RHALIBI, Cloud-to-end rendering and storage management for virtual reality in experimental education Virtual Reality & Intelligent Hardware 2020, 2(4): 368-380
27. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329
28. Haoyu WANG, Jianhuang WU, A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction Virtual Reality & Intelligent Hardware 2021, 3(4): 302-314
29. Na ZHANG, Liwen TAN, Fengying LI, Bing HAN, Yifa XU, Development and application of digital assistive teaching system for anatomy Virtual Reality & Intelligent Hardware 2021, 3(4): 315-335
30. Daniel VANKOV, David JANKOVSZKY, Effects of using headset-delivered virtual reality in road safety research: A systematic review of empirical studies Virtual Reality & Intelligent Hardware 2021, 3(5): 351-368
31. Xiaolong LIU, Lili WANG, Redirected jumping in virtual scenes with alleys Virtual Reality & Intelligent Hardware 2021, 3(6): 470-483
32. Liming WANG, Xianwei CHEN, Tianyang DONG, Jing FAN, Virtual climbing: An immersive upslope walking system using passive haptics Virtual Reality & Intelligent Hardware 2021, 3(6): 435-450
33. Muhammad IRFAN, Muhammad MUNSIF, Deepdive: a learning-based approach for virtual camera in immersive contents Virtual Reality & Intelligent Hardware 2022, 4(3): 247-262
34. Dangxiao WANG, Yuan GUO, Shiyi LIU, Yuru ZHANG, Weiliang XU, Jing XIAO, Haptic display for virtual reality: progress and challenges Virtual Reality & Intelligent Hardware 2019, 1(2): 136-162
35. Aiguo SONG, Liyue FU, Multi-dimensional force sensor for haptic interaction: a review Virtual Reality & Intelligent Hardware 2019, 1(2): 121-135
36. Wenmin ZHU, Xiumin FAN, Yanxin ZHANG, Applications and research trends of digital human models in the manufacturing industry Virtual Reality & Intelligent Hardware 2019, 1(6): 558-579
37. Mohib ULLAH, Sareer Ul AMIN, Muhammad MUNSIF, Muhammad Mudassar YAMIN, Utkurbek SAFAEV, Habib KHAN, Salman KHAN, Habib ULLAH, Serious games in science education: a systematic literature review Virtual Reality & Intelligent Hardware 2022, 4(3): 189-209
38. Yuan WEI, Dongdong GUAN, Qiuchen WANG, Xiangxian LI, Yulong BIAN, Pu QIN, Yanning XU, Chenglei YANG, Virtual fire drill system supporting co-located collaboration Virtual Reality & Intelligent Hardware 2019, 1(3): 290-302
39. Haochen HU, Yue LIU, Kang YUE, Yongtian WANG, Navigation in virtual and real environment using brain computer interface:a progress report Virtual Reality & Intelligent Hardware 2022, 4(2): 89-114
40. Hayat ULLAH, Sitara AFZAL, Imran Ullah KHAN, Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey Virtual Reality & Intelligent Hardware 2022, 4(3): 223-246