Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board

Articles

    Editorial

  • Virtual reality and augmented reality in medical simulation
    Aimin HAO

    DOI:10.3724/SP.J.2096-5796.2021.03.04

    2021, 3(4) : 1-1

    PDF (6) HTML (113)
  • Article

  • Orthodontic simulation system with force feedback for training complete bracket placement procedures
    Fan YE, Luwei LIU, Bin YAN, Xiaohan ZHAO, Aimin HAO

    DOI:10.1016/j.vrih.2021.08.001

    2021, 3(4) : 261-273

    Abstract(142) PDF (3) HTML (132)
    Background
    A virtual system that simulates the complete process of orthodontic bracket placement can be used for pre-clinical skill training to help students gain confidence by performing the required tasks on a virtual patient.
    Methods
    The hardware for the virtual simulation system is built using two force feedback devices to support bi-manual force feedback operation. A 3D mouse is used to adjust the position of the virtual patient. A multi-threaded computational methodology is adopted to satisfy the requirements of the frame rate. The computation threads mainly consist of the haptic thread running at a frequency of >1000Hz and the graphic thread at >30Hz. The graphic thread allows the graphics engine to effectively display the visual effects of biofilm removal and acid erosion through texture mapping. Using the haptic thread, the physics engine adopts the hierarchy octree collision-detection algorithm to simulate the multi-point and multi-region interaction between the tools and the virtual environment. Its high efficiency guarantees that the time cost can be controlled within 1 ms. The physics engine also performs collision detection between the tools and particles, making it possible to simulate paint and removal of colloids. The surface-contact constraints are defined in the system; this ensures that the bracket will not divorce from or embed into the tooth during the adjustment of the bracket. Therefore, the simulated adjustment is more realistic and natural.
    Results
    A virtual system to simulate the complete process of orthodontic bracket bonding was developed. In addition to bracket bonding and adjustment, the system simulates the necessary auxiliary steps such as smearing, acid etching, and washing. Furthermore, the system supports personalized case training.
    Conclusions
    The system provides a new method for students to practice orthodontic skills.
  • Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems
    Yonghang TAI, Junsheng SHI, Junjun PAN, Aimin HAO, Victor CHANG

    DOI:10.1016/j.vrih.2021.08.002

    2021, 3(4) : 274-286

    Abstract(117) PDF (4) HTML (126)
    Background
    Compared with traditional thoracotomy, video-assisted thoracoscopic surgery (VATS) has less minor trauma, faster recovery, higher patient compliance, but higher requirements for surgeons. Virtual surgery training simulation systems are important and have been widely used in Europe and America. Augmented reality (AR) in surgical training simulation systems significantly improve the training effect of virtual surgical training, although AR technology is still in its initial stage. Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice.
    Methods
    This study proposed an immersive AR lobectomy within a thoracoscope surgery training system, using visual and haptic modeling to study the potential benefits of this critical technology. The content included immersive AR visual rendering, based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling. Furthermore, we designed an AR haptic rendering systems, whose model architecture consisted of multi-touch interaction points, including kinesthetic and pressure-sensitive points. Finally, based on the above theoretical research, we developed an AR interactive VATS surgical training platform.
    Results
    Twenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system. Face, content, and construct validation methods were used to assess the tactile sense, visual sense, scene authenticity, and simulator performance.
    Conclusions
    The results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time. The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices.
  • Topological distance-constrained feature descriptor learning model for vessel matching in coronary angiographies
    Xiaojiao SONG, Jianjun ZHU, Jingfan FAN, Danni AI, Jian YANG

    DOI:10.1016/j.vrih.2021.08.003

    2021, 3(4) : 287-301

    Abstract(154) PDF (1) HTML (112)
    Background
    Feature matching technology is vital to establish the association between virtual and real objects in virtual reality and augmented reality systems. Specifically, it provides them with the ability to match a dynamic scene. Many image matching methods, of which most are deep learning-based, have been proposed over the past few decades. However, vessel fracture, stenosis, artifacts, high background noise, and uneven vessel gray-scale make vessel matching in coronary angiography extremely difficult. Traditional matching methods perform poorly in this regard.
    Methods
    In this study, a topological distance-constrained feature descriptor learning model is proposed. This model regards the topology of the vasculature as the connection relationship of the centerline. The topological distance combines the geodesic distance between the input patches and constrains the descriptor network by maximizing the feature difference between connected and unconnected patches to obtain more useful potential feature relationships.
    Results
    Matching patches of different sequences of angiographic images are generated for the experiments. The matching accuracy and stability of the proposed method is superior to those of the existing models.
    Conclusions
    The proposed method solves the problem of matching coronary angiographies by generating a topological distance-constrained feature descriptor.
  • A virtual reality based surgical skills training simulator for catheter ablation with real-time and robust interaction
    Haoyu WANG, Jianhuang WU

    DOI:10.1016/j.vrih.2021.08.004

    2021, 3(4) : 302-314

    Abstract(136) PDF (1) HTML (119)
    Background
    Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia that can cause severe heart problems. Catheter ablation is one of the most ideal procedures for the treatment of AF. Physicians qualified to perform this procedure need to be highly skilled in manipulating the relevant surgical devices. This study proposes an interactive surgical simulator with high fidelity to facilitate efficient training and low-cost medical education.
    Methods
    We used a shared centerline model to simulate the interaction between multiple surgical devices. An improved adaptive deviation-feedback approach is proposed to accelerate the convergence of each iteration. The periodical beating of the human heart was also simulated in real time using the position-based dynamics (PBD) framework to achieve higher fidelity. We then present a novel method for handling the interaction between the devices and the beating heart mesh model. Experiments were conducted in a homemade simulator prototype to evaluate the robustness, performance, and flexibility of the proposed method. Preliminary evaluation of the simulator was performed by medical students, residents, and surgeons.
    Results
    The interaction between surgical devices, static vascular meshes, and beating heart mesh was stably simulated in a frame rate suitable for interaction.
    Conclusion
    Our simulator is capable of simulating the procedure of catheter ablation with high fidelity and provides immersive visual experiences and haptic feedback.
  • Development and application of digital assistive teaching system for anatomy
    Na ZHANG, Liwen TAN, Fengying LI, Bing HAN, Yifa XU

    DOI:10.1016/j.vrih.2021.08.005

    2021, 3(4) : 315-335

    Abstract(126) PDF (4) HTML (110)
    Background
    Anatomy is a required course for all medicine-related industries. In recent decades, the teaching quality and effect of anatomy have been compromised by factors including a decrease in human body specimens, dampened enthusiasm for the discipline, reduced teaching hours of anatomy, scale expansion of medical education, and obstacles in performing field autopsies and observations.
    Methods
    Based on China's digitalized visible human research achievements, this article extracts the boundary information of anatomic structures from tomographic images, constructs three-dimensional (3D) digital anatomical models with authentic texture information, and develops an anatomy assistive teaching system for teachers and students based on the knowledge points of anatomy, to meet the anatomy teaching requirements of different majors at various levels.
    Results
    This scientific, complete, and holistic system has produced over 6000 3D digital anatomical models, 5000 anatomy knowledge points, 50 anatomical operation videos, and 150 micro demonstration classes, with teaching contents for different majors and levels, such as systematic anatomy, topographic anatomy, sectional anatomy, anatomy of motion, and virtual anatomical operation table. Ranging from network terminals, desktops, touchscreen 3D displays, desktops, and projection 3D volumetric displays to augmented reality, its diversified interactive forms meet the requirements for a learning environment in different settings.
    Conclusions
    With multiple teaching and learning links covered, such as teaching environment, teaching resources, instructional slides, autonomous learning, and learning effect evaluation, this novel teaching system serves as a vital component and a necessary resource in anatomy teaching and functions as an important supplement to traditional anatomy teaching. Applied and promoted in most medical colleges and schools in China, this system has been recognized and approved by anatomy teachers and students, and plays a positive role in guaranteeing the effect and quality of anatomy teaching.
  • A marching cube algorithm based on edge growth
    Xin WANG, Su GAO, Monan WANG, Zhenghua DUAN

    DOI:10.1016/j.vrih.2021.08.006

    2021, 3(4) : 336-349

    Abstract(109) PDF (1) HTML (97)
    The marching cube algorithm is currently one of the most popular three-dimensional (3D) reconstruction surface rendering algorithms. It forms cube voxels based on an input image and then uses 15 basic topological configurations to extract isosurfaces from the voxels. The algorithm processes each cube voxel in a traversal-based manner, but it does not consider the relationship between the isosurfaces in adjacent cubes. Owing to ambiguity, the final reconstructed model may have holes. In this paper, we propose a marching cube algorithm based on edge growth. The algorithm first extracts seed triangles, grows these seed triangles, and then reconstructs the entire 3D model. According to the position of the growth edge, we propose 17 topological configurations with isosurfaces. The reconstruction results showed that the algorithm can reconstruct the 3D model well. When only the main contour of the 3D model is required, the algorithm performs well. In addition, when there are multiple scattered parts in the data, the algorithm can extract only the 3D contours of the parts connected to the seed by setting the region selected based on the seed.
  • Review

  • Review of dynamic gesture recognition
    Yuanyuan SHI, Yunan LI, Xiaolong FU, Kaibin MIAO, Qiguang MIAO

    DOI:10.1016/j.vrih.2021.05.001

    2021, 3(3) : 183-206

    Abstract(362) PDF (12) HTML (335)
    In recent years, gesture recognition has been widely used in the fields of intelligent driving, virtual reality, and human-computer interaction. With the development of artificial intelligence, deep learning has achieved remarkable success in computer vision. To help researchers better understanding the development status of gesture recognition in video, this article provides a detailed survey of the latest developments in gesture recognition technology for videos based on deep learning. The reviewed methods are broadly categorized into three groups based on the type of neural networks used for recognition: two-stream convolutional neural networks, 3D convolutional neural networks, and Long-short Term Memory (LSTM) networks. In this review, we discuss the advantages and limitations of existing technologies, focusing on the feature extraction method of the spatiotemporal structure information in a video sequence, and consider future research directions.

Aims & Scope

Virtual Reality & Intelligent Hardware (VRIH) is an open access journal that aims to showcase and promote distinguished research in the field of virtual reality and intelligent hardware. It provides a global publishing and academic exchange platform for researchers, professionals and industry practitioners. The journal offers high-quality single-blind peer review and is published bimonthly in English.

Special Issues

  • Virtual reality and augmented reality in medical simulation

    169 Browse

    Virtual reality and augmented reality in medical simulation

    Intro:Virtual reality/augmented reality (VR/AR) technologies have been widely used in medical fields, such ...

    2021 Vol. 3 No. 4

  • Hand and gesture

    280 Browse

    Hand and gesture

    Intro:Hands play an important role in our daily life. We use our hands for manipulation in working, emphasi...

    2021 Vol. 3 No. 3

  • Simulation and interaction of fluid and solid dynamics

    457 Browse

    Simulation and interaction of fluid and solid dynamics

    Intro:Fluid and solid simulation is to generate a realistic simulation of fluids and solids, in particular ...

    2021 Vol. 3 No. 2

  • Emotion recognition for human-computer interaction

    598 Browse

    Emotion recognition for human-computer interaction

    Intro:Emotion recognition is to quantify,describe and recognize different emotional states through thebehav...

    2021 Vol. 3 No. 1

  • VR/AR research and commercial applications in Singapore

    585 Browse

    VR/AR research and commercial applications in Singapore

    Intro:The COVID-19 pandemic has virtually changed every aspect of our lives and society globally. This spec...

    2020 Vol. 2 No. 5

  • VR and experiment simulation

    592 Browse

    VR and experiment simulation

    Intro:In this special issue,the theme is on the application of VR/AR to experiments and training.We include...

    2020 Vol. 2 No. 4