Surgery: researchers are working on VR training with haptic feedback

Contents

Scientists are currently working on a virtual training environment for surgical operations – VIRTOSHA (authoring tool for virtual reality training simulations of osteosynthesis with haptic feedback and tissue simulation). Haptic feedback is to play a central role in the three-year research project. This is according to a statement from TH Köln, which is developing an AI-based simulation system for this purpose. The University Hospital Bonn (UKB), Mindport GmbH and Haption GmbH are involved in the project.

Making drilling and screwing tangible

Virtual patients are also used to prepare surgeons for operations. However, according to TH Köln, previous options lack "tangible feedback via fingers or hands", i.e. haptic feedback. "Prospective surgeons need to experience what operations feel like and train their fine motor skills during their training [...] We are designing a drilling and screwing simulation with haptic arms, a robot with gripper arms, and will implement the characteristics of different types of tissue. This will make it possible to feel the resistance when drilling and screwing into the bone and train interventions realistically," says Kristian Welle, senior consultant at the UKB.

The aim is to record and analyze data from real interventions – how tissue and implants behave in relation to the surgical tools –. Among other things, this should also optimize the force feedback of the "haptic arms". The aim is to then combine different procedures and avatars, create exercises and train the surgical participants to work together with other specialist groups.

"We use software to design various scenarios, whereby tissue and bone properties can be adapted to simulate different patients. A realistic representation of the tissues and bones as well as the precise integration of hand movements are crucial for the real sensation in the VR environment," says Prof. Björn Krüger, head of the "Personalized Digital Health and Telemedicine" research group at the UKB's Department of Epileptology.

Software for creating your own content

According to Mindport founder David Lähner, what is new about the approach is that surgeons "can create new content and update existing content themselves using an integrated authoring tool [...]. This means that many more different cases can be covered," explains Lähner. Another vision is that in the future, surgery can be practiced on a digital twin of the patient. It should also be possible to better understand the movements of the hand based on sensor data and additional data sources such as cameras in the future.

"All components are connected via a data hub that manages all processes and structures and serves as the system's interface," says the project page. The team intends to publish the progress of the project on its website. The project is being funded with 1.5 million euros as part of the Gesünder.IN.NRW innovation competition.

VR training in other areas

Together with Frankfurt University Hospital, PatientZero Games GmbH has also developed a voice-controlled online 3D training environment for anaesthetists. The software simulates realistic complications that can occur during anesthesia. In this environment, the anesthetist can administer anesthesia, adjust ventilation and react to possible complications. The training can be carried out both in VR and on a conventional monitor.

The use of VR is also conceivable in the treatment of anorexia, as the results of a study published in the journal "Psychotherapy and Psychosomatics" show. A team of scientists from the Department of Psychosomatic Medicine and Psychotherapy at Tübingen University Hospital and the Max Planck Institute for Intelligent Systems has developed a VR environment to help people with anorexia to reduce their fear of gaining weight. In this environment, patients can imagine what a healthy body weight feels like – both from a first-person perspective and in a virtual mirror. The VR environment developed uses a digital body model based on thousands of body scans that can be customized for each patient.

(mack)