Surgical robot learns from videos to perform surgical procedures
A surgical robot learns surgical procedures from videos of operations. Researchers assume that it will also be able to learn and perform more complex procedures.
The specially trained "Da Vinci" robot sutures an artificial wound independently.
(Image: Ji Woong Brian Kim u. a.)
Scientists at John Hopkins University and Stanford University have trained a "Da Vinci" surgical robot to perform simple surgical procedures independently using videos of operations on humans. The researchers presented the results of their research work at the Conference on Robot Learning in Munich.
The training material for the Da Vinci Surgical Robot is based on hundreds of videos recorded by wrist cameras during surgical procedures performed on humans by the robots. The videos are used and archived by surgeons for post-operative analysis. Around 50,000 surgeons worldwide have been trained on the "Da Vinci" system. Around 7000 "Da Vinci" robots are in use. This also explains the large number of videos available for training.
Learning by imitation
The scientists aimed to teach the robot three basic surgical skills: manipulating a needle, lifting body tissue and suturing a wound.
"All we need is an image input, and then this AI system finds the right action," says Ji Woong "Brian" Kim, lead author of the scientific paper "Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks" (PDF). "We found that after just a few hundred demos, the model is able to learn the procedure and generalize to new environments that it is not yet familiar with."
The scientists used imitation learning to teach the system to perform simple surgical procedures. However, this was not easy because "Da Vinci" robots are notoriously imprecise. However, the researchers were able to solve this problem by training the model to perform relative rather than absolute movements. The latter are too imprecise.
The model also learned things that it had not been explicitly taught. For example, the robot can pick up a dropped needle on its own. Some videos of the robot can be seen on the project's GitHub page.
The AI model can be used to quickly train a surgical robot. This eliminates the need for time-consuming programming. It is also possible to teach the robot more complex surgical tasks, such as complete operations, say the scientists.
(olb)