Artificial Intelligence in Surgery: Augmented Reality provides Support in the Operating Room
A team from the Embedded Intelligence research department at DFKI is working with doctors at Klinikum Saarbrücken to to develop new methods for AR-based assistance systems to support operations on the pancreas.
24 Nov, 2023. 1 minute read
Lead surgeon using application pre-operatively for planning
This article was first published on
www.dfki.deClinical surgery grequires great precision, years of experience, specialised knowledge and the highest level of concentration on the part of the operating surgeons. A current research project aims to provide them with an interactive tool to improve precision and performance. A team from the Embedded Intelligence research department at DFKI is working with doctors at Klinikum Saarbrücken to to develop new methods for AR-based assistance systems to support operations on the pancreas.
The assistance system was developed specifically for the visualisation of vital anatomical structures during the operation on the patient, using a mobile AR system. With the help of special glasses (HoloLens), the information is displayed in the surgeon's real field of vision. Anatomical features are used as reference points to position the 3D model precisely on the patient.
Evaluation in clinical practice
The DFKI researchers worked closely with the team of Gregor Stavrou, Chief of General and Visceral Surgery, Department of Surgical Oncology at the Saarbrücken Klinikum. Thanks to the cooperation with the hospital, the applications can be evaluated in real surgery and important research data can be collected through observations during the operation and post-hoc interviews after the procedures. "In this ecologically highly valid environment, such insights into practice are invaluable," says project manager Hamraz Javaheri, Researcher at DFKI's Embedded Intelligence Department.
The prototype was tested in a patient study and proved successful in registering and visualizing the 3D reconstructed model of the patient's vital anatomical structure and thereby supporting surgeons in decision making during surgery. The assistance system has already been used successfully by Dr. Gregor Stavrou during an extremely difficult operation to remove a malignant tumor using the so-called "Appleby procedure", in which a main artery had to be rerouted, among other things. It also has great potential for medical training and surgery preparation.
HumanE-AI-Net - human-centered AI in the European network
The project is being carried out within the framework of the European network of excellence HumanAINet (and is funded by it). The system is currently being successively improved. The valuable findings from studies done by DFKI researchers will be shared with the research community in upcoming scientific articles.