Skip to Main content Skip to Navigation
Conference papers

Physics-based Deep Neural Network for Augmented Reality during Liver Surgery

Abstract : In this paper we present an approach combining a finite element method and a deep neural network to learn complex elastic deformations with the objective of providing augmented reality during hep-atic surgery. Derived from the U-Net architecture, our network is built entirely from physically-based simulations of a preoperative segmenta-tion of the organ. These simulations are performed using an immersed-boundary method, which offers several numerical and practical benefits, such as not requiring boundary-conforming volume elements. We perform a quantitative assessment of the method using synthetic and ex vivo patient data. Results show that the network is capable of solving the deformed state of the organ using only a sparse partial surface displacement data and achieve similar accuracy as a FEM solution, while being about 100x faster. When applied to an ex vivo liver example, we achieve the registration in only 3 ms with a mean target registration error (TRE) of 2.9 mm.
Complete list of metadatas

Cited literature [21 references]  Display  Hide  Download
Contributor : Andrea Mendizabal <>
Submitted on : Wednesday, June 19, 2019 - 4:27:06 PM
Last modification on : Wednesday, August 19, 2020 - 11:17:53 AM


Files produced by the author(s)




Jean-Nicolas Brunet, Andrea Mendizabal, Antoine Petit, Nicolas Golse, Eric Vibert, et al.. Physics-based Deep Neural Network for Augmented Reality during Liver Surgery. MICCAI 2019 - 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, Oct 2019, Shenzhen, China. pp.8, ⟨10.1007/978-3-030-32254-0_16⟩. ⟨hal-02158862v2⟩



Record views


Files downloads