Monocular 3D Reconstruction and Augmentation of Elastic Surfaces with Self-occlusion Handling

Nazim Haouchine 1 Jeremie Dequidt 2, 3 Marie-Odile Berger 4 Stephane Cotin 1
2 DEFROST - Deformable Robots Simulation Team
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
4 MAGRIT - Visual Augmentation of Complex Environments
Inria Nancy - Grand Est, LORIA - ALGO - Department of Algorithms, Computation, Image and Geometry
Abstract : This paper focuses on the 3D shape recovery and augmented reality on elastic objects with self-occlusions handling, using only single view images. Shape recovery from a monocular video sequence is an underconstrained problem and many approaches have been proposed to enforce constraints and resolve the ambiguities. State-of-the art solutions enforce smoothness or geometric constraints, consider specific deformation properties such as inextensibility or resort to shading constraints. However, few of them can handle properly large elastic deformations. We propose in this paper a real-time method that uses a mechanical model and able to handle highly elastic objects. The problem is formulated as an energy minimization problem accounting for a non-linear elastic model constrained by external image points acquired from a monocular camera. This method prevents us from formulating restrictive assumptions and specific constraint terms in the minimization. In addition, we propose to handle self-occluded regions thanks to the ability of mechanical models to provide appropriate predictions of the shape. Our method is compared to existing techniques with experiments conducted on computer-generated and real data that show the effectiveness of recovering and augmenting 3D elastic objects. Additionally, experiments in the context of minimally invasive liver surgery are also provided and results on deformations with the presence of self-occlusions are exposed.
Document type :
Journal articles
Complete list of metadatas

Cited literature [63 references]  Display  Hide  Download
Contributor : Nazim Haouchine <>
Submitted on : Sunday, August 23, 2015 - 12:01:47 PM
Last modification on : Wednesday, April 3, 2019 - 1:22:59 AM
Long-term archiving on : Wednesday, April 26, 2017 - 10:21:04 AM


Files produced by the author(s)



Nazim Haouchine, Jeremie Dequidt, Marie-Odile Berger, Stephane Cotin. Monocular 3D Reconstruction and Augmentation of Elastic Surfaces with Self-occlusion Handling. IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2015, pp.14. ⟨10.1109/TVCG.2015.2452905⟩. ⟨hal-01186011⟩



Record views


Files downloads