Skip to Main content Skip to Navigation
Journal articles

Volume Sweeping: Learning Photoconsistency for Multi-View Shape Reconstruction

Vincent Leroy 1, 2 Jean-Sébastien Franco 2 Edmond Boyer 2
2 MORPHEO - Capture and Analysis of Shapes in Motion
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
Abstract : We propose a full study and methodology for multi-view stereo reconstruction with performance capture data. Multi-view 3D reconstruction has largely been studied with general, high resolution and high texture content inputs, where classic low-level feature extraction and matching are generally successful. However in performance capture scenarios, texture content is limited by wider angle shots resulting in smaller subject projection areas, and intrinsically low image content of casual clothing. We present a dedicated pipeline, based on a per-camera depth map sweeping strategy, analyzing in particular how recent deep network advances allow to replace classic multi-view photoconsistency functions with one that is learned. We show that learning based on a volumetric receptive field around a 3D depth candidate improves over using per-view 2D windows, giving the photoconsistency inference more visibility over local 3D correlations in viewpoint color aggrega-tion. Despite being trained on a standard dataset of scanned static objects, the proposed method is shown to generalize and significantly outperform existing approaches on performance capture data, while achieving competitive results on recent benchmarks.
Complete list of metadatas

Cited literature [52 references]  Display  Hide  Download

https://hal.inria.fr/hal-02975479
Contributor : Vincent Leroy <>
Submitted on : Thursday, October 22, 2020 - 5:02:31 PM
Last modification on : Tuesday, November 24, 2020 - 4:00:09 PM

File

IJCV2019submission.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Vincent Leroy, Jean-Sébastien Franco, Edmond Boyer. Volume Sweeping: Learning Photoconsistency for Multi-View Shape Reconstruction. International Journal of Computer Vision, Springer Verlag, 2020, ⟨10.1007/s11263-020-01377-0⟩. ⟨hal-02975479⟩

Share

Metrics

Record views

63

Files downloads

148