Skip to Main content Skip to Navigation
Conference papers

Automatic alignment of paintings and photographs depicting a 3D scene

Bryan Russell 1, 2, * Josef Sivic 3, 1 Jean Ponce 3, 1 Helene Dessales 4
* Corresponding author
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
2 Science and Technology Center for Visual Computing
LIENS - Laboratoire d'informatique de l'école normale supérieure, INTEL
Abstract : This paper addresses the problem of automatically align- ing historical architectural paintings with 3D models obtained using multi-view stereo technology from modern photographs. This is a challenging task because of the variations in appearance, geometry, color and texture due to environmental changes over time, the non-photorealistic nature of architectural paintings, and differences in the view- points used by the painters and photographers. Our contribution is two-fold: (i) we combine the gist descriptor [23] with the view-synthesis/retrieval of Irschara et al. [14] to obtain a coarse alignment of the painting to the 3D model by view-sensitive retrieval; (ii) we develop an ICP-like viewpoint refinement procedure, where 3D surface orientation discontinuities (folds and creases) and view-dependent occlusion boundaries are rendered from the automatically obtained and noisy 3D model in a view-dependent manner and matched to contours extracted from the paintings. We demonstrate the alignment of XIXth Century architectural watercolors of the Casa di Championnet in Pompeii with a 3D model constructed from modern photographs using the PMVS public-domain multi-view stereo software.
Document type :
Conference papers
Complete list of metadata

Cited literature [30 references]  Display  Hide  Download

https://hal.inria.fr/hal-01053879
Contributor : Josef Sivic <>
Submitted on : Sunday, August 3, 2014 - 6:55:08 PM
Last modification on : Tuesday, May 4, 2021 - 2:06:02 PM
Long-term archiving on: : Tuesday, November 25, 2014 - 11:45:28 PM

File

russell11.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01053879, version 1

Collections

Citation

Bryan Russell, Josef Sivic, Jean Ponce, Helene Dessales. Automatic alignment of paintings and photographs depicting a 3D scene. 3rd International IEEE Workshop on 3D Representation for Recognition, 2011, Barcelona, Spain. ⟨hal-01053879⟩

Share

Metrics

Record views

542

Files downloads

504