Automatic alignment of paintings and photographs depicting a 3D scene

Bryan Russell 1, 2, * Josef Sivic 3, 1 Jean Ponce 3, 1 Helene Dessales 4
* Auteur correspondant
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
2 Science and Technology Center for Visual Computing
LIENS - Laboratoire d'informatique de l'école normale supérieure, INTEL
Abstract : This paper addresses the problem of automatically align- ing historical architectural paintings with 3D models obtained using multi-view stereo technology from modern photographs. This is a challenging task because of the variations in appearance, geometry, color and texture due to environmental changes over time, the non-photorealistic nature of architectural paintings, and differences in the view- points used by the painters and photographers. Our contribution is two-fold: (i) we combine the gist descriptor [23] with the view-synthesis/retrieval of Irschara et al. [14] to obtain a coarse alignment of the painting to the 3D model by view-sensitive retrieval; (ii) we develop an ICP-like viewpoint refinement procedure, where 3D surface orientation discontinuities (folds and creases) and view-dependent occlusion boundaries are rendered from the automatically obtained and noisy 3D model in a view-dependent manner and matched to contours extracted from the paintings. We demonstrate the alignment of XIXth Century architectural watercolors of the Casa di Championnet in Pompeii with a 3D model constructed from modern photographs using the PMVS public-domain multi-view stereo software.
Type de document :
Communication dans un congrès
3rd International IEEE Workshop on 3D Representation for Recognition, 2011, Barcelona, Spain. 2011
Liste complète des métadonnées

Littérature citée [30 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01053879
Contributeur : Josef Sivic <>
Soumis le : dimanche 3 août 2014 - 18:55:08
Dernière modification le : jeudi 11 janvier 2018 - 06:27:14
Document(s) archivé(s) le : mardi 25 novembre 2014 - 23:45:28

Fichier

russell11.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01053879, version 1

Collections

Citation

Bryan Russell, Josef Sivic, Jean Ponce, Helene Dessales. Automatic alignment of paintings and photographs depicting a 3D scene. 3rd International IEEE Workshop on 3D Representation for Recognition, 2011, Barcelona, Spain. 2011. 〈hal-01053879〉

Partager

Métriques

Consultations de la notice

343

Téléchargements de fichiers

121