Wide baseline image-based rendering based on shape prior regularisation

Abstract : We consider the synthesis of intermediate views of an object captured by two widely spaced and calibrated cameras. This problem is challenging because foreshortening effects and occlusions induce significant differences between the reference images when the cameras are far apart. That makes the association or disappearance/appearance of their pixels difficult to estimate. Our main contribution lies in disambiguating this illposed problem by making the interpolated views consistent with a plausible transformation of the object silhouette between the reference views. This plausible transformation is derived from an object-specific prior that consists of a nonlinear shape manifold learned from multiple previous observations of this object by the two reference cameras. The prior is used to estimate how the epipolar silhouette segments observed in the reference views evolve between those views. This information directly supports the definition of epipolar silhouette segments in the intermediate views, and the synthesis of textures in those segments. It permits to reconstruct the Epipolar Plane Images (EPIs) and the continuum of views associated with the Epipolar Plane Image Volume, obtained by aggregating the EPIs. Experiments on synthetic and natural images show that our method preserves the object topology in intermediate views and deals effectively with the selfoccluded regions and the severe foreshortening effect associated with wide-baseline camera configurations.
Type de document :
Article dans une revue
IEEE Transactions on Image Processing, Institute of Electrical and Electronics Engineers, 2017
Liste complète des métadonnées

https://hal.inria.fr/hal-01575559
Contributeur : Thomas Maugey <>
Soumis le : lundi 21 août 2017 - 10:47:03
Dernière modification le : mercredi 16 mai 2018 - 11:24:14

Identifiants

  • HAL Id : hal-01575559, version 1

Citation

Cédric Verleysen, Thomas Maugey, Christophe De Vleeschouwer, Pascal Frossard. Wide baseline image-based rendering based on shape prior regularisation. IEEE Transactions on Image Processing, Institute of Electrical and Electronics Engineers, 2017. 〈hal-01575559〉

Partager

Métriques

Consultations de la notice

280