Neural Point Catacaustics for Novel-View Synthesis of Reflections - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue ACM Transactions on Graphics Année : 2022

Neural Point Catacaustics for Novel-View Synthesis of Reflections

Résumé

View-dependent effects such as reflections pose a substantial challenge for image-based and neural rendering algorithms. Above all, curved reflectors are particularly hard, as they lead to highly non-linear reflection flows as the camera moves. We introduce a new point-based representation to compute Neural Point Catacaustics allowing novel-view synthesis of scenes with curved reflectors, from a set of casually-captured input photos. At the core of our method is a neural warp field that models catacaustic trajectories of reflections, so complex specular effects can be rendered using efficient point splatting in conjunction with a neural renderer. One of our key contributions is the explicit representation of reflections with a reflection point cloud which is displaced by the neural warp field, and a primary point cloud which is optimized to represent the rest of the scene. After a short manual annotation step, our approach allows interactive high-quality renderings of novel views with accurate reflection flow. Additionally, the explicit representation of reflection flow supports several forms of scene manipulation in captured scenes, such as reflection editing, cloning of specular objects, reflection tracking across views, and comfortable stereo viewing. We provide the source code and other supplemental material on https://repo-sam.inria.fr/fungraph/neural_catacaustics/
Fichier principal
Vignette du fichier
neural-catacaustics.pdf (116.57 Mo) Télécharger le fichier

Dates et versions

hal-03799441 , version 1 (06-10-2022)

Identifiants

Citer

Georgios Kopanas, Thomas Leimkühler, Gilles Rainer, Clément Jambon, George Drettakis. Neural Point Catacaustics for Novel-View Synthesis of Reflections. ACM Transactions on Graphics, 2022, 41 (6), pp.Article 201. ⟨10.1145/3550454.3555497⟩. ⟨hal-03799441⟩
102 Consultations
9 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More