Skip to Main content Skip to Navigation
Journal articles

Point-Based Neural Rendering with Per-View Optimization

Abstract : There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS, but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel view synthesis. A key element of our approach is a differentiable point-based splatting pipeline, based on our bi-directional Elliptical Weighted Average solution. To further improve quality and efficiency of our point-based method, we introduce a probabilistic depth test and efficient camera selection. We use these elements together in our neural renderer, allowing us to achieve a good compromise between quality and speed. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel view synthesis.
Document type :
Journal articles
Complete list of metadata

https://hal.inria.fr/hal-03268140
Contributor : Team Reves Connect in order to contact the contributor
Submitted on : Monday, June 28, 2021 - 2:00:19 PM
Last modification on : Saturday, June 25, 2022 - 11:50:57 PM

File

KopanasPointBasedNeuralRenderi...
Files produced by the author(s)

Identifiers

Collections

Citation

Georgios Kopanas, Julien Philip, Thomas Leimkühler, George Drettakis. Point-Based Neural Rendering with Per-View Optimization. Computer Graphics Forum, Wiley, 2021, 40 (4), ⟨10.1111/cgf.14339⟩. ⟨hal-03268140v3⟩

Share

Metrics

Record views

113

Files downloads

112