Skip to Main content Skip to Navigation
Conference papers

Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network

Yezi Zhao 1 Beibei Wang 2 Yanning Xu 1 Zheng Zeng 1 Lu Wang 1 Nicolas Holzschuch 3
3 MAVERICK - Models and Algorithms for Visualization and Rendering
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
Abstract : We want to recreate spatially-varying bi-directional reflectance distribution functions (SVBRDFs) from a single image. Producing these SVBRDFs from single images will allow designers to incorporate many new materials in their virtual scenes, increasing their realism. A single image contains incomplete information about the SVBRDF, making reconstruction difficult. Existing algorithms can produce high-quality SVBRDFs with single or few input photographs using supervised deep learning. The learning step relies on a huge dataset with both input photographs and the ground truth SVBRDF maps. This is a weakness as ground truth maps are not easy to acquire. For practical use, it is also important to produce large SVBRDF maps. Existing algorithms rely on a separate texture synthesis step to generate these large maps, which leads to the loss of consistency between generated SVBRDF maps. In this paper, we address both issues simultaneously. We present an unsupervised generative adversarial neural network that addresses both SVBRDF capture from a single image and synthesis at the same time. From a low-resolution input image, we generate a large resolution SVBRDF, much larger than the input images. We train a generative adversarial network (GAN) to get SVBRDF maps, which have both a large spatial extent and detailed texels. We employ a two-stream generator that divides the training of maps into two groups (normal and roughness as one, diffuse and specular as the other) to better optimize those four maps. In the end, our method is able to generate high-quality large scale SVBRDF maps from a single input photograph with repetitive structures and provides higher quality rendering results with more details compared to the previous works. Each input for our method requires individual training, which costs about 3 hours.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03156291
Contributor : Beibei Wang <>
Submitted on : Tuesday, March 2, 2021 - 1:28:51 PM
Last modification on : Wednesday, March 10, 2021 - 3:04:06 PM

File

paper1023_DLOnlyTrack.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Yezi Zhao, Beibei Wang, Yanning Xu, Zheng Zeng, Lu Wang, et al.. Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network. EGSR 2020 - The 31th Eurographics Symposium on Rendering (DL-only Track), Jun 2020, London / Virtual, United Kingdom. ⟨10.2312/sr.20201136⟩. ⟨hal-03156291⟩

Share

Metrics

Record views

43

Files downloads

229