Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network

Yezi Zhao
  • Fonction : Auteur
Beibei Wang
Yanning Xu
  • Fonction : Auteur
Zheng Zeng
  • Fonction : Auteur
Lu Wang
  • Fonction : Auteur
  • PersonId : 761519
  • IdRef : 185505201

Résumé

We want to recreate spatially-varying bi-directional reflectance distribution functions (SVBRDFs) from a single image. Producing these SVBRDFs from single images will allow designers to incorporate many new materials in their virtual scenes, increasing their realism. A single image contains incomplete information about the SVBRDF, making reconstruction difficult. Existing algorithms can produce high-quality SVBRDFs with single or few input photographs using supervised deep learning. The learning step relies on a huge dataset with both input photographs and the ground truth SVBRDF maps. This is a weakness as ground truth maps are not easy to acquire. For practical use, it is also important to produce large SVBRDF maps. Existing algorithms rely on a separate texture synthesis step to generate these large maps, which leads to the loss of consistency between generated SVBRDF maps. In this paper, we address both issues simultaneously. We present an unsupervised generative adversarial neural network that addresses both SVBRDF capture from a single image and synthesis at the same time. From a low-resolution input image, we generate a large resolution SVBRDF, much larger than the input images. We train a generative adversarial network (GAN) to get SVBRDF maps, which have both a large spatial extent and detailed texels. We employ a two-stream generator that divides the training of maps into two groups (normal and roughness as one, diffuse and specular as the other) to better optimize those four maps. In the end, our method is able to generate high-quality large scale SVBRDF maps from a single input photograph with repetitive structures and provides higher quality rendering results with more details compared to the previous works. Each input for our method requires individual training, which costs about 3 hours.
Fichier principal
Vignette du fichier
paper1023_DLOnlyTrack.pdf (159.49 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03156291 , version 1 (02-03-2021)

Identifiants

Citer

Yezi Zhao, Beibei Wang, Yanning Xu, Zheng Zeng, Lu Wang, et al.. Joint SVBRDF Recovery and Synthesis From a Single Image using an Unsupervised Generative Adversarial Network. EGSR 2020 - The 31th Eurographics Symposium on Rendering (DL-only Track), Jun 2020, London / Virtual, United Kingdom. pp.1-14, ⟨10.2312/sr.20201136⟩. ⟨hal-03156291⟩
111 Consultations
7 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More