MesoGAN: Generative Neural Reflectance Shells - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Computer Graphics Forum Année : 2023

MesoGAN: Generative Neural Reflectance Shells

J Novák
F Rousselle
J Granskog
M Aittala
R Ramamoorthi

Résumé

We introduce MesoGAN, a model for generative 3D neural textures. This new graphics primitive represents mesoscale appearance by combining the strengths of generative adversarial networks (StyleGAN) and volumetric neural field rendering. The primitive can be applied to surfaces as a neural reflectance shell; a thin volumetric layer above the surface with appearance parameters defined by a neural network. To construct the neural shell, we first generate a 2D feature texture using StyleGAN with carefully randomized Fourier features to support arbitrarily sized textures without repeating artifacts. We augment the 2D feature texture with a learned height feature, which aids the neural field renderer in producing volumetric parameters from the 2D texture. To facilitate filtering, and to enable end-to-end training within memory constraints of current hardware, we utilize a hierarchical texturing approach and train our model on multi-scale synthetic datasets of 3D mesoscale structures. We propose one possible approach for conditioning MesoGAN on artistic parameters (e.g., fiber length, density of strands, lighting direction) and demonstrate and discuss integration into physically based renderers.
Fichier principal
Vignette du fichier
mesogan_EG.pdf (74.83 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04105889 , version 1 (25-05-2023)

Identifiants

Citer

S Diolatzis, J Novák, F Rousselle, J Granskog, M Aittala, et al.. MesoGAN: Generative Neural Reflectance Shells. Computer Graphics Forum, 2023, ⟨10.1111/cgf.14846⟩. ⟨hal-04105889⟩
61 Consultations
5 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More