Skip to Main content Skip to Navigation
Conference papers

ImaGINator: Conditional Spatio-Temporal GAN for Video Generation

Abstract : Generating human videos based on single images entails the challenging simultaneous generation of realistic and visual appealing appearance and motion. In this context, we propose a novel conditional GAN architecture, namely ImaGINator, which given a single image, a condition (la-bel of a facial expression or action) and noise, decomposes appearance and motion in both latent and high level feature spaces, generating realistic videos. This is achieved by (i) a novel spatio-temporal fusion scheme, which generates dynamic motion, while retaining appearance throughout the full video sequence by transmitting appearance (originating from the single image) through all layers of the network. In addition, we propose (ii) a novel transposed (1+2)D convo-lution, factorizing the transposed 3D convolutional filters into separate transposed temporal and spatial components, which yields significantly gains in video quality and speed. We extensively evaluate our approach on the facial expression datasets MUG and UvA-NEMO, as well as on the action datasets NATOPS and Weizmann. We show that our approach achieves significantly better quantitative and qualitative results than the state-of-the-art.
Document type :
Conference papers
Complete list of metadatas

Cited literature [50 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02368319
Contributor : Yaohui Wang <>
Submitted on : Monday, November 18, 2019 - 2:48:50 PM
Last modification on : Monday, November 23, 2020 - 4:15:07 PM

File

WACV-2020.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02368319, version 1

Citation

Yaohui Wang, Piotr Bilinski, Francois Bremond, Antitza Dantcheva. ImaGINator: Conditional Spatio-Temporal GAN for Video Generation. WACV 2020 - Winter Conference on Applications of Computer Vision, Mar 2020, Snowmass Village, United States. ⟨hal-02368319⟩

Share

Metrics

Record views

452

Files downloads

1504