Skip to Main content Skip to Navigation
Journal articles

Bilinear image translation for temporal analysis of photo collections

Abstract : We propose an approach for analyzing unpaired visual data annotated with time stamps by generating how images would have looked like if they were from different times. To isolate and transfer time dependent appearance variations, we introduce a new trainable bilinear factor separation module. We analyze its relation to classical factored representations [1] and concatenation-based auto-encoders [2]. We demonstrate this new module has clear advantages compared to standard concatenation when used in a bottleneck encoder-decoder convolutional neural network architecture. We also show that it can be inserted in a recent adversarial image translation architecture [3], enabling the image transformation to multiple different target time periods using a single network. We apply our model to a challenging collection of more than 13,000 cars manufactured between 1920 and 2000 [4] and a dataset of high school yearbook portraits from 1930 to 2009 [5]. This allows us, for a given new input image, to generate a "history-lapse video" revealing changes over time by simply varying the target year. We show that by analyzing the generated history-lapse videos we can identify object deformations across time, extracting interesting changes in visual style over decades.
Complete list of metadata

Cited literature [71 references]  Display  Hide  Download

https://hal.inria.fr/hal-02452091
Contributor : Josef Sivic <>
Submitted on : Thursday, January 23, 2020 - 1:15:30 PM
Last modification on : Tuesday, May 4, 2021 - 2:06:03 PM
Long-term archiving on: : Friday, April 24, 2020 - 2:29:18 PM

File

sub2018.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Théophile Dalens, Mathieu Aubry, Josef Sivic. Bilinear image translation for temporal analysis of photo collections. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2019, ⟨10.1109/TPAMI.2019.2950317⟩. ⟨hal-02452091⟩

Share

Metrics

Record views

190

Files downloads

1268