Deep Bilateral Learning for Real-Time Image Enhancement - Archive ouverte HAL Access content directly
Journal Articles ACM Transactions on Graphics Year : 2017

Deep Bilateral Learning for Real-Time Image Enhancement

(1) , (2) , (2) , (2) , (1, 3)
1
2
3

Abstract

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images , we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.
Fichier principal
Vignette du fichier
1707.02880 (1).pdf (5.57 Mo) Télécharger le fichier
Loading...

Dates and versions

hal-01676188 , version 1 (05-01-2018)

Identifiers

Cite

Michaël Gharbi, Jiawen Chen, Jonathan T Barron, Samuel W Hasinoff, Frédo Durand. Deep Bilateral Learning for Real-Time Image Enhancement. ACM Transactions on Graphics, 2017, 36 (4), pp.1 - 12. ⟨10.1145/3072959.3073592⟩. ⟨hal-01676188⟩
355 View
608 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More