Skip to Main content Skip to Navigation
Journal articles

Deep Bilateral Learning for Real-Time Image Enhancement

Abstract : Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images , we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.
Document type :
Journal articles
Complete list of metadatas

Cited literature [44 references]  Display  Hide  Download

https://hal.inria.fr/hal-01676188
Contributor : Team Reves <>
Submitted on : Friday, January 5, 2018 - 1:37:18 PM
Last modification on : Wednesday, September 18, 2019 - 4:36:03 PM
Long-term archiving on: : Wednesday, May 23, 2018 - 2:00:32 PM

Identifiers

Collections

Citation

Michaël Gharbi, Jiawen Chen, Jonathan Barron, Samuel Hasinoff, Frédo Durand. Deep Bilateral Learning for Real-Time Image Enhancement. ACM Transactions on Graphics, Association for Computing Machinery, 2017, 36 (4), pp.1 - 12. ⟨10.1145/3072959.3073592⟩. ⟨hal-01676188⟩

Share

Metrics

Record views

414

Files downloads

669