Deformable Kernel Networks for Joint Image Filtering - PaRis AI Research InstitutE Accéder directement au contenu
Article Dans Une Revue International Journal of Computer Vision Année : 2020

Deformable Kernel Networks for Joint Image Filtering

Résumé

Joint image filters are used to transfer structural details from a guidance picture used as a prior to a target image, in tasks such as enhancing spatial resolution and suppressing noise. Previous methods based on convolutional neural networks (CNNs) combine nonlinear activations of spatially-invariant kernels to estimate structural details and regress the filtering result. In this paper, we instead learn explicitly sparse and spatially-variant kernels. We propose a CNN architecture and its efficient implementation, called the deformable kernel network (DKN), that outputs sets of neighbors and the corresponding weights adaptively for each pixel. The filtering result is then computed as a weighted average. We also propose a fast version of DKN that runs about four times faster for an image of size 640 × 480. We demonstrate the effectiveness and flexibility of our models on the tasks of depth map upsampling, saliency map upsampling, cross-modality image restoration, texture removal, and semantic segmentation. In particular, we show that the weighted averaging process with sparsely sampled 3 × 3 kernels outperforms the state of the art by a significant marginin all cases.
Fichier principal
Vignette du fichier
main.pdf (9.86 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01857016 , version 1 (14-08-2018)
hal-01857016 , version 2 (10-10-2018)
hal-01857016 , version 3 (04-03-2019)
hal-01857016 , version 4 (16-10-2019)
hal-01857016 , version 5 (15-01-2020)
hal-01857016 , version 6 (14-09-2020)
hal-01857016 , version 7 (21-10-2020)

Identifiants

Citer

Beomjun Kim, Jean Ponce, Bumsub Ham. Deformable Kernel Networks for Joint Image Filtering. International Journal of Computer Vision, 2020, ⟨10.1007/s11263-020-01386-z⟩. ⟨hal-01857016v7⟩
1105 Consultations
1010 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More