Cross-Paced Representation Learning with Partial Curricula for Sketch-based Image Retrieval

Abstract : In this paper we address the problem of learning robust cross-domain representations for sketch-based image retrieval (SBIR). While most SBIR approaches focus on extracting low-and mid-level descriptors for direct feature matching, recent works have shown the benefit of learning coupled feature representations to describe data from two related sources. However, cross-domain representation learning methods are typically cast into non-convex minimization problems that are difficult to optimize, leading to unsatisfactory performance. Inspired by self-paced learning, a learning methodology designed to overcome convergence issues related to local optima by exploiting the samples in a meaningful order (i.e. easy to hard), we introduce the cross-paced partial curriculum learning (CPPCL) framework. Compared with existing self-paced learning methods which only consider a single modality and cannot deal with prior knowledge, CPPCL is specifically designed to assess the learning pace by jointly handling data from dual sources and modality-specific prior information provided in the form of partial curricula. Additionally, thanks to the learned dictionaries, we demonstrate that the proposed CPPCL embeds robust coupled representations for SBIR. Our approach is extensively evaluated on four publicly available datasets (i.e. CUFS, Flickr15K, QueenMary SBIR and TU-Berlin Extension datasets), showing superior performance over competing SBIR methods.
Complete list of metadatas

Cited literature [63 references]  Display  Hide  Download

https://hal.inria.fr/hal-01803694
Contributor : Team Perception <>
Submitted on : Wednesday, May 30, 2018 - 4:59:55 PM
Last modification on : Sunday, September 23, 2018 - 5:30:10 PM
Long-term archiving on : Friday, August 31, 2018 - 6:50:01 PM

File

Xu-TIP-2018.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Dan Xu, Xavier Alameda-Pineda, Jingkuan Song, Elisa Ricci, Nicu Sebe. Cross-Paced Representation Learning with Partial Curricula for Sketch-based Image Retrieval. IEEE Transactions on Image Processing, Institute of Electrical and Electronics Engineers, 2018, 27 (9), pp. 4410-4421. ⟨10.1109/TIP.2018.2837381⟩. ⟨hal-01803694⟩

Share

Metrics

Record views

200

Files downloads

141