Learning structured prediction models for interactive image labeling

Thomas Mensink 1, 2 Jakob Verbeek 1 Gabriela Csurka 2
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : We propose structured models for image labeling that take into account the dependencies among the image labels explicitly. These models are more expressive than independent label predictors, and lead to more accurate predictions. While the improvement is modest for fully-automatic image labeling, the gain is significant in an interactive scenario where a user provides the value of some of the image labels. Such an interactive scenario offers an interesting trade-off between accuracy and manual labeling effort. The structured models are used to decide which labels should be set by the user, and transfer the user input to more accurate predictions on other image labels. We also apply our models to attribute-based image classification, where attribute predictions of a test image are mapped to class probabilities by means of a given attribute-class mapping. In this case the structured models are built at the attribute level. We also consider an interactive system where the system asks a user to set some of the attribute values in order to maximally improve class prediction performance. Experimental results on three publicly available benchmark data sets show that in all scenarios our structured models lead to more accurate predictions, and leverage user input much more effectively than state-of-the-art independent models.
Document type :
Conference papers
Liste complète des métadonnées

Cited literature [20 references]  Display  Hide  Download


https://hal.inria.fr/inria-00567374
Contributor : Thoth Team <>
Submitted on : Monday, May 9, 2011 - 9:51:01 AM
Last modification on : Tuesday, February 12, 2019 - 10:30:05 AM
Document(s) archivé(s) le : Saturday, December 3, 2016 - 11:58:35 PM

Files

0300.web.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Thomas Mensink, Jakob Verbeek, Gabriela Csurka. Learning structured prediction models for interactive image labeling. CVPR 2011 - IEEE Conference on Computer Vision & Pattern Recognition, Jun 2011, Colorado Springs, United States. pp.833-840, ⟨10.1109/CVPR.2011.5995380⟩. ⟨inria-00567374⟩

Share

Metrics

Record views

663

Files downloads

3341