Skip to Main content Skip to Navigation
Conference papers

Scene Segmentation with CRFs Learned from Partially Labeled Images

Jakob Verbeek 1 William Triggs 2 
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
2 AI - Artificial Intelligence
LJK - Laboratoire Jean Kuntzmann
Abstract : Conditional Random Fields (CRFs) are an effective tool for a variety of different data segmentation and labeling tasks including visual scene interpretation, which seeks to partition images into their constituent semantic-level regions and assign appropriate class labels to each region. For accurate labeling it is important to capture the global context of the image as well as local information. We introduce a CRF based scene labeling model that incorporates both local features and features aggregated over the whole image or large sections of it. Secondly, traditional CRF learning requires fully labeled datasets which can be costly and troublesome to produce. We introduce a method for learning CRFs from datasets with many unlabeled nodes by marginalizing out the unknown labels so that the log-likelihood of the known ones can be maximized by gradient ascent. Loopy Belief Propagation is used to approximate the marginals needed for the gradient and log-likelihood calculations and the Bethe free-energy approximation to the log-likelihood is monitored to control the step size. Our experimental results show that effective models can be learned from fragmentary labelings and that incorporating top-down aggregate features significantly improves the segmentations. The resulting segmentations are compared to the state-of-the-art on three different image datasets.
Document type :
Conference papers
Complete list of metadata

Cited literature [14 references]  Display  Hide  Download
Contributor : Jakob Verbeek Connect in order to contact the contributor
Submitted on : Monday, April 11, 2011 - 2:36:10 PM
Last modification on : Saturday, June 25, 2022 - 7:44:52 PM


  • HAL Id : inria-00321051, version 2



Jakob Verbeek, William Triggs. Scene Segmentation with CRFs Learned from Partially Labeled Images. NIPS 2007 - Advances in Neural Information Processing Systems, Dec 2007, Vancouver, Canada. pp.1553-1560. ⟨inria-00321051v2⟩



Record views


Files downloads