Abstract: In this article, we consider the task of category level object segmentation. Object models based on bag-of-words representations achieve state-of-the-art performance for object recognition. However, they fail to accurately locate object boundaries and thus produce inaccurate object segmentation. On the other hand, Markov Random Field based models used for image segmentation focus on object boundaries but can hardly use global object constraints, which is required when dealing with object categories whose appearance may vary significantly. The key contribution of this paper is to combine the advantages of these two approaches. First, a blob-based mechanism allows to detect objects using visual word occurrences, and produces rough image segmentation. Second, a MRF component produces clean cuts, guided by local image cues (color, texture and edge cues) and by long-distance dependency given by the blob model, which enforces label consistency. Our approach is validated on standard public datasets, containing different object classes, in presence of cluttered backgrounds and large view point changes.