Enhancing Energy Minimization Framework for Scene Text Recognition with Top-Down Cues

Abstract : Recognizing scene text is a challenging problem, even more so than the recognition of scanned documents. This problem has gained significant attention from the computer vision community in recent years, and several methods based on energy minimization frameworks and deep learning approaches have been proposed. In this work, we focus on the energy minimization framework and propose a model that exploits both bottom-up and top-down cues for recognizing cropped words extracted from street images. The bottom-up cues are derived from individual character detections from an image. We build a conditional random field model on these detections to jointly model the strength of the detections and the interactions between them. These interactions are top-down cues obtained from a lexicon-based prior, i.e., language statistics. The optimal word represented by the text image is obtained by minimizing the energy function corresponding to the random field model. We evaluate our proposed algorithm extensively on a number of cropped scene text benchmark datasets, namely Street View Text, ICDAR 2003, 2011 and 2013 datasets, and IIIT 5K-word, and show better performance than comparable methods. We perform a rigorous analysis of all the steps in our approach and analyze the results. We also show that state-of-the-art convolutional neural network features can be integrated in our framework to further improve the recognition performance.
Document type :
Journal articles
Complete list of metadatas

Contributor : Karteek Alahari <>
Submitted on : Wednesday, January 27, 2016 - 4:24:05 PM
Last modification on : Monday, May 28, 2018 - 3:10:06 PM
Long-term archiving on : Thursday, April 28, 2016 - 11:22:09 AM


Files produced by the author(s)




Anand Mishra, Karteek Alahari, C.V. Jawahar. Enhancing Energy Minimization Framework for Scene Text Recognition with Top-Down Cues. Computer Vision and Image Understanding, Elsevier, 2016, 145, pp.30-42. ⟨10.1016/j.cviu.2016.01.002⟩. ⟨hal-01263322⟩



Record views


Files downloads