Skip to Main content Skip to Navigation
Journal articles

Medical Image Retrieval based on Knowledge-Assisted Text and Image Indexing

Abstract : Voluminous medical images are generated daily. They are critical assets for medical diagnosis, research, and teaching. To facilitate automatic indexing and retrieval of large medical image databases, both images and associated texts are indexed using medical concepts from the Unified Medical Language System (UMLS) meta-thesaurus. We propose a structured learning framework based on Support Vector Machines to facilitate modular design and learning of medical semantics from images. We present two complementary visual indexing approaches within this framework: a global indexing to access image modality, and a local indexing to access semantic local features. Two fusion approaches are developed to improve textual retrieval using the UMLS-based image indexing. First, a simple fusion of the textual and visual retrieval approaches is proposed, improving significantly the retrieval results of both text and image retrieval. Second, a visual modality filtering is designed to remove visually aberrant images according to the query modality concept(s). Using the ImageCLEFmed database, we demonstrate the effectiveness of our framework that is superior when compared to the automatic runs evaluated in 2005 on the same Medical Image Retrieval task.
Document type :
Journal articles
Complete list of metadata
Contributor : Marie-Christine Fauvet Connect in order to contact the contributor
Submitted on : Friday, February 28, 2014 - 4:00:31 PM
Last modification on : Sunday, June 26, 2022 - 4:58:58 AM


  • HAL Id : hal-00953799, version 1


Caroline Lacoste, Joo-Hwee Lim, Jean-Pierre Chevallet, Thi Hoang Diem Le. Medical Image Retrieval based on Knowledge-Assisted Text and Image Indexing. IEEE Transactions on Circuits and Systems for Video Technology, Institute of Electrical and Electronics Engineers, 2007, 17 (7), pp.889--900. ⟨hal-00953799⟩



Record views