Skip to Main content Skip to Navigation
New interface
Other publications

Image-based Semantic Learning Software for Automatic Detection of Discriminative Criteria used for probe-based Confocal Laser Endomicroscopy (pCLE) Diagnosis of Colorectal Polyps

Abstract : BACKGROUND AND AIMS pCLE (Cellvizio, Mauna Kea Technologies) enables in vivo microscopic imaging of the epithelium in real-time during ongoing endoscopy. As pCLE is a recent technology, the in vivo interpretation of pCLE images of colorectal polyps is still challenging for many endoscopists. This study aims at supporting pCLE diagnosis of colorectal polyps, by developing a software based on image retrieval for the automatic extraction of semantic concepts in pCLE sequences. METHODS Intravenous fluorescein pCLE imaging of colorectal lesions was performed on patients undergoing surveillance colonoscopies. The pCLE video sequences, recorded for each polyp, were manually annotated with 6 discriminative criteria either present or absent in the videos. These criteria, annotated by expert endoscopists with the support of the modified Mainz criteria, were: "visible blood vessel", "normal goblet cell", "round crypt", "elongated/tortuous crypt", "visible lumen" and "star-shaped opening". These semantic criteria were then learned by the proposed software based on a content-based image retrieval technique followed by a Fisher-based transformation method. For each discriminative criterion, the performance of automatic detection performed by the proposed software were compared to that of state-of-the-art machine learning methods (support vector machines) using 30x3 fold cross-validation. RESULTS 118 colorectal lesions were imaged in 66 patients. Based on histopathology, 83 of these 118 lesions were neoplastic and 35 were non-neoplastic. Table 1 reports the area under the receiver operating characteristic (ROC) curves indicating the performance of automatic criteria detection. The proposed detection software performs overall statistically better than the state-of-the-art machine learning methods (p < 0.05). Figure 1 shows a typical example of a pCLE query, for which the most visually similar pCLE sequence is automatically extracted, together with "semantic" star plots showing the probability that each discriminative criterion is present in the pCLE sequences. Possible disagreements between automatic criteria detection and ground-truth annotations may reveal ambiguous pCLE sequences that are difficult to interpret. CONCLUSIONS This study is a proof of concept that pCLE clinical knowledge can be automatically extracted from pCLE sequences of colorectal polyps. The proposed software for automatic semantic detection combined with image retrieval provides the endoscopists with clinically relevant information, both visual and semantic, which should be easily interpretable to make an informed pCLE diagnosis. Further studies are needed to improve software performances and to evaluate the software as a second reader tool for pCLE diagnosis.
Complete list of metadata
Contributor : Project-Team Asclepios Connect in order to contact the contributor
Submitted on : Thursday, July 4, 2013 - 6:56:15 PM
Last modification on : Friday, November 18, 2022 - 9:28:01 AM
Long-term archiving on: : Saturday, October 5, 2013 - 2:55:12 AM


Files produced by the author(s)


  • HAL Id : hal-00813816, version 1



Barbara André, Tom Vercauteren, Anna M. Buchner, Michael B. Wallace, Nicholas Ayache. Image-based Semantic Learning Software for Automatic Detection of Discriminative Criteria used for probe-based Confocal Laser Endomicroscopy (pCLE) Diagnosis of Colorectal Polyps. 2012. ⟨hal-00813816⟩



Record views


Files downloads