Conceptual Feedback for Semantic Multimedia Indexing

Abstract : In this paper, we consider the problem of automat- ically detecting a large number of visual concepts in images or video shots. State of the art systems involve feature (descriptor) extraction, classification (supervised learning) and fusion when several descriptors and/or classifiers are used. Though direct multi-label approaches are considered in some works, detection scores are often computed independently for each target concept. We propose here a method that we call "conceptual feedback" for improving the overall detection performance that implicitly takes into account the relations between concepts. The vector of normalized detection scores is added to the pool of available descriptors. It is then processed just as the other descriptors for the normalization, optimization and classification steps. The resulting detection scores are finally fused with the already avail- able detection scores obtained with the original descriptors. The feedback of the global detection scores in the pool of descriptors can be iterated several times. It is also compatible with the use of the temporal context that also improves the overall performance by taking into account the local homogeneity of video contents. The method has been evaluated in the context of the TRECVID 2012 semantic indexing task involving the detection of 346 visual or multimodal concepts. Combined with temporal re-scoring, the proposed method increased the global system performance (MAP) from 0.2613 to 0.3014 (+15.3% of relative improvement) while the temporal re-scoring alone increased it only from 0.2613 to 0.2691 (+3.0%).
Type de document :
Communication dans un congrès
Content-Based Multimedia Indexing (CBMI), 2013 11th International Workshop on, 2013, Veszprém, Hungary. 53-58, 2013
Liste complète des métadonnées

https://hal.inria.fr/hal-00953085
Contributeur : Marie-Christine Fauvet <>
Soumis le : vendredi 28 février 2014 - 10:31:42
Dernière modification le : jeudi 11 janvier 2018 - 06:21:05

Identifiants

  • HAL Id : hal-00953085, version 1

Collections

Citation

Abdelkader Hamadi, Georges Quénot, Philippe Mulhem. Conceptual Feedback for Semantic Multimedia Indexing. Content-Based Multimedia Indexing (CBMI), 2013 11th International Workshop on, 2013, Veszprém, Hungary. 53-58, 2013. 〈hal-00953085〉

Partager

Métriques

Consultations de la notice

278