inria-00623489, version 1
Finding Audio-Visual Events in Informal Social Gatherings
IEEE/ACM International Conference on Multimodal Interaction (2011)
Résumé : In this paper we address the problem of detecting and localizing objects that can be both seen and heard, e.g., people. This may be solved within the framework of data clustering. We propose a new multimodal clustering algorithm based on a Gaussian mixture model, where one of the modalities (visual data) is used to super- vise the clustering process. This is made possible by mapping both modalities into the same metric space. To this end, we fully ex- ploit the geometric and physical properties of an audio-visual sen- sor based on binocular vision and binaural hearing. We propose an EM algorithm that is theoretically well justified, intuitive, and extremely efficient from a computational point of view. This ef- ficiency makes the method implementable on advanced platforms such as humanoid robots. We describe in detail tests and experi- ments performed with publicly available data sets that yield very interesting results.
- a – INRIA Grenoble Rhône-Alpes
- b – INRIA
- 1 :
- INRIA – Laboratoire Jean Kuntzmann – CNRS : UMR5224 – Université Joseph Fourier - Grenoble I – Institut National Polytechnique de Grenoble (INPG) – Université Pierre-Mendès-France - Grenoble II
- 2 :
- CNRS : UMR5224 – Université Joseph Fourier - Grenoble I – Université Pierre-Mendès-France - Grenoble II – Institut Polytechnique de Grenoble - Grenoble Institute of Technology
- 3 :
- IDIAP Research Institute
- 4 :
- INRIA – Laboratoire Jean Kuntzmann
- Domaine : Informatique/Synthèse d'image et réalité virtuelle
- inria-00623489, version 1
- http://hal.inria.fr/inria-00623489
- oai:hal.inria.fr:inria-00623489
- Contributeur :
- Soumis le : Mercredi 14 Septembre 2011, 13:47:51
- Dernière modification le : Lundi 19 Décembre 2011, 18:25:53
Documents associés
Exporter