Conjugate Mixture Models for Clustering Multimodal Data

Vasil Khalidov 1 Florence Forbes 1 Radu Horaud 2
1 MISTIS - Modelling and Inference of Complex and Structured Stochastic Systems
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
2 PERCEPTION - Interpretation and Modelling of Images and Videos
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : The problem of multimodal clustering arises whenever the data are gathered with several physically different sensors. Observations from different modalities are not necessarily aligned in the sense that there is no obvious way to associate or to compare them in some common space. A solution may consist in considering multiple clustering tasks independently for each modality. The main difficulty with such an approach is to guarantee that the unimodal clusterings are mutually consistent. In this paper we show that multimodal clustering can be addressed within a novel framework, namely conjugate mixture models. These models exploit the explicit transformations that are often available between an unobserved parameter space (objects) and each one of the observation spaces (sensors). We formulate the problem as a likelihood maximization task and we derive the associated conjugate expectation-maximization algorithm. The convergence properties of the proposed algorithm are thouroughly investigated. Several local/global optimization techniques are proposed in order to increase its convergence speed. Two initialization strategies are proposed and compared. A consistent model-selection criterion is proposed. The algorithm and its variants are tested and evaluated within the task of 3D localization of several speakers using both auditory and visual data.
Type de document :
[Research Report] RR-7117, INRIA. 2009, pp.36
Liste complète des métadonnées
Contributeur : Radu Horaud <>
Soumis le : jeudi 26 novembre 2009 - 18:06:23
Dernière modification le : jeudi 11 janvier 2018 - 01:48:44
Document(s) archivé(s) le : jeudi 17 juin 2010 - 22:18:17


Fichiers produits par l'(les) auteur(s)


  • HAL Id : inria-00436468, version 1


Vasil Khalidov, Florence Forbes, Radu Horaud. Conjugate Mixture Models for Clustering Multimodal Data. [Research Report] RR-7117, INRIA. 2009, pp.36. 〈inria-00436468〉



Consultations de la notice


Téléchargements de fichiers