Learning Semantic Components from Subsymbolic Multimodal Perception

Olivier Mangin 1 Pierre-Yves Oudeyer 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : Perceptual systems often include sensors from several modalities. However, existing robots do not yet sufficiently discover patterns that are spread over the flow of multimodal data they receive. In this paper we present a framework that learns a dictionary of words from full spoken utterances, together with a set of gestures from human demonstrations and the semantic connection between words and gestures. We explain how to use a nonnegative matrix factorization algorithm to learn a dictionary of components that represent meaningful elements present in the multimodal perception, without providing the system with a symbolic representation of the semantics. We illustrate this framework by showing how a learner discovers word-like components from observation of gestures made by a human together with spoken descriptions of the gestures, and how it captures the semantic association between the two.
Type de document :
Communication dans un congrès
Joint IEEE International Conference on Development and Learning an on Epigenetic Robotics (ICDL-EpiRob), Aug 2013, Osaka, Japan. IEEE, 2013
Liste complète des métadonnées

Littérature citée [21 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00842453
Contributeur : Olivier Mangin <>
Soumis le : lundi 8 juillet 2013 - 15:45:00
Dernière modification le : jeudi 16 novembre 2017 - 17:12:01
Document(s) archivé(s) le : mercredi 5 avril 2017 - 08:14:42

Fichier

mangin.2013.icdl.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00842453, version 1

Collections

Citation

Olivier Mangin, Pierre-Yves Oudeyer. Learning Semantic Components from Subsymbolic Multimodal Perception. Joint IEEE International Conference on Development and Learning an on Epigenetic Robotics (ICDL-EpiRob), Aug 2013, Osaka, Japan. IEEE, 2013. 〈hal-00842453〉

Partager

Métriques

Consultations de la notice

214

Téléchargements de fichiers

191