Using Natural Language Feedback in a Neuro-inspired Integrated Multimodal Robotic Architecture

Johannes Twiefel 1 Xavier Hinaut 2, 1 Marcelo Borghetti 1 Erik Strahl 1 Stefan Wermter 1
1 KT - Knowledge Technology group [Hamburg]
Department of Informatics [Hamburg]
2 Mnemosyne - Mnemonic Synergy
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest, IMN - Institut des Maladies Neurodégénératives [Bordeaux]
Abstract : In this paper we present a multi-modal human robot interaction architecture which is able to combine information coming from different sensory inputs, and can generate feedback for the user which helps to teach him/her implicitly how to interact with the robot. The system combines vision, speech and language with inference and feedback. The system environment consists of a Nao robot which has to learn objects situated on a table only by understanding absolute and relative object locations uttered by the user and afterwards points on a desired object to show what it has learned. The results of a user study and performance test show the usefulness of the feedback produced by the system and also justify the usage of the system in a real-world applications, as its classification accuracy of multi-modal input is around 80.8%. In the experiments, the system was able to detect inconsistent input coming from different sensory modules in all cases and could generate useful feedback for the user from this information.
Type de document :
Communication dans un congrès
25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Aug 2016, New York City, United States. pp.52 - 57, 2016, Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). 〈http://www.tc.columbia.edu/conferences/roman2016/〉. 〈10.1109/ROMAN.2016.7745090〉
Liste complète des métadonnées

Littérature citée [14 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/hal-01417706
Contributeur : Xavier Hinaut <>
Soumis le : jeudi 15 décembre 2016 - 19:21:50
Dernière modification le : jeudi 7 février 2019 - 14:22:40

Fichier

twiefel_ro-man_2016_camery_rea...
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

INRIA-BORDEAUX | UNIV-BORDEAUX | LABRI | INRIA

Citation

Johannes Twiefel, Xavier Hinaut, Marcelo Borghetti, Erik Strahl, Stefan Wermter. Using Natural Language Feedback in a Neuro-inspired Integrated Multimodal Robotic Architecture. 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Aug 2016, New York City, United States. pp.52 - 57, 2016, Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). 〈http://www.tc.columbia.edu/conferences/roman2016/〉. 〈10.1109/ROMAN.2016.7745090〉. 〈hal-01417706〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

503

Téléchargements de fichiers

238

Contact


archive-ouverte@inria.fr