Using Natural Language Feedback in a Neuro-inspired Integrated Multimodal Robotic Architecture

Johannes Twiefel 1 Xavier Hinaut 2, 1 Marcelo Borghetti 1 Erik Strahl 1 Stefan Wermter 1
1 KT - Knowledge Technology group [Hamburg]
Department of Informatics [Hamburg]
2 Mnemosyne - Mnemonic Synergy
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest, IMN - Institut des Maladies Neurodégénératives [Bordeaux]
Abstract : In this paper we present a multi-modal human robot interaction architecture which is able to combine information coming from different sensory inputs, and can generate feedback for the user which helps to teach him/her implicitly how to interact with the robot. The system combines vision, speech and language with inference and feedback. The system environment consists of a Nao robot which has to learn objects situated on a table only by understanding absolute and relative object locations uttered by the user and afterwards points on a desired object to show what it has learned. The results of a user study and performance test show the usefulness of the feedback produced by the system and also justify the usage of the system in a real-world applications, as its classification accuracy of multi-modal input is around 80.8%. In the experiments, the system was able to detect inconsistent input coming from different sensory modules in all cases and could generate useful feedback for the user from this information.
Document type :
Conference papers
Complete list of metadatas

Cited literature [14 references]  Display  Hide  Download
https://hal.inria.fr/hal-01417706
Contributor : Xavier Hinaut <>
Submitted on : Thursday, December 15, 2016 - 7:21:50 PM
Last modification on : Thursday, February 7, 2019 - 2:22:40 PM

File

twiefel_ro-man_2016_camery_rea...
Files produced by the author(s)

Identifiers

Collections

INRIA-BORDEAUX | UNIV-BORDEAUX | LABRI | INRIA

Citation

Johannes Twiefel, Xavier Hinaut, Marcelo Borghetti, Erik Strahl, Stefan Wermter. Using Natural Language Feedback in a Neuro-inspired Integrated Multimodal Robotic Architecture. 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Aug 2016, New York City, United States. pp.52 - 57, ⟨10.1109/ROMAN.2016.7745090⟩. ⟨hal-01417706⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

590

Files downloads

321

Contact


archive-ouverte@inria.fr