Skip to Main content Skip to Navigation
New interface
Poster communications

A Multimodal Human-Robot Interaction Dataset

Abstract : This works presents a multimodal dataset for Human-Robot Interactive Learning. 1 The dataset contains synchronized recordings of several human users, from a stereo 2 microphone and three cameras mounted on the robot. The focus of the dataset is 3 incremental object learning, oriented to human-robot assistance and interaction. To 4 learn new object models from interactions with a human user, the robot needs to 5 be able to perform multiple tasks: (a) recognize the type of interaction (pointing, 6 showing or speaking), (b) segment regions of interest from acquired data (hands and 7 objects), and (c) learn and recognize object models. We illustrate the advantages 8 of multimodal data over camera-only datasets by presenting an approach that 9 recognizes the user interaction by combining simple image and language features.
Document type :
Poster communications
Complete list of metadata

Cited literature [12 references]  Display  Hide  Download
Contributor : Florian Golemo Connect in order to contact the contributor
Submitted on : Wednesday, December 7, 2016 - 4:10:42 PM
Last modification on : Saturday, June 25, 2022 - 9:10:27 PM
Long-term archiving on: : Wednesday, March 22, 2017 - 11:49:41 PM


Files produced by the author(s)


  • HAL Id : hal-01402479, version 1



Pablo Azagra, Yoan Mollard, Florian Golemo, Ana Cristina Murillo, Manuel Lopes, et al.. A Multimodal Human-Robot Interaction Dataset. NIPS 2016, workshop Future of Interactive Learning Machines, Dec 2016, Barcelona, Spain. ⟨hal-01402479⟩



Record views


Files downloads