A Multimodal Dataset for Interactive and Incremental Learning of Object Models

Abstract : This work presents an incremental object learning framework oriented to human-robot assistance and interaction. To learn new object models from interactions with a human user, the robot needs to be able to perform multiple recognition tasks: (a) recognize the type of interaction, (b) segment regions of interest from acquired data, and (c) learn and recognize object models. The contributions on this work are focused on the recognition modules of this human-robot interactive framework. First, we illustrate the advantages of multimodal data over camera-only datasets. We present an approach that recognizes the user interaction by combining simple image and language features. Second, we propose an incremental approach to learn visual object models, which is shown to achieve comparable performance to a typical offline-trained system. We utilize two public datasets, one of them presented and released in this work. This dataset contains synchronized recordings from user speech and three cameras mounted on a robot, which captured the user teaching object names to the robot.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [26 references]  Display  Hide  Download

https://hal.inria.fr/hal-01402493
Contributor : Florian Golemo <>
Submitted on : Thursday, November 24, 2016 - 5:00:35 PM
Last modification on : Wednesday, July 3, 2019 - 10:48:04 AM
Long-term archiving on : Tuesday, March 21, 2017 - 4:10:19 AM

File

multimodal-dataset-interactive...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01402493, version 1

Citation

Pablo Azagra, Yoan Mollard, Florian Golemo, Ana Murillo, Manuel Lopes, et al.. A Multimodal Dataset for Interactive and Incremental Learning of Object Models. 2016. ⟨hal-01402493⟩

Share

Metrics

Record views

478

Files downloads

524