Skip to Main content Skip to Navigation
New interface
Preprints, Working Papers, ...

A Multimodal Dataset for Interactive and Incremental Learning of Object Models

Abstract : This work presents an incremental object learning framework oriented to human-robot assistance and interaction. To learn new object models from interactions with a human user, the robot needs to be able to perform multiple recognition tasks: (a) recognize the type of interaction, (b) segment regions of interest from acquired data, and (c) learn and recognize object models. The contributions on this work are focused on the recognition modules of this human-robot interactive framework. First, we illustrate the advantages of multimodal data over camera-only datasets. We present an approach that recognizes the user interaction by combining simple image and language features. Second, we propose an incremental approach to learn visual object models, which is shown to achieve comparable performance to a typical offline-trained system. We utilize two public datasets, one of them presented and released in this work. This dataset contains synchronized recordings from user speech and three cameras mounted on a robot, which captured the user teaching object names to the robot.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

Cited literature [26 references]  Display  Hide  Download
Contributor : Florian Golemo Connect in order to contact the contributor
Submitted on : Thursday, November 24, 2016 - 5:00:35 PM
Last modification on : Saturday, June 25, 2022 - 9:10:28 PM
Long-term archiving on: : Tuesday, March 21, 2017 - 4:10:19 AM


Files produced by the author(s)


  • HAL Id : hal-01402493, version 1



Pablo Azagra, Yoan Mollard, Florian Golemo, Ana C Murillo, Manuel Lopes, et al.. A Multimodal Dataset for Interactive and Incremental Learning of Object Models. 2016. ⟨hal-01402493⟩



Record views


Files downloads