Feature learning for multi-task inverse reinforcement learning

Olivier Mangin 1 Pierre-Yves Ouedeyer 1
1 Flowers - Flowing Epigenetic Robots and Systems
Inria Bordeaux - Sud-Ouest, U2IS - Unité d'Informatique et d'Ingénierie des Systèmes
Abstract : In this paper we study the question of life long learning of behaviors from human demonstrations by an intelligent system. One approach is to model the observed demonstrations by a stationary policy. Inverse rein-forcement learning, on the other hand, searches a reward function that makes the observed policy closed to optimal in the corresponding Markov decision process. This approach provides a model of the task solved by the demonstrator and has been shown to lead to better generalization in un-known contexts. However both approaches focus on learning a single task from the expert demonstration. In this paper we propose a feature learn-ing approach for inverse reinforcement learning in which several different tasks are demonstrated, but in which each task is modeled as a mixture of several, simpler, primitive tasks. We present an algorithm based on an al-ternate gradient descent to learn simultaneously a dictionary of primitive tasks (in the form of reward functions) and their combination into an ap-proximation of the task underlying observed behavior. We illustrate how this approach enables efficient re-use of knowledge from previous demon-strations. Namely knowledge on tasks that were previously observed by the learner is used to improve the learning of a new composite behavior, thus achieving transfer of knowledge between tasks.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [22 references]  Display  Hide  Download

https://hal.inria.fr/hal-01098040
Contributor : Olivier Mangin <>
Submitted on : Thursday, February 14, 2019 - 4:17:58 PM
Last modification on : Wednesday, July 3, 2019 - 10:48:04 AM
Long-term archiving on : Wednesday, May 15, 2019 - 7:57:04 PM

File

firl.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution - NonCommercial - NoDerivatives 4.0 International License

Identifiers

  • HAL Id : hal-01098040, version 1

Collections

Citation

Olivier Mangin, Pierre-Yves Ouedeyer. Feature learning for multi-task inverse reinforcement learning. 2014. ⟨hal-01098040⟩

Share

Metrics

Record views

57

Files downloads

247