MESSI: Maximum Entropy Semi-Supervised Inverse Reinforcement Learning

Julien Audiffren 1 Michal Valko 2 Alessandro Lazaric 2 Mohammad Ghavamzadeh 2
1 CMLA - Centre de Mathématiques et de Leurs Applications
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal, Inria Lille - Nord Europe
Abstract : A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert's behavior. In this paper, we study an AL setting in which in addition to the expert's trajectories, a number of unsupervised trajectories is available. We introduce MESSI, a novel algorithm that combines MaxEnt-IRL with principles coming from semi-supervised learning. In particular, MESSI integrates the unsupervised data into the MaxEnt-IRL framework using a pairwise penalty on trajectories. Empirical results in a highway driving and grid-world problems indicate that MESSI is able to take advantage of the unsupervised trajectories and improve the performance of MaxEnt-IRL.
Type de document :
Communication dans un congrès
NIPS Workshop on Novel Trends and Applications in Reinforcement Learning, 2014, Montreal, Canada
Liste complète des métadonnées

Littérature citée [9 références]  Voir  Masquer  Télécharger
https://hal.inria.fr/hal-01177446
Contributeur : Michal Valko <>
Soumis le : jeudi 16 juillet 2015 - 23:19:52
Dernière modification le : jeudi 11 janvier 2018 - 06:22:13
Document(s) archivé(s) le : mercredi 26 avril 2017 - 06:53:34

Fichier

audiffren2014messi.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01177446, version 1

Collections

UNIV-LILLE3 | LAGIS | INRIA | CRISTAL | CRISTAL-SEQUEL | ENS-CACHAN

Citation

Julien Audiffren, Michal Valko, Alessandro Lazaric, Mohammad Ghavamzadeh. MESSI: Maximum Entropy Semi-Supervised Inverse Reinforcement Learning. NIPS Workshop on Novel Trends and Applications in Reinforcement Learning, 2014, Montreal, Canada. 〈hal-01177446〉

Exporter

BibTeX TEI DC DCterms EndNote

Partager

Métriques

Consultations de la notice

299

Téléchargements de fichiers

209

Contact


archive-ouverte@inria.fr