Skip to Main content Skip to Navigation
Conference papers

MESSI: Maximum Entropy Semi-Supervised Inverse Reinforcement Learning

Julien Audiffren 1 Michal Valko 2 Alessandro Lazaric 2 Mohammad Ghavamzadeh 2
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert's behavior. In this paper, we study an AL setting in which in addition to the expert's trajectories, a number of unsupervised trajectories is available. We introduce MESSI, a novel algorithm that combines MaxEnt-IRL with principles coming from semi-supervised learning. In particular, MESSI integrates the unsupervised data into the MaxEnt-IRL framework using a pairwise penalty on trajectories. Empirical results in a highway driving and grid-world problems indicate that MESSI is able to take advantage of the unsupervised trajectories and improve the performance of MaxEnt-IRL.
Document type :
Conference papers
Complete list of metadata

Cited literature [9 references]  Display  Hide  Download
Contributor : Michal Valko Connect in order to contact the contributor
Submitted on : Thursday, July 16, 2015 - 11:19:52 PM
Last modification on : Thursday, January 20, 2022 - 4:16:50 PM
Long-term archiving on: : Wednesday, April 26, 2017 - 6:53:34 AM


Files produced by the author(s)


  • HAL Id : hal-01177446, version 1


Julien Audiffren, Michal Valko, Alessandro Lazaric, Mohammad Ghavamzadeh. MESSI: Maximum Entropy Semi-Supervised Inverse Reinforcement Learning. NIPS Workshop on Novel Trends and Applications in Reinforcement Learning, 2014, Montreal, Canada. ⟨hal-01177446⟩



Les métriques sont temporairement indisponibles