Compatible Reward Inverse Reinforcement Learning

Alberto Metelli 1 Matteo Pirotta 2 Marcello Restelli 1
2 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : Inverse Reinforcement Learning (IRL) is an effective approach to recover a reward function that explains the behavior of an expert by observing a set of demonstrations. This paper is about a novel model-free IRL approach that, differently from most of the existing IRL algorithms, does not require to specify a function space where to search for the expert's reward function. Leveraging on the fact that the policy gradient needs to be zero for any optimal policy, the algorithm generates a set of basis functions that span the subspace of reward functions that make the policy gradient vanish. Within this subspace, using a second-order criterion, we search for the reward function that penalizes the most a deviation from the expert's policy. After introducing our approach for finite domains, we extend it to continuous ones. The proposed approach is empirically compared to other IRL methods both in the (finite) Taxi domain and in the (continuous) Linear Quadratic Gaussian (LQG) and Car on the Hill environments.
Type de document :
Communication dans un congrès
The Thirty-first Annual Conference on Neural Information Processing Systems - NIPS 2017, Dec 2017, Long Beach, United States
Liste complète des métadonnées

Littérature citée [40 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01653328
Contributeur : Alessandro Lazaric <>
Soumis le : vendredi 1 décembre 2017 - 12:17:50
Dernière modification le : vendredi 13 avril 2018 - 01:28:46

Fichier

6800-compatible-reward-inverse...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01653328, version 1

Collections

Citation

Alberto Metelli, Matteo Pirotta, Marcello Restelli. Compatible Reward Inverse Reinforcement Learning. The Thirty-first Annual Conference on Neural Information Processing Systems - NIPS 2017, Dec 2017, Long Beach, United States. 〈hal-01653328〉

Partager

Métriques

Consultations de la notice

210

Téléchargements de fichiers

39