Skip to Main content Skip to Navigation
Conference papers

Primal Wasserstein Imitation Learning

Robert Dadashi 1, * Léonard Hussenot 1, 2 Matthieu Geist 1 Olivier Pietquin 1
* Corresponding author
2 Scool - Scool
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
Abstract : Imitation Learning (IL) methods seek to match the behavior of an agent with that of an expert. In the present work, we propose a new IL method based on a conceptually simple algorithm: Primal Wasserstein Imitation Learning (PWIL), which ties to the primal form of the Wasserstein distance between the expert and the agent state-action distributions. We present a reward function which is derived offline, as opposed to recent adversarial IL algorithms that learn a reward function through interactions with the environment, and which requires little fine-tuning. We show that we can recover expert behavior on a variety of continuous control tasks of the MuJoCo domain in a sample efficient manner in terms of agent interactions and of expert interactions with the environment. Finally, we show that the behavior of the agent we train matches the behavior of the expert with the Wasserstein distance, rather than the commonly used proxy of performance.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03162526
Contributor : Léonard Hussenot <>
Submitted on : Monday, March 8, 2021 - 4:01:06 PM
Last modification on : Tuesday, March 9, 2021 - 3:27:47 AM

File

2006.04678.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03162526, version 1
  • ARXIV : 2006.04678

Collections

Citation

Robert Dadashi, Léonard Hussenot, Matthieu Geist, Olivier Pietquin. Primal Wasserstein Imitation Learning. ICLR 2021 - Ninth International Conference on Learning Representations, May 2021, Vienna / Virtual, Austria. ⟨hal-03162526⟩

Share

Metrics

Record views

24

Files downloads

180