Skip to Main content Skip to Navigation
Conference papers

Learning a Behavioral Repertoire from Demonstrations

Abstract : Imitation Learning (IL) is a machine learning approach to learn a policy from a set of demonstrations. IL can be useful to kick-start learning before applying reinforcement learning (RL) but it can also be useful on its own, e.g. to learn to imitate human players in video games. Despite the success of systems that use IL and RL, how such systems can adapt in-between game rounds is a neglected area of study but an important aspect of many strategy games. In this paper, we present a new approach called Behavioral Repertoire Imitation Learning (BRIL) that learns a repertoire of behaviors from a set of demonstrations by augmenting the state-action pairs with behavioral descriptions. The outcome of this approach is a single neural network policy conditioned on a behavior description that can be precisely modulated. We apply this approach to train a policy on 7,777 human demonstrations for the build-order planning task in StarCraft II. Dimensionality reduction is applied to construct a low-dimensional behavioral space from a high-dimensional description of the army unit composition of each human replay. The results demonstrate that the learned policy can be effectively manipulated to express distinct behaviors. Additionally, by applying the UCB1 algorithm, the policy can adapt its behavior-in-between games-to reach a performance beyond that of the traditional IL baseline approach.
Document type :
Conference papers
Complete list of metadata

Cited literature [42 references]  Display  Hide  Download

https://hal.inria.fr/hal-02868800
Contributor : Jean-Baptiste Mouret <>
Submitted on : Monday, June 15, 2020 - 4:03:32 PM
Last modification on : Tuesday, March 9, 2021 - 10:41:49 AM

File

CoG_2020___Learning_a_behavior...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02868800, version 1

Collections

Citation

Niels Justesen, Miguel González-Duque, Daniel Cabarcas, Jean-Baptiste Mouret, Sebastian Risi. Learning a Behavioral Repertoire from Demonstrations. CoG 2020 - IEEE Conference on Games, 2020, Osaka / Virtual, Japan. ⟨hal-02868800⟩

Share

Metrics

Record views

43

Files downloads

312