Actor-Critic Fictitious Play in Simultaneous Move Multistage Games

Julien Pérolat 1, 2 Bilal Piot 3 Olivier Pietquin 3
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : Fictitious play is a game theoretic iterative procedure meant to learn an equilibrium in normal form games. However, this algorithm requires that each player has full knowledge of other players' strategies. Using an architecture inspired by actor-critic algorithms, we build a stochastic approximation of the fictitious play process. This procedure is on-line, decentralized (an agent has no information of others' strategies and rewards) and applies to multistage games (a generalization of normal form games). In addition, we prove convergence of our method towards a Nash equilibrium in both the cases of zero-sum two-player multistage games and cooperative multistage games. We also provide empirical evidence of the soundness of our approach on the game of Alesia with and without function approximation.
Document type :
Conference papers
Complete list of metadatas

Cited literature [31 references]  Display  Hide  Download

https://hal.inria.fr/hal-01724227
Contributor : Julien Pérolat <>
Submitted on : Tuesday, March 6, 2018 - 11:30:04 AM
Last modification on : Friday, March 22, 2019 - 1:36:26 AM
Long-term archiving on : Thursday, June 7, 2018 - 1:43:32 PM

File

AISTATS2018 (13).pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01724227, version 1

Citation

Julien Pérolat, Bilal Piot, Olivier Pietquin. Actor-Critic Fictitious Play in Simultaneous Move Multistage Games. AISTATS 2018 - 21st International Conference on Artificial Intelligence and Statistics, Apr 2018, Playa Blanca, Lanzarote, Canary Islands, Spain. ⟨hal-01724227⟩

Share

Metrics

Record views

396

Files downloads

743