Consistency Modifications for Automatically Tuned Monte-Carlo Tree Search

Vincent Berthier 1 Hassen Doghmen 2 Olivier Teytaud 1, 2
2 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
Abstract : Monte-Carlo Tree Search algorithms (MCTS [4, 6]), includ- ing upper confidence trees (UCT [9]), are known for their impressive ability in high dimensional control problems. Whilst the main testbed is the game of Go, there are increasingly many applications [13, 12, 7]; these algorithms are now widely accepted as strong candidates for high- dimensional control applications. Unfortunately, it is known that for op- timal performance on a given problem, MCTS requires some tuning; this tuning is often handcrafted or automated, with in some cases a loss of consistency, i.e. a bad behavior asymptotically in the computational power. This highly undesirable property led to a stupid behavior of our main MCTS program MoGo in a real-world situation described in sec- tion 3. This is a big trouble for our several works on automatic parameter tuning [3] and the genetic programming of new features in MoGo. We will see in this paper: - A theoretical analysis of MCTS consistency; - Detailed examples of consistent and inconsistent known algorithms; - How to modify a MCTS implementation in order to ensure con- sistency, independently of the modifications to the "scoring" mod- ule (the module which is automatically tuned and genetically pro- grammed in MoGo); - As a by product of this work, we'll see the interesting property that some heavily tuned MCTS implementations are better than UCT in the sense that they do not visit the complete tree (whereas UCT asymptotically does), whilst preserving the consistency at least if "consistency" modifications above have been made.
Document type :
Conference papers
Complete list of metadatas

Cited literature [15 references]  Display  Hide  Download

https://hal.inria.fr/inria-00437146
Contributor : Olivier Teytaud <>
Submitted on : Sunday, November 29, 2009 - 4:03:12 PM
Last modification on : Thursday, April 5, 2018 - 12:30:12 PM
Long-term archiving on : Thursday, June 17, 2010 - 10:42:17 PM

File

consistency.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00437146, version 1

Collections

Citation

Vincent Berthier, Hassen Doghmen, Olivier Teytaud. Consistency Modifications for Automatically Tuned Monte-Carlo Tree Search. Lion4, 2010, venice, Italy. 14 p. ⟨inria-00437146⟩

Share

Metrics

Record views

607

Files downloads

690