HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Consistency Modifications for Automatically Tuned Monte-Carlo Tree Search

Vincent Berthier 1 Hassen Doghmen 2 Olivier Teytaud 1, 2
2 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : Monte-Carlo Tree Search algorithms (MCTS [4, 6]), includ- ing upper confidence trees (UCT [9]), are known for their impressive ability in high dimensional control problems. Whilst the main testbed is the game of Go, there are increasingly many applications [13, 12, 7]; these algorithms are now widely accepted as strong candidates for high- dimensional control applications. Unfortunately, it is known that for op- timal performance on a given problem, MCTS requires some tuning; this tuning is often handcrafted or automated, with in some cases a loss of consistency, i.e. a bad behavior asymptotically in the computational power. This highly undesirable property led to a stupid behavior of our main MCTS program MoGo in a real-world situation described in sec- tion 3. This is a big trouble for our several works on automatic parameter tuning [3] and the genetic programming of new features in MoGo. We will see in this paper: - A theoretical analysis of MCTS consistency; - Detailed examples of consistent and inconsistent known algorithms; - How to modify a MCTS implementation in order to ensure con- sistency, independently of the modifications to the "scoring" mod- ule (the module which is automatically tuned and genetically pro- grammed in MoGo); - As a by product of this work, we'll see the interesting property that some heavily tuned MCTS implementations are better than UCT in the sense that they do not visit the complete tree (whereas UCT asymptotically does), whilst preserving the consistency at least if "consistency" modifications above have been made.
Document type :
Conference papers
Complete list of metadata

Cited literature [15 references]  Display  Hide  Download

Contributor : Olivier Teytaud Connect in order to contact the contributor
Submitted on : Sunday, November 29, 2009 - 4:03:12 PM
Last modification on : Thursday, July 8, 2021 - 3:48:35 AM
Long-term archiving on: : Thursday, June 17, 2010 - 10:42:17 PM


Files produced by the author(s)


  • HAL Id : inria-00437146, version 1


Vincent Berthier, Hassen Doghmen, Olivier Teytaud. Consistency Modifications for Automatically Tuned Monte-Carlo Tree Search. Lion4, 2010, venice, Italy. 14 p. ⟨inria-00437146⟩



Record views


Files downloads