Approximate Modified Policy Iteration - Archive ouverte HAL Access content directly
Conference Papers Year : 2012

Approximate Modified Policy Iteration

(1) , (2) , (2) , (3, 4)
Bruno Scherrer
Mohammad Ghavamzadeh
  • Function : Author
  • PersonId : 868946
Victor Gabillon
  • Function : Author
  • PersonId : 925091


Modified policy iteration (MPI) is a dynamic programming (DP) algorithm that contains the two celebrated policy and value iteration methods. Despite its generality, MPI has not been thoroughly studied, especially its approximation form which is used when the state and/or action spaces are large or infinite. In this paper, we propose three implementations of approximate MPI (AMPI) that are extensions of well-known approximate DP algorithms: fitted-value iteration, fitted-Q iteration, and classification-based policy iteration. We provide error propagation analysis that unifies those for approximate policy and value iteration. For the classification-based implementation, we develop a finite-sample analysis that shows that MPI's main parameter allows to control the balance between the estimation error of the classifier and the overall value function approximation.
Fichier principal
Vignette du fichier
icml-short.pdf (423.97 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-00758882 , version 1 (29-11-2012)


  • HAL Id : hal-00758882 , version 1


Bruno Scherrer, Mohammad Ghavamzadeh, Victor Gabillon, Matthieu Geist. Approximate Modified Policy Iteration. 29th International Conference on Machine Learning - ICML 2012, Jun 2012, Edinburgh, United Kingdom. ⟨hal-00758882⟩
229 View
149 Download


Gmail Facebook Twitter LinkedIn More