An Efficient Policy Iteration Algorithm for Dynamic Programming Equations

Abstract : We present an accelerated algorithm for the solution of static Hamilton-Jacobi-Bellman equations related to optimal control problems. Our scheme is based on a classic policy iteration procedure, which is known to have superlinear convergence in many relevant cases provided the initial guess is sufficiently close to the solution. In many cases, this limitation degenerates into a behavior similar to a value iteration method, with an increased computation time. The new scheme circumvents this problem by combining the advantages of both algorithms with an efficient coupling. The method starts with a value iteration phase and then switches to a policy iteration procedure when a certain error threshold is reached. A delicate point is to determine this threshold in order to avoid cumbersome computation with the value iteration and, at the same time, to be reasonably sure that the policy iteration method will finally converge to the optimal solution. We analyze the methods and efficient coupling in a number of examples in dimension two, three and four illustrating its properties.
Type de document :
Article dans une revue
SIAM Journal on Scientific Computing, Society for Industrial and Applied Mathematics, 2015, 37 (1), pp.A181-A200. 〈10.1137/130932284〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01068295
Contributeur : Estelle Bouzat <>
Soumis le : jeudi 25 septembre 2014 - 13:47:28
Dernière modification le : mercredi 27 juillet 2016 - 14:48:48

Lien texte intégral

Identifiants

Collections

Citation

Alessandro Alla, Maurizio Falcone, Dante Kalise. An Efficient Policy Iteration Algorithm for Dynamic Programming Equations. SIAM Journal on Scientific Computing, Society for Industrial and Applied Mathematics, 2015, 37 (1), pp.A181-A200. 〈10.1137/130932284〉. 〈hal-01068295〉

Partager

Métriques

Consultations de la notice

127