HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Non linear programming for stochastic dynamic programming

Olivier Teytaud 1 Sylvain Gelly 1
1 TANC - Algorithmic number theory for cryptology
Inria Saclay - Ile de France, LIX - Laboratoire d'informatique de l'École polytechnique [Palaiseau]
Abstract : Many stochastic dynamic programming tasks in continuous action-spaces are tackled through discretization. We here avoid discretization; then, approximate dynamic programming (ADP) involves (i) many learning tasks, performed here by Support Vector Machines, for Bellman-function-regression (ii) many non-linearoptimization tasks for action-selection, for which we compare many algorithms. We include discretizations of the domain as particular non-linear-programming-tools in our experiments, so that by the way we compare optimization approaches and discretization methods. We conclude that robustness is strongly required in the non-linear-optimizations in ADP, and experimental results show that (i) discretization is sometimes inefficient, but some specific discretization is very efficient for "bang-bang" problems (ii) simple evolutionary tools outperform quasi-random in a stable manner (iii) gradient-based techniques are much less stable (iv) for most high-dimensional "less unsmooth" problems Covariance-Matrix-Adaptation is first ranked.
Document type :
Conference papers
Complete list of metadata

Cited literature [38 references]  Display  Hide  Download

Contributor : Olivier Teytaud Connect in order to contact the contributor
Submitted on : Wednesday, September 19, 2007 - 2:15:45 PM
Last modification on : Friday, February 4, 2022 - 3:15:43 AM
Long-term archiving on: : Friday, April 9, 2010 - 2:28:16 AM


Files produced by the author(s)


  • HAL Id : inria-00173202, version 1



Olivier Teytaud, Sylvain Gelly. Non linear programming for stochastic dynamic programming. Icinco 2007, 2007, Angers, France. ⟨inria-00173202⟩



Record views


Files downloads