Skip to Main content Skip to Navigation
Conference papers

Non linear programming for stochastic dynamic programming

Olivier Teytaud 1 Sylvain Gelly 1
1 TANC - Algorithmic number theory for cryptology
Inria Saclay - Ile de France, LIX - Laboratoire d'informatique de l'École polytechnique [Palaiseau]
Abstract : Many stochastic dynamic programming tasks in continuous action-spaces are tackled through discretization. We here avoid discretization; then, approximate dynamic programming (ADP) involves (i) many learning tasks, performed here by Support Vector Machines, for Bellman-function-regression (ii) many non-linearoptimization tasks for action-selection, for which we compare many algorithms. We include discretizations of the domain as particular non-linear-programming-tools in our experiments, so that by the way we compare optimization approaches and discretization methods. We conclude that robustness is strongly required in the non-linear-optimizations in ADP, and experimental results show that (i) discretization is sometimes inefficient, but some specific discretization is very efficient for "bang-bang" problems (ii) simple evolutionary tools outperform quasi-random in a stable manner (iii) gradient-based techniques are much less stable (iv) for most high-dimensional "less unsmooth" problems Covariance-Matrix-Adaptation is first ranked.
Document type :
Conference papers
Complete list of metadata

Cited literature [38 references]  Display  Hide  Download

https://hal.inria.fr/inria-00173202
Contributor : Olivier Teytaud <>
Submitted on : Wednesday, September 19, 2007 - 2:15:45 PM
Last modification on : Wednesday, March 27, 2019 - 4:41:29 PM
Long-term archiving on: : Friday, April 9, 2010 - 2:28:16 AM

File

sefordp.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00173202, version 1

Collections

Citation

Olivier Teytaud, Sylvain Gelly. Non linear programming for stochastic dynamic programming. Icinco 2007, 2007, Angers, France. ⟨inria-00173202⟩

Share

Metrics

Record views

326

Files downloads

714