Computable approximations for average Markov decision processes in continuous time

Abstract : In this paper we study the numerical approximation of the optimal long-run average cost of a continuous-time Markov decision process, with Borel state and action spaces, and with bounded transition and reward rates. Our approach uses a suitable discretization of the state and action spaces to approximate the original control model. The approximation error for the optimal average reward is then bounded by a linear combination of coefficients related to the discretization of the state and action spaces, namely, the Wasserstein distance between an underlying probability measure μ and a measure with finite support, and the Hausdorff distance between the original and the discretized actions sets. When approximating μ with its empirical probability measure we obtain convergence in probability at an exponential rate. An application to a queueing system is presented.
Document type :
Journal articles
Complete list of metadatas

https://hal.inria.fr/hal-01949945
Contributor : Jonatha Anselmi <>
Submitted on : Monday, December 10, 2018 - 2:46:49 PM
Last modification on : Thursday, February 7, 2019 - 5:26:04 PM

Identifiers

Collections

Citation

Jonatha Anselmi, François Dufour, Tomás Prieto-Rumeau. Computable approximations for average Markov decision processes in continuous time. Journal of Applied Probability, Applied Probability Trust, 2018, 55 (02), pp.571-592. ⟨10.1017/jpr.2018.36⟩. ⟨hal-01949945⟩

Share

Metrics

Record views

44