Constrained Markov Decision Processes

Abstract : This report presents a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs. We consider a single controller having several objectives; it is desirable to design a controller that minimize one of cost objective, subject to inequality constraints on other cost objectives. The objectives that we study are both the expected average cost, as well as the expected total cost (of which the discounted cost is a special case). We provide two frameworks: the case were costs are bounded below, as well as the contracting framework. We characterize the set of achievable expected occupation measures as well as performance vectors. This allows us to reduce the original control dynamic problem into an infinite Linear Programming. We present a Lagrangian approach that enables us to obtain sensitivity analysis. In particular, we obtain asymptotical results for the constrained control problem: convergence of both the value and the policies in the time horizon and in the discount factor. Finally, we present and several state truncation algorithms that enable to approximate the solution of the original control problem via finite linear programs.
Type de document :
Rapport
RR-2574, INRIA. 1995
Liste complète des métadonnées

https://hal.inria.fr/inria-00074109
Contributeur : Rapport de Recherche Inria <>
Soumis le : mercredi 24 mai 2006 - 14:32:27
Dernière modification le : samedi 27 janvier 2018 - 01:30:56
Document(s) archivé(s) le : lundi 17 septembre 2012 - 14:45:42

Fichiers

Identifiants

  • HAL Id : inria-00074109, version 1

Collections

Citation

Eitan Altman. Constrained Markov Decision Processes. RR-2574, INRIA. 1995. 〈inria-00074109〉

Partager

Métriques

Consultations de la notice

266

Téléchargements de fichiers

250