A Partially-Observable Markov Decision Process for Dealing with Dynamically Changing Environments

Abstract : Partially Observable Markov Decision Processes (POMDPs) have been met with great success in planning domains where agents must balance actions that provide knowledge and actions that provide reward. Recently, nonparametric Bayesian methods have been successfully applied to POMDPs to obviate the need of a priori knowledge of the size of the state space, allowing to assume that the number of visited states may grow as the agent explores its environment. These approaches rely on the assumption that the agent’s environment remains stationary; however, in real-world scenarios the environment may change over time. In this work, we aim to address this inadequacy by introducing a dynamic nonparametric Bayesian POMDP model that both allows for automatic inference of the (distributional) representations of POMDP states, and for capturing non-stationarity in the modeled environments. Formulation of our method is based on imposition of a suitable dynamic hierarchical Dirichlet process (dHDP) prior over state transitions. We derive efficient algorithms for model inference and action planning and evaluate it on several benchmark tasks.
Type de document :
Communication dans un congrès
Lazaros Iliadis; Ilias Maglogiannis; Harris Papadopoulos. 10th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Sep 2014, Rhodes, Greece. Springer, IFIP Advances in Information and Communication Technology, AICT-436, pp.111-120, 2014, Artificial Intelligence Applications and Innovations. 〈10.1007/978-3-662-44654-6_11〉
Liste complète des métadonnées

Littérature citée [16 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01391299
Contributeur : Hal Ifip <>
Soumis le : jeudi 3 novembre 2016 - 10:52:13
Dernière modification le : vendredi 1 décembre 2017 - 01:16:37
Document(s) archivé(s) le : samedi 4 février 2017 - 12:54:49

Fichier

978-3-662-44654-6_11_Chapter.p...
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Sotirios Chatzis, Dimitrios Kosmopoulos. A Partially-Observable Markov Decision Process for Dealing with Dynamically Changing Environments. Lazaros Iliadis; Ilias Maglogiannis; Harris Papadopoulos. 10th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Sep 2014, Rhodes, Greece. Springer, IFIP Advances in Information and Communication Technology, AICT-436, pp.111-120, 2014, Artificial Intelligence Applications and Innovations. 〈10.1007/978-3-662-44654-6_11〉. 〈hal-01391299〉

Partager

Métriques

Consultations de la notice

25

Téléchargements de fichiers

48