Sensitivity Analysis in Particle Filters. Application to Policy Optimization in POMDPs

Pierre Arnaud Coquelin 1 Romain Deguest 1 Rémi Munos 2
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal, Inria Lille - Nord Europe
Abstract : Our setting is a Partially Observable Markov Decision Process with continuous state, observation and action spaces. Decisions are based on a Particle Filter for estimating the belief state given past observations. We consider a policy gradient approach for parameterized policy optimization. For that purpose, we investigate sensitivity analysis of the performance measure with respect to the parameters of the policy, focusing on Finite Difference (FD) techniques. We show that the naive FD is subject to variance explosion because of the non-smoothness of the resampling procedure. We propose a more sophisticated FD method which overcomes this problem and establish its consistency.
Type de document :
Rapport
[Research Report] RR-6710, INRIA. 2008
Liste complète des métadonnées

Littérature citée [2 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00336203
Contributeur : Rémi Munos <>
Soumis le : lundi 3 novembre 2008 - 11:14:21
Dernière modification le : jeudi 10 mai 2018 - 02:04:21
Document(s) archivé(s) le : lundi 7 juin 2010 - 22:39:22

Fichier

RR6710.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00336203, version 1

Citation

Pierre Arnaud Coquelin, Romain Deguest, Rémi Munos. Sensitivity Analysis in Particle Filters. Application to Policy Optimization in POMDPs. [Research Report] RR-6710, INRIA. 2008. 〈inria-00336203〉

Partager

Métriques

Consultations de la notice

303

Téléchargements de fichiers

104