HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Reports

Sensitivity Analysis in Particle Filters. Application to Policy Optimization in POMDPs

Pierre Arnaud Coquelin 1 Romain Deguest 1 Rémi Munos 2
2 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : Our setting is a Partially Observable Markov Decision Process with continuous state, observation and action spaces. Decisions are based on a Particle Filter for estimating the belief state given past observations. We consider a policy gradient approach for parameterized policy optimization. For that purpose, we investigate sensitivity analysis of the performance measure with respect to the parameters of the policy, focusing on Finite Difference (FD) techniques. We show that the naive FD is subject to variance explosion because of the non-smoothness of the resampling procedure. We propose a more sophisticated FD method which overcomes this problem and establish its consistency.
Document type :
Reports
Complete list of metadata

Cited literature [2 references]  Display  Hide  Download

https://hal.inria.fr/inria-00336203
Contributor : Rémi Munos Connect in order to contact the contributor
Submitted on : Monday, November 3, 2008 - 11:14:21 AM
Last modification on : Thursday, January 20, 2022 - 4:16:31 PM
Long-term archiving on: : Monday, June 7, 2010 - 10:39:22 PM

File

RR6710.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00336203, version 1

Citation

Pierre Arnaud Coquelin, Romain Deguest, Rémi Munos. Sensitivity Analysis in Particle Filters. Application to Policy Optimization in POMDPs. [Research Report] RR-6710, INRIA. 2008. ⟨inria-00336203⟩

Share

Metrics

Record views

171

Files downloads

87