inria-00535560, version 1
A POMDP Extension with Belief-dependent Rewards
Neural Information Processing Systems - NIPS 2010 (2010)
Abstract: Partially Observable Markov Decision Processes (POMDPs) model sequential decision-making problems under uncertainty and partial observability. Unfortunately, some problems cannot be modeled with state-dependent reward functions, e.g., problems whose objective explicitly implies reducing the uncertainty on the state. To that end, we introduce ρPOMDPs, an extension of POMDPs where the reward function ρ depends on the belief state. We show that, under the common assumption that ρ is convex, the value function is also convex, what makes it possible to (1) approximate ρ arbitrarily well with a piecewise linear and convex (PWLC) function, and (2) use state-of-the-art exact or approximate solving algorithms with limited changes.
- a – INRIA
- b – Université Nancy II
- 1:
- INRIA – CNRS : UMR7503 – Université Henri Poincaré - Nancy I – Université Nancy II – Institut National Polytechnique de Lorraine (INPL)
- Domain : Computer Science/Artificial Intelligence
- Keywords : Partially Observable Markov Decision Processes – reward function – active sensing – piecewise linear and convex approximation
- Available versions : v1 (2010-11-12) v2 (2010-12-15)
- inria-00535560, version 1
- http://hal.inria.fr/inria-00535560
- oai:hal.inria.fr:inria-00535560
- From:
- Submitted on: Saturday, 11 December 2010 07:00:05
- Updated on: Saturday, 11 December 2010 07:00:05
Associated documents
Export