Solving infinite-horizon Dec-POMDPs using Finite State Controllers within JESP - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year :

Solving infinite-horizon Dec-POMDPs using Finite State Controllers within JESP

(1) , (1) , (1) , (1)
1

Abstract

This paper looks at solving collaborative planning problems formalized as Decentralized POMDPs (Dec-POMDPs) by searching for Nash equilibria, i.e., situations where each agent's policy is a best response to the other agents' (fixed) policies. While the Joint Equilibrium-based Search for Policies (JESP) algorithm does this in the finite-horizon setting relying on policy trees, we propose here to adapt it to infinite-horizon Dec-POMDPs by using finite state controller (FSC) policy representations. In this article, we (1) explain how to turn a Dec-POMDP with $N-1$ fixed FSCs into an infinite-horizon POMDP whose solution is an $N^\text{th}$ agent best response; (2) propose a JESP variant, called \infJESP, using this to solve infinite-horizon Dec-POMDPs; (3) introduce heuristic initializations for JESP aiming at leading to good solutions; and (4) conduct experiments on state-of-the-art benchmark problems to evaluate our approach.

Dates and versions

hal-03523415 , version 1 (12-01-2022)

Identifiers

Cite

Yang You, Vincent Thomas, Francis Colas, Olivier Buffet. Solving infinite-horizon Dec-POMDPs using Finite State Controllers within JESP. 2021. ⟨hal-03523415⟩
21 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More