3526 articles – 5249 Notices  [english version]

inria-00320645, version 1

Using linear programming duality for solving finite horizon Dec-POMDPs

Raghav Aras () 1, Alain Dutech () 1, François Charpillet () 1

N° RR-6641 (2008)

Résumé : This paper studies the problem of finding an optimal finite horizon joint policy for a decentralized partially observable Markov decision process (Dec-POMDP). We present a new algorithm for finding an optimal joint policy. The algorithm is based on the fact that the necessary condition for a joint policy to be optimal is that it be locally optimal (that is, a Nash equilibrium). Through the application of linear programming duality, the necessary condition can be transformed to a nonlinear program which can then further be transformed to a 0-1 mixed integer linear program (MILP) whose optimal solution is an optimal joint policy (in the sequence form). The proposed algorithm thus consists of solving this 0-1 MILP. Computational experience of the 0-1 MILP on two and three agent DEC-POMDPs gives mixed results. On some problems it is faster than existing algorithms, on others it is slower.

  • 1 :  MAIA (INRIA Lorraine - LORIA)
  • INRIA – CNRS : UMR7503 – Université Henri Poincaré - Nancy I – Université Nancy II – Institut National Polytechnique de Lorraine (INPL)
  • Domaine : Informatique/Système multi-agents
    Informatique/Informatique et théorie des jeux
    Informatique/Recherche opérationnelle
  • Mots-clés : Dec-POMDPs – decentralized problems
  • Référence interne : RR-6641
 
  • inria-00320645, version 1
  • oai:hal.inria.fr:inria-00320645
  • Contributeur : 
  • Soumis le : Mardi 16 Septembre 2008, 14:49:56
  • Dernière modification le : Mardi 30 Septembre 2008, 14:02:51