HSVI Can Solve Zero-Sum Partially Observable Stochastic Games - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Dynamic Games and Applications Année : 2023

HSVI Can Solve Zero-Sum Partially Observable Stochastic Games

Résumé

State-of-the-art methods for solving 2-player zero-sum imperfect information games rely on linear programming or regret minimization, though not on dynamic programming (DP) or heuristic search (HS), while the latter are often at the core of state-of-the-art solvers for other sequential decision-making problems. In partially observable or collaborative settings (e.g., POMDPs and Dec-POMDPs), DP and HS require introducing an appropriate statistic that induces a fully observable problem as well as bounding (convex) approximators of the optimal value function. This approach has succeeded in some subclasses of 2-player zero-sum partially observable stochastic games (zs-POSGs) as well, but how to apply it in the general case still remains an open question. We answer it by (i) rigorously defining an equivalent game to work with, (ii) proving mathematical properties of the optimal value function that allow deriving bounds that come with solution strategies, (iii) proposing for the first time an HSVI-like solver that provably converges to an $\epsilon$-optimal solution in finite time, and (iv) empirically analyzing it. This opens the door to a novel family of promising approaches complementing those relying on linear programming or iterative methods.
Fichier sous embargo
Fichier sous embargo
0 4 14
Année Mois Jours
Avant la publication
lundi 2 septembre 2024
Fichier sous embargo
lundi 2 septembre 2024
Connectez-vous pour demander l'accès au fichier

Dates et versions

hal-04382756 , version 1 (11-01-2023)
hal-04382756 , version 2 (16-01-2024)

Licence

Paternité

Identifiants

Citer

Aurélien Delage, Olivier Buffet, Jilles Dibangoye, Abdallah Saffidine. HSVI Can Solve Zero-Sum Partially Observable Stochastic Games. Dynamic Games and Applications, 2023, pp.58. ⟨10.1007/s13235-023-00519-6⟩. ⟨hal-04382756v2⟩
76 Consultations
77 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More