A kernel-based approach to non-stationary reinforcement learning in metric spaces - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

A kernel-based approach to non-stationary reinforcement learning in metric spaces

Omar D Domingues
  • Fonction : Auteur
Emilie Kaufmann
Michal Valko

Résumé

In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in nonstationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.
Fichier principal
Vignette du fichier
domingues2021kernel-based-non-stationary.pdf (12.85 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03289026 , version 1 (16-07-2021)

Identifiants

  • HAL Id : hal-03289026 , version 1

Citer

Omar D Domingues, Pierre Ménard, Matteo Pirotta, Emilie Kaufmann, Michal Valko. A kernel-based approach to non-stationary reinforcement learning in metric spaces. International Conference on Artificial Intelligence and Statistics, Apr 2021, San Diego / Virtual, United States. ⟨hal-03289026⟩
96 Consultations
77 Téléchargements

Partager

Gmail Facebook X LinkedIn More