Skip to Main content Skip to Navigation
Conference papers

A kernel-based approach to non-stationary reinforcement learning in metric spaces

Abstract : In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in nonstationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03289026
Contributor : Michal Valko Connect in order to contact the contributor
Submitted on : Friday, July 16, 2021 - 4:06:57 PM
Last modification on : Tuesday, June 14, 2022 - 11:58:48 AM
Long-term archiving on: : Sunday, October 17, 2021 - 7:06:12 PM

File

domingues2021kernel-based-non-...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03289026, version 1

Collections

Citation

Omar D Domingues, Pierre Ménard, Matteo Pirotta, Emilie Kaufmann, Michal Valko. A kernel-based approach to non-stationary reinforcement learning in metric spaces. International Conference on Artificial Intelligence and Statistics, Apr 2021, San Diego / Virtual, United States. ⟨hal-03289026⟩

Share

Metrics

Record views

61

Files downloads

35