Skip to Main content Skip to Navigation
Conference papers

Momentum in Reinforcement Learning

Abstract : We adapt the optimization's concept of momentum to reinforcement learning. Seeing the state-action value functions as an analog to the gradients in optimization, we interpret momentum as an average of consecutive q-functions. We derive Momentum Value Iteration (MoVI), a variation of Value iteration that incorporates this momentum idea. Our analysis shows that this allows MoVI to average errors over successive iterations. We show that the proposed approach can be readily extended to deep learning. Specifically,we propose a simple improvement on DQN based on MoVI, and experiment it on Atari games.
Document type :
Conference papers
Complete list of metadata

https://hal.inria.fr/hal-03137343
Contributor : Bruno Scherrer Connect in order to contact the contributor
Submitted on : Wednesday, February 10, 2021 - 1:37:57 PM
Last modification on : Wednesday, November 3, 2021 - 4:47:12 AM
Long-term archiving on: : Tuesday, May 11, 2021 - 6:36:26 PM

File

vieillard20a-supp.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03137343, version 1

Collections

Citation

Nino Vieillard, Bruno Scherrer, Olivier Pietquin, Matthieu Geist. Momentum in Reinforcement Learning. AISTATS 2020 - 23rd International Conference on Artificial Intelligence and Statistics, Aug 2020, Palermo / Virtual, Italy. ⟨hal-03137343⟩

Share

Metrics

Les métriques sont temporairement indisponibles