Skip to Main content Skip to Navigation
Conference papers

Fighting Boredom in Recommender Systems with Linear Reinforcement Learning

Romain Warlop 1 Alessandro Lazaric 2, 1 Jérémie Mary 3, 1
1 SEQUEL - Sequential Learning
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
Abstract : A common assumption in recommender systems (RS) is the existence of a best fixed recommendation strategy. Such strategy may be simple and work at the item level (e.g., in multi-armed bandit it is assumed one best fixed arm/item exists) or implement more sophisticated RS (e.g., the objective of A/B testing is to find the best fixed RS and execute it thereafter). We argue that this assumption is rarely verified in practice, as the recommendation process itself may impact the user's preferences. For instance, a user may get bored by a strategy, while she may gain interest again, if enough time passed since the last time that strategy was used. In this case, a better approach consists in alternating different solutions at the right frequency to fully exploit their potential. In this paper, we first cast the problem as a Markov decision process, where the rewards are a linear function of the recent history of actions, and we show that a policy considering the long-term influence of the recommendations may outperform both fixed-action and contextual greedy policies. We then introduce an extension of the UCRL algorithm (LINUCRL) to effectively balance exploration and exploitation in an unknown environment, and we derive a regret bound that is independent of the number of states. Finally, we empirically validate the model assumptions and the algorithm in a number of realistic scenarios.
Document type :
Conference papers
Complete list of metadatas

Cited literature [24 references]  Display  Hide  Download
Contributor : Romain Warlop <>
Submitted on : Wednesday, November 7, 2018 - 4:11:55 PM
Last modification on : Friday, December 11, 2020 - 6:44:05 PM
Long-term archiving on: : Friday, February 8, 2019 - 3:34:28 PM


Files produced by the author(s)


  • HAL Id : hal-01915468, version 1


Romain Warlop, Alessandro Lazaric, Jérémie Mary. Fighting Boredom in Recommender Systems with Linear Reinforcement Learning. Neural Information Processing Systems, Dec 2018, Montreal, Canada. ⟨hal-01915468⟩



Record views


Files downloads