Skip to Main content Skip to Navigation
Conference papers

Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats

Alain Dutech 1 Etienne Coutureau 2 Alain Marchand 2, * 
* Corresponding author
1 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Goal directed action involves a representation of the consequences of an action. Rats with lesions of the medial prefrontal cortex do not adapt their instrumental response in a Skinner box when food delivery becomes unrelated to lever pressing. This indicates a role for the prefrontal region in adapting to contingency changes, a form of causal learning. We attempted to model this phenomenon in a reinforcement learning framework. Behavioural sequences of normal and lesioned rats were used to feed models based on the SARSA algorithm. One model (factorized-states) focused on temporal factors, representing continuous states as vectors of decaying event traces. The second model (event sequence) emphasized sequences, representing states as n-uplets of events. The values of state-action pairs were incorporated into a softmax policy to derive predicted action probabilities and adjust model parameters. Both models revealed a number of discrepancies between predicted and actual behaviour, emphasising changes in magazine visits rather that lever presses. The models also did not reproduce the differential adaptation of normal and prefrontal lesioned rats to contingency degradation. These data suggest that temporal difference learning models fail to capture causal relationships involved in the adaptation to contingency changes.
Keywords : Action selection
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download
Contributor : Alain Dutech Connect in order to contact the contributor
Submitted on : Monday, September 13, 2010 - 1:39:49 PM
Last modification on : Saturday, June 25, 2022 - 7:46:53 PM
Long-term archiving on: : Tuesday, December 14, 2010 - 2:46:26 AM


Files produced by the author(s)


  • HAL Id : inria-00517011, version 1



Alain Dutech, Etienne Coutureau, Alain Marchand. Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats. Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. ⟨inria-00517011⟩



Record views


Files downloads