Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2010

Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats

Résumé

Goal directed action involves a representation of the consequences of an action. Rats with lesions of the medial prefrontal cortex do not adapt their instrumental response in a Skinner box when food delivery becomes unrelated to lever pressing. This indicates a role for the prefrontal region in adapting to contingency changes, a form of causal learning. We attempted to model this phenomenon in a reinforcement learning framework. Behavioural sequences of normal and lesioned rats were used to feed models based on the SARSA algorithm. One model (factorized-states) focused on temporal factors, representing continuous states as vectors of decaying event traces. The second model (event sequence) emphasized sequences, representing states as n-uplets of events. The values of state-action pairs were incorporated into a softmax policy to derive predicted action probabilities and adjust model parameters. Both models revealed a number of discrepancies between predicted and actual behaviour, emphasising changes in magazine visits rather that lever presses. The models also did not reproduce the differential adaptation of normal and prefrontal lesioned rats to contingency degradation. These data suggest that temporal difference learning models fail to capture causal relationships involved in the adaptation to contingency changes.

Mots clés

Fichier principal
Vignette du fichier
DutechCoutureauMarchand_RLInstrumentalContingency_NeuroComp10.pdf (376.59 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

inria-00517011 , version 1 (13-09-2010)

Identifiants

  • HAL Id : inria-00517011 , version 1

Citer

Alain Dutech, Etienne Coutureau, Alain Marchand. Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats. Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. ⟨inria-00517011⟩
279 Consultations
277 Téléchargements

Partager

Gmail Facebook X LinkedIn More