Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats

Alain Dutech 1 Etienne Coutureau 2 Alain Marchand 2, *
* Auteur correspondant
1 MAIA - Autonomous intelligent machine
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : Goal directed action involves a representation of the consequences of an action. Rats with lesions of the medial prefrontal cortex do not adapt their instrumental response in a Skinner box when food delivery becomes unrelated to lever pressing. This indicates a role for the prefrontal region in adapting to contingency changes, a form of causal learning. We attempted to model this phenomenon in a reinforcement learning framework. Behavioural sequences of normal and lesioned rats were used to feed models based on the SARSA algorithm. One model (factorized-states) focused on temporal factors, representing continuous states as vectors of decaying event traces. The second model (event sequence) emphasized sequences, representing states as n-uplets of events. The values of state-action pairs were incorporated into a softmax policy to derive predicted action probabilities and adjust model parameters. Both models revealed a number of discrepancies between predicted and actual behaviour, emphasising changes in magazine visits rather that lever presses. The models also did not reproduce the differential adaptation of normal and prefrontal lesioned rats to contingency degradation. These data suggest that temporal difference learning models fail to capture causal relationships involved in the adaptation to contingency changes.
Keywords : Action selection
Type de document :
Communication dans un congrès
Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. 2010
Liste complète des métadonnées

Littérature citée [19 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/inria-00517011
Contributeur : Alain Dutech <>
Soumis le : lundi 13 septembre 2010 - 13:39:49
Dernière modification le : jeudi 11 janvier 2018 - 06:21:10
Document(s) archivé(s) le : mardi 14 décembre 2010 - 02:46:26

Fichier

DutechCoutureauMarchand_RLInst...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : inria-00517011, version 1

Collections

Citation

Alain Dutech, Etienne Coutureau, Alain Marchand. Reinforcement Learning Approaches to Instrumental Contingency Degradation in Rats. Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. 2010. 〈inria-00517011〉

Partager

Métriques

Consultations de la notice

439

Téléchargements de fichiers

135