Skip to Main content Skip to Navigation
Conference papers

Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling

Alexis Juven 1 Xavier Hinaut 1 
1 Mnemosyne - Mnemonic Synergy
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest, IMN - Institut des Maladies Neurodégénératives [Bordeaux]
Abstract : Understanding the mechanisms enabling children to learn rapidly word-to-meaning mapping through cross-situational learning in uncertain conditions is still a matter of debate. In particular, many models simply look at the word level, and not at the full sentence comprehension level. We present a model of language acquisition, applying cross-situational learning on Recurrent Neural Networks with the Reservoir Computing paradigm. Using the co-occurrences between words and visual perceptions, the model learns to ground a complex sentence, describing a scene involving different objects, into a perceptual representation space. The model processes sentences describing scenes it perceives simultaneously via a simulated vision module: sentences are inputs and simulated vision are target outputs of the RNN. Evaluations of the model show its capacity to extract the semantics of virtually hundred of thousands possible combinations of sentences (based on a context-free grammar); remarkably the model generalises only after a few hundred of partially described scenes via cross-situational learning. Furthermore, it handles polysemous and synonymous words, and deals with complex sentences where word order is crucial for understanding. Finally, further improvements of the model are discussed in order to reach proper reinforced and self-supervised learning schemes, with the goal to enable robots to acquire and ground language by themselves (with no oracle supervision).
Complete list of metadata

Cited literature [26 references]  Display  Hide  Download
Contributor : Xavier Hinaut Connect in order to contact the contributor
Submitted on : Friday, May 15, 2020 - 7:13:04 PM
Last modification on : Saturday, July 23, 2022 - 3:53:07 AM


Files produced by the author(s)


  • HAL Id : hal-02594725, version 1


Alexis Juven, Xavier Hinaut. Cross-Situational Learning with Reservoir Computing for Language Acquisition Modelling. 2020 International Joint Conference on Neural Networks (IJCNN 2020), Jul 2020, Glasgow, Scotland, United Kingdom. ⟨hal-02594725⟩



Record views


Files downloads