Skip to Main content Skip to Navigation
Conference papers

Learning to Parse Grounded Language using Reservoir Computing

Xavier Hinaut 1 Michael Spranger 2
1 Mnemosyne - Mnemonic Synergy
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest, IMN - Institut des Maladies Neurodégénératives [Bordeaux]
Abstract : Recently new models for language processing and learning using Reservoir Computing have been popular. However, these models are typically not grounded in sensorimotor systems and robots. In this paper, we develop a model of Reservoir Computing called Reservoir Parser (ResPars) for learning to parse Natural Language from grounded data coming from humanoid robots. Previous work showed that ResPars is able to do syntactic generalization over different sentences (surface structure) with the same meaning (deep structure). We argue that such ability is key to guide linguistic generalization in a grounded architecture. We show that ResPars is able to generalize on grounded compositional semantics by combining it with Incremental Recruitment Language (IRL). Additionally, we show that ResPars is able to learn to generalize on the same sentences, but not processed word by word, but as an unsegmented sequence of phonemes. This ability enables the architecture to not rely only on the words recognized by a speech recognizer, but to process the sub-word level directly. We additionally test the model's robustness to word error recognition.
Complete list of metadata

Cited literature [35 references]  Display  Hide  Download

https://hal.inria.fr/hal-02422157
Contributor : Xavier Hinaut <>
Submitted on : Friday, December 20, 2019 - 7:22:38 PM
Last modification on : Monday, January 6, 2020 - 4:30:17 PM
Long-term archiving on: : Saturday, March 21, 2020 - 8:42:42 PM

File

HinautSpranger2019_ICDL_camera...
Files produced by the author(s)

Identifiers

Collections

Citation

Xavier Hinaut, Michael Spranger. Learning to Parse Grounded Language using Reservoir Computing. ICDL-Epirob 2019 - Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics, Aug 2019, Olso, Norway. ⟨10.1109/devlrn.2019.8850718⟩. ⟨hal-02422157⟩

Share

Metrics

Record views

116

Files downloads

440