Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Hierarchical-Task Reservoir for Online Semantic Analysis from Continuous Speech

Luca Pedrelli 1 Xavier Hinaut 1
1 Mnemosyne - Mnemonic Synergy
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest, IMN - Institut des Maladies Neurodégénératives [Bordeaux]
Abstract : In this paper, we propose a novel architecture called Hierarchical-Task Reservoir (HTR) suitable for real-time applications for which different levels of abstraction are available. We apply it to semantic role labelling based on continuous speech recognition. Taking inspiration from the brain, that demonstrates hierarchies of representations from perceptive to integrative areas, we consider a hierarchy of four sub-tasks with increasing levels of abstraction (phone, word, part-of-speech and semantic role tags). These tasks are progressively learned by the layers of the HTR architecture. Interestingly, quantitative and qualitative results show that the hierarchical-task approach provides an advantage to improve the prediction. In particular, the qualitative results show that a shallow or a hierarchical reservoir considered as baselines do not produce a quality of estimation as the HTR model. Moreover, we show that it is possible to further improve the accuracy of the model by designing skip connections and by considering word embedding in the internal representations. Overall, the HTR outperformed the other state-of-the-art reservoir-based approaches. The HTR architecture is proposed as a step toward the modeling of online and hierarchical processes at work in the brain during language comprehension. It is also developed for real-time and efficient Human-Robot Interaction (HRI) for which the availability of different levels of abstraction would provide more robustness.
Complete list of metadatas

https://hal.inria.fr/hal-03031413
Contributor : Xavier Hinaut <>
Submitted on : Monday, November 30, 2020 - 2:38:52 PM
Last modification on : Wednesday, December 2, 2020 - 1:32:16 PM

File

PedrelliHinaut2020_preprint_HA...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03031413, version 1

Collections

Citation

Luca Pedrelli, Xavier Hinaut. Hierarchical-Task Reservoir for Online Semantic Analysis from Continuous Speech. 2020. ⟨hal-03031413⟩

Share

Metrics

Record views

41

Files downloads

90