Temporal Aspects of CARE-based Multimodal Fusion: From a Fusion Mechanism to Composition Components and WoZ Components

Marcos Serrano 1 Laurence Nigay 1
1 LIG Laboratoire d'Informatique de Grenoble - IIHM
LIG - Laboratoire d'Informatique de Grenoble, Inria - Institut National de Recherche en Informatique et en Automatique
Abstract : The CARE properties (Complementarity, Assignment, Redundancy and Equivalence) define various forms that multimodal input interaction can take. While Equivalence and Assignment express the availability and respective absence of choice between multiple input modalities for performing a given task, Complementarity and Redundancy describe relationships between modalities and require fusion mechanisms. In this paper we present a summary of the works we have carried using the CARE properties for conceiving and implementing multimodal interaction, as well as a new approach using WoZ components. Firstly, we present different technical solutions for implementing the Complementarity and Redundancy of modalities with a focus on the temporal aspects of the fusion. Starting from a monolithic fusion mechanism, we then explain our component-based approach and the composition components (i.e., Redundancy and Complementarity components). As a new contribution for exploring solutions before implementing an adequate fusion mechanism as well as for tuning the temporal aspects of the performed fusion, we introduce Wizard of Oz (WoZ) fusion components. We illustrate the composition components as well as the implemented tools exploiting them using several multimodal systems including a multimodal slide viewer and a multimodal map navigator.
Type de document :
Communication dans un congrès
Proceedings of the 11th international conference on Multimodal interfaces (ICMI '09), 2009, Cambridge, United States. ACM, pp.177-184, 2009
Liste complète des métadonnées

Littérature citée [19 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00953294
Contributeur : Renaud Blanch <>
Soumis le : vendredi 28 février 2014 - 11:29:52
Dernière modification le : jeudi 11 janvier 2018 - 01:49:05
Document(s) archivé(s) le : mercredi 28 mai 2014 - 11:12:14

Fichier

ICMI09-Serrano.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00953294, version 1

Collections

Citation

Marcos Serrano, Laurence Nigay. Temporal Aspects of CARE-based Multimodal Fusion: From a Fusion Mechanism to Composition Components and WoZ Components. Proceedings of the 11th international conference on Multimodal interfaces (ICMI '09), 2009, Cambridge, United States. ACM, pp.177-184, 2009. 〈hal-00953294〉

Partager

Métriques

Consultations de la notice

136

Téléchargements de fichiers

121