Skip to Main content Skip to Navigation
Conference papers

Temporal Aspects of CARE-based Multimodal Fusion: From a Fusion Mechanism to Composition Components and WoZ Components

Marcos Serrano 1 Laurence Nigay 1
1 IIHM - Ingénierie de l’Interaction Homme-Machine
LIG - Laboratoire d'Informatique de Grenoble
Abstract : The CARE properties (Complementarity, Assignment, Redundancy and Equivalence) define various forms that multimodal input interaction can take. While Equivalence and Assignment express the availability and respective absence of choice between multiple input modalities for performing a given task, Complementarity and Redundancy describe relationships between modalities and require fusion mechanisms. In this paper we present a summary of the works we have carried using the CARE properties for conceiving and implementing multimodal interaction, as well as a new approach using WoZ components. Firstly, we present different technical solutions for implementing the Complementarity and Redundancy of modalities with a focus on the temporal aspects of the fusion. Starting from a monolithic fusion mechanism, we then explain our component-based approach and the composition components (i.e., Redundancy and Complementarity components). As a new contribution for exploring solutions before implementing an adequate fusion mechanism as well as for tuning the temporal aspects of the performed fusion, we introduce Wizard of Oz (WoZ) fusion components. We illustrate the composition components as well as the implemented tools exploiting them using several multimodal systems including a multimodal slide viewer and a multimodal map navigator.
Document type :
Conference papers
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download

https://hal.inria.fr/hal-00953294
Contributor : Renaud Blanch <>
Submitted on : Friday, February 28, 2014 - 11:29:52 AM
Last modification on : Tuesday, November 24, 2020 - 4:34:03 PM
Long-term archiving on: : Wednesday, May 28, 2014 - 11:12:14 AM

File

ICMI09-Serrano.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00953294, version 1

Collections

Citation

Marcos Serrano, Laurence Nigay. Temporal Aspects of CARE-based Multimodal Fusion: From a Fusion Mechanism to Composition Components and WoZ Components. Proceedings of the 11th international conference on Multimodal interfaces (ICMI '09), 2009, Cambridge, United States. pp.177-184. ⟨hal-00953294⟩

Share

Metrics

Record views

220

Files downloads

193