Acting on a visual world : the role of perception in multimodal HCI

Frédéric Wolff 1 Antonella de Angeli 1, 2 Laurent Romary 1
1 LANGUE ET DIALOGUE - Human-machine dialogue with a significant language component
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
2 CTL - Cognitive Technology Laboratory, Department of Psychology
Abstract : Following the ecological theory of perception, this paper investigates multimodal referring acts in Human-Computer Interaction. Preliminary results from a simulation experiment allow to: (a) clarify the effect of perceptual organization on multimodal communication; (b) provide guidelines for designing effective multimodal interfaces. Demonstrating that both the verbal and the gestural part of a referential act are influenced by perception, we confirm the need and the utility of taking into account perceptual organization to analyze referential expressions in a more robust way. As a conclusion, we show how these results can lead to an actual implementation of a gesture interpretation module directed by the reference analysis process.
keyword : geste Wizard of OZ gesture multimodal reference multimodale Magicien d'Oz
Document type :
Conference papers
Complete list of metadatas

Cited literature [14 references]  Display  Hide  Download
https://hal.inria.fr/inria-00107515
Contributor : Laurent Romary <>
Submitted on : Monday, March 1, 2010 - 12:23:25 PM
Last modification on : Friday, March 22, 2019 - 2:22:12 PM
Long-term archiving on : Thursday, September 23, 2010 - 11:57:35 AM

Files

AAAIBIS.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : inria-00107515, version 2

Collections

INRIA | LORIA | UNIV-LORRAINE

Citation

Frédéric Wolff, Antonella de Angeli, Laurent Romary. Acting on a visual world : the role of perception in multimodal HCI. AAAI '98 Workshop: Representations for Multi-modal Human-Computer Interaction, Jul 1998, Madison, Wisconsin, United States. 6 p. ⟨inria-00107515v2⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

193

Files downloads

152

Contact


archive-ouverte@inria.fr