Multimodal Observation and Interpretation of Subjects Engaged in Problem Solving

Abstract : In this paper we present the first results of a pilot experiment in the capture and interpretation of multimodal signals of human experts engaged in solving challenging chess problems. Our goal is to investigate the extent to which observations of eye-gaze, posture, emotion and other physiological signals can be used to model the cognitive state of subjects, and to explore the integration of multiple sensor modalities to improve the reliability of detection of human displays of awareness and emotion. We observed chess players engaged in problems of increasing difficulty while recording their behavior. Such recordings can be used to estimate a participant's awareness of the current situation and to predict ability to respond effectively to challenging situations. Results show that a multimodal approach is more accurate than a unimodal one. By combining body posture, visual attention and emotion, the multimodal approach can reach up to 93% of accuracy when determining player's chess expertise while unimodal approach reaches 86%. Finally this experiment validates the use of our equipment as a general and reproducible tool for the study of participants engaged in screen-based interaction and/or problem solving.
Type de document :
Communication dans un congrès
1st Workshop on “Behavior, Emotion and Representation: Building Blocks of Interaction”, Oct 2017, Bielefeld, Germany. 2017
Liste complète des métadonnées

Littérature citée [29 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01615461
Contributeur : Dominique Vaufreydaz <>
Soumis le : jeudi 12 octobre 2017 - 14:52:54
Dernière modification le : mercredi 11 avril 2018 - 01:55:17

Fichiers

main.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01615461, version 1
  • ARXIV : 1710.04486

Citation

Thomas Guntz, Raffaella Balzarini, Dominique Vaufreydaz, James L. Crowley. Multimodal Observation and Interpretation of Subjects Engaged in Problem Solving. 1st Workshop on “Behavior, Emotion and Representation: Building Blocks of Interaction”, Oct 2017, Bielefeld, Germany. 2017. 〈hal-01615461〉

Partager

Métriques

Consultations de la notice

305

Téléchargements de fichiers

41