Multimodal Observation and Classification of People Engaged in Problem Solving: Application to Chess Players

Thomas Guntz 1 Raffaella Balzarini 1 Dominique Vaufreydaz 1 James L. Crowley 1
1 PERVASIVE - Interaction située avec les objets et environnements intelligents
Inria Grenoble - Rhône-Alpes, Université de Grenoble-Alpes, LIG - Laboratoire d'Informatique de Grenoble, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
Abstract : In this paper we present the first results of a pilot experiment in the interpretation of multimodal observations of human experts engaged in solving challenging chess problems. Our goal is to investigate the extent to which observations of eye-gaze, posture, emotion and other physiological signals can be used to model the cognitive state of subjects, and to explore the integration of multiple sensor modalities to improve the reliability of detection of human displays of awareness and emotion. Domains of application for such cognitive model based systems are, for instance, healthy autonomous ageing or automated training systems. Abilities to observe cognitive abilities and emotional reactions can allow artificial systems to provide appropriate assistance in such contexts. We observed chess players engaged in problems of increasing difficulty while recording their behavior. Such recordings can be used to estimate a participant's awareness of the current situation and to predict ability to respond effectively to challenging situations. Feature selection has been performed to construct a multimodal classifier relying on the most relevant features from each modality. Initial results indicate that eye-gaze, body posture and emotion are good features to capture such awareness. This experiment also validates the use of our equipment as a general and reproducible tool for the study of participants engaged in screen-based interaction and/or problem solving.
Type de document :
Article dans une revue
Multimodal Technologies and Interaction, 2018, 2 (2), 〈10.3390/mti2020011〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01886354
Contributeur : Dominique Vaufreydaz <>
Soumis le : mardi 2 octobre 2018 - 19:49:52
Dernière modification le : jeudi 11 octobre 2018 - 08:48:05

Fichier

mti-02-00011.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Thomas Guntz, Raffaella Balzarini, Dominique Vaufreydaz, James L. Crowley. Multimodal Observation and Classification of People Engaged in Problem Solving: Application to Chess Players. Multimodal Technologies and Interaction, 2018, 2 (2), 〈10.3390/mti2020011〉. 〈hal-01886354〉

Partager

Métriques

Consultations de la notice

77

Téléchargements de fichiers

16