Defining and Quantifying Users' Mental Imagery-based BCI skills: a first step

Fabien Lotte 1 Camille Jeunet 2, 3
1 Potioc - Popular interaction with 3d content
LaBRI - Laboratoire Bordelais de Recherche en Informatique, Inria Bordeaux - Sud-Ouest
2 Hybrid - 3D interaction with virtual environments using body and mind
Inria Rennes – Bretagne Atlantique , IRISA_D6 - MEDIA ET INTERACTIONS
Abstract : Objective: While promising for many applications, Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are still scarcely used outside laboratories , due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires the use of appropriate reliability metrics to quantify both the classification algorithm and the BCI user's performances. So far, Classification Accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study BCI users' skills. Here, we propose a definition and new metrics to quantify such BCI skills for Mental Imagery (MI) BCIs, independently of any classification algorithm. Approach: We first show in this paper that CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful self-modulation of EEG patterns by the user. We then propose a definition of MI-BCI skills that reflects how well the user can self-modulate EEG patterns, and thus how well he could control an MI-BCI. Finally, we propose new performance metrics, classDis, restDist and classStab that specifically measure how distinct and stable the EEG patterns produced by the user are, independently of any classifier. Main results: By re-analyzing EEG data sets with such new metrics, we indeed confirmed that CA may hide some increase in MI-BCI skills or hide the user inability to self-modulate a given EEG pattern. On the other hand, our new met-rics could reveal such skill improvements as well as identify when a mental task performed by a user was no different than rest EEG. Significance: Our results showed that when studying MI-BCI users' skills, CA should be used with care, and complemented with metrics such as the new ones proposed. Our results also stressed the need to redefine BCI user training by considering the different BCI subskills and their measures. To promote the complementary use of our new metrics, we provide the Matlab code to compute them for free and open-source.
Complete list of metadatas

Cited literature [17 references]  Display  Hide  Download

https://hal.inria.fr/hal-01846434
Contributor : Fabien Lotte <>
Submitted on : Saturday, July 21, 2018 - 11:37:50 PM
Last modification on : Friday, May 10, 2019 - 1:42:19 AM
Long-term archiving on : Monday, October 22, 2018 - 1:36:36 PM

File

JNE18_MetricsBCISkills_revised...
Files produced by the author(s)

Identifiers

Citation

Fabien Lotte, Camille Jeunet. Defining and Quantifying Users' Mental Imagery-based BCI skills: a first step. Journal of Neural Engineering, IOP Publishing, 2018, 15 (4), pp.1-37. ⟨10.1088/1741-2552/aac577⟩. ⟨hal-01846434⟩

Share

Metrics

Record views

299

Files downloads

241