ML & VR

ML-Driven Cognitive Workload Estimation in a VR-based Sustained Attention Task

Dominik Szczepaniak, Monika Harvey, Fani Deligianni,

Social AI CDT, Bio-AIm Lab, School of Computing Science, School of Psychology
University of Glasgow
ISMAR 2024

Abstract

Cognitive training can boost and sharpen the brain’s abilities to remember, focus, and switch between different tasks. One of the key elements of cognitive training is that it manipulates 'cognitive load', by adjusting the intensity of the intervention to suit the participant’s ability level and keep the session enjoyable. This study introduces a novel sustained attention task in Virtual Reality (VR) to predict cognitive load dynamically. Unlike previous research, which often used non-VR settings, simpler tasks, or performance metrics as predictors, our approach aims to measure cognitive load objectively in a more ecologically valid and gamified VR environment. We employed machine learning techniques to enable real-time, personalized cognitive training. This work contributes to the development of more effective cognitive training interventions that can adapt to individual differences and maintain optimal engagement levels.

The main contributions of our work are as follows:

Introducing sustained attention training into a Virtual Reality environment while collecting eye-tracking and physiological data.
Designing an ecologically valid task that does not require specialist equipment and can be used at home by the users.
Training classifiers to identify both objective difficulty changes as well as perceived cognitive load as experienced by participants.

High level overview of the relationship between attentional capacity and selectivity. The balance between both is key to designing effective and engaging interventions targetting attention.

In-game view of progressively increasing difficulties.

The frequencies of perceived difficulties mapped over objective difficulties (objective difficulty is balanced for each participant).

barcharts of accuracies within participants

Model performance for different validation protocols comparing objective and perceived difficulty predictions using different combinations of data types (Within Subject Cross-Validation Protocol (same participants seen during testing and training)).

barcharts of accuracies across participants

Model performance for different validation protocols comparing objective and perceived difficulty predictions using different combinations of data types (Across Subject Cross-Validation Protocol (participants left out during training and seen only during validation and testing)).

Receiver Operating Characteristic Curves for objective difficulty (left) and perceived difficulty (right) prediction using SVM (eye-tracking data only).

Poster

BibTeX

@article{Dominik2024,
  title={ML-Driven Cognitive Workload Estimation in a VR-based Sustained Attention Task},
  author={Szczepaniak, Dominik and Harvey, Monika and Deligianni, Fani},
  journal={IEEE International Symposium on Mixed and Augmented Reality},
  year={2024}
}