OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

A neuroergonomic approach to performance estimation in a psychomotor vigilance task

Jahanpour, Emilie and Xu, Xiaoqi and Hinss, Marcel F. and Drougard, Nicolas and Roy, Raphaëlle N. A neuroergonomic approach to performance estimation in a psychomotor vigilance task. (2021) In: Neuroergonomics Conference 2021, 11 September 2021 - 16 September 2021 (Munich, Germany).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


Introduction: Passive brain-computer interfaces (pBCI; tools that enable an implicit mental state estimation) have gained attention in a wide range of applications, including performance and vigilance monitoring in high-risk work settings (Lotte & Roy, 2019). Vigilance can be defined as the ability to maintain sustained attention to a stimulus for an extended period of time (Al-Shargie et al., 2019), and is influenced by the time of day and fatigue (Lim and Dinges, 2008). A vigilance decrement impacts performance over time (called time-on-task -TOT) during tedious monitoring tasks, resulting in slower reaction times or increased errors (Pattyn et al., 2008). This effect is experienced in all kinds of activities such as in aeronautics where pilots can experience a performance drop during the flight (Wiggins, 2011). Hence, vigilance and performance estimation is a crucial step towards the implementation of safer work settings. Machine learning applied to physiological measures, such as cerebral activity (via electroencephalogram -EEG- recordings), is a promising way to estimate performance. EEG’s spectral activity is impacted by fluctuations in vigilance (Matousek & Petersén, 1983) and the power in both theta and alpha bands can be considered as robust biomarkers of mental fatigue (Tran et al., 2020). Numerous studies have attempted to estimate performance during a vigilance task based on EEG measures (Tian et al. 2018), or electrocardiographic (ECG) measures (e.g., heart rate -HR-, and its variability -HRV; Chua et al., 2012). To our knowledge, performance estimation during monotonous tasks has not reached a high accuracy, and pBCI pipelines could be improved. Hence, the objective of this study is to employ a comprehensive neuroergonomic approach to vigilance and performance characterization for a typical vigilance task: the Psychomotor Vigilance Task (PVT; Dinges & Powell, 1985) encompassing statistical analyses, as well as EEG-based performance classification using pre-stimulus signal. Methods: Ten volunteers (3 females; Mage=25, sd=3; ethical number from Univ. Toulouse: 2021-342) performed a 10-minute PVT (i.e., 90 stimuli) from a task battery. They had to complete a fatigue questionnaire (Karolinska Sleepiness Scale; Åkerstedt & Gillberg, 1990) and their response time (RT) was measured. In addition, EEG and ECG activities were recorded using an ActiCHamp system (63+1 electrodes). Data were processed in two ways: i) TOT analysis: data were split into ten 1-minute windows; and ii) Performance analysis: response-based epoching (-2:0s for EEG; -10:0s for ECG), only the 30 best and 30 worst trials were kept (labeled according to RT), and also used for performance classification. Considering statistical analyses, one-way repeated measures ANOVAs and paired samples t-tests (Student and Wilcoxon), were performed separately on each dataset. To perform TOT analysis on RT, the signal was cut into 10 periods of 9 simultaneous stimuli and the reciprocal RT (mean 1/RT) was calculated on each period. EEG data were filtered (1-40 Hz) and ocular artifacts were automatically removed (SOBI method). The theta, alpha, and beta power were extracted for three electrode clusters (frontal, central, posterior). The Task Load Index (TLI: thetaFz/alphaPz) and the engagement ratio (beta/[theta+alpha]; average on all electrodes) were also calculated. ECG data were filtered (1-40 Hz) and normalized using a 1-minute eyes-open resting state period. HR and HRV (as SDNN) were then extracted. For EEG-based estimation, a dimensionality reduction method based on Laplacian was applied (Xu et al., 2021), and a minimum distance to mean with geodesic filtering classifier (FgMDM) was trained and tested using a 10-fold cross-validation procedure. Results: There was no significant difference in subjective fatigue (regardless of PVT order in battery). There was a significant linear downward trend on reciprocal RT (p<.05; other contrasts n.s.; Fig.1.A). There was a significant effect of TOT on alpha power at frontal (p<.05, ηp²=.26) and posterior (p<.05, ηp²=.27) sites, as well as on the engagement ratio (p<.05, ηp²=.31; Fig.1.B). TOT also significantly impacted HR (p<.001, ηp²=.31; Fig.1.C) and HRV (p<.05, ηp²=.27). Regarding performance (best/worst trials), alpha power was significantly lower for the best than worst trials at frontal (p<.01, rB=-.93), and posterior (p<.01, rB=-.89) sites, while the TLI was higher for the best trials (p<.05, d=.87). In terms of performance classification, the FgMDM classifier achieved a mean accuracy of 58.2% without dimensionality reduction. By selecting the number of dimensions that gives the best accuracy for each subject, the mean accuracy with dimensionality reduction was 70.5% (mean number of dimensions = 27.3, Fig.2.A). By comparison, the principal component analysis (PCA) achieved a mean accuracy of 71.1% (mean number of dimensions = 23). Discussion: The results showed that the neuroergonomic approach employed in this study enabled us to assess physiological modulations due to vigilance fluctuations, such as alpha power and engagement ratio decreased over time. However, the HR drop observed in the first minutes may be due to the initial presentation novelty (Kelsey et al., 1999). Also, analyses of the best and worst trials showed that vigilance decreased during the bad trials. However, some results were not as strong as those obtained in the literature. This may be due to either the short task duration (10 minutes) compared to the several sessions performed over several hours or days in the literature, or because our participants were not sleep deprived. Besides, these analyses were performed on a small number of participants (n=10). Regarding performance estimation, the classification accuracy was low for several reasons. Firstly, there is little difference in terms of vigilance during the 10-minute PVT task, even though we have chosen the 30 best and worst trials. Secondly, the number of training samples is quite limited (i.e., 60 per subject). In this case, dimensionality reduction becomes essential as shown by the results. Both dimensionality reduction methods (non-supervised) significantly improve accuracy while reducing the computation time. Moreover, on average the Laplacian-based method performs better in lower dimensions than PCA (Fig.2.B). In summary, this study shows that even with a short task and a normal level of fatigue, it is possible to observe an impact of a monotonous task on behavioral and physiological measures at the group level. Yet, it remains difficult to implement an accurate performance estimation pipeline using a single short session at the individual level.

Item Type:Conference or Workshop Item (Paper)
HAL Id:hal-03576486
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:Université de Toulouse > Institut Supérieur de l'Aéronautique et de l'Espace - ISAE-SUPAERO (FRANCE)
Other partners > Maastricht University - UM (NETHERLANDS)
Laboratory name:
Deposited On:16 Feb 2022 08:43

Repository Staff Only: item control page