Open Access
Add to BibSonomy
Abstract
We propose an effective and practical decoding method of different mental states for potential applications for the design of brain-computer interfaces, prediction of cognitive behaviour, and investigation of cognitive mechanism. Functional near infrared spectroscopy (fNIRS) signals that interrogated the prefrontal and parietal cortices and were evaluated by generalized linear model were recorded when nineteen healthy adults performed the operation span (OSPAN) task. The oxygenated hemoglobin changes during OSPAN, response, and rest periods were classified with a support vector machine (SVM). The relevance vector regression algorithm was utilized for prediction of cognitive performance based on multidomain features of fNIRS signals from the OSPAN task. We acquired decent classification accuracies for OSPAN vs. response (above 91.2%) and for OSPAN vs. rest (above 94.7%). Eight of the ten cognitive testing scores could be predicted from the combination of OSPAN and response features, which indicated the brain hemodynamic responses contain meaningful information suitable for predicting cognitive performance.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Corrections
29 June 2021: Typographical corrections were made to the author affiliations.
1. Introduction
Decoding human mental states non-invasively has a number of potential applications such as brain-computer interface (BCI) [1,2], understanding the perceptual and cognitive mechanisms [2], prediction of cognitive behaviour and assistant diagnoses, etc. [3,4] Among them, BCI systems by invasive or noninvasive means provide a promising strategy to establish communication with individuals who suffer from severe motor disabilities but preserve mental abilities such as stroke and amyotrophic lateral sclerosis (ALS) [5–7]. However, the need to implant objects limits the applications of invasive approach [8].
Several non-invasive neuroimaging modalities have garnered increasing attention in BCIs such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), and magnetoencephalography (MEG) in order to decode different mental states [9–12]. Among these methods, EEG and fNIRS are regarded as the most appropriate candidates for developing BCIs due to their portability, practicality and relatively low cost of equipment [13]. Although EEG can provide amazing temporal resolution, which makes it ideal for real-time BCIs, this technology suffers from the source localization problems and its inherent sensitivity to motion artifacts leading to large spikes that are difficult to correct [14,15].
fNIRS is a novel neuroimaging modality that non-invasively measures brain hemodynamic changes induced by cortical activation with low levels of near infrared light [16]. Analogous to fMRI, fNIRS measures hemodynamic responses associated with neural activities − that is, the relative concentration increase in oxygenated hemoglobin (HbO) and the decrease in deoxygenated hemoglobin (HbR) that occur due to the increased cerebral blood flow [17]. By measuring light absorption caused by HbO and HbR in cerebral vessels, fNIRS estimated the concentration changes of HbO and HbR between the emitter and detector [18,19]. Compared with EEG, fNIRS has the major advantage of being less sensitive to motion artifacts and providing a better spatial resolution. Furthermore, fNIRS-based systems satisfy the characteristics of an ideal BCI system such as easy-to-use, short set-up time and calibration, portability, low operating costs, safety and so on.
To date, fNIRS has been applied to measure brain activation from the prefrontal cortices during various BCI compatible cognitive tasks including the verbal fluency task, n-back task, Stroop task, etc., in the meanwhile different mental states have been decoded and identified by machine learning algorithms [20–22]. Similar decoding methods are also applied to cognition monitoring, drowsiness detection, cognitive load detection, and deception identification [23–26]. Decoding working memory brain activity using fNIRS has the potential to assist assessment and prediction of cognitive status, help improving the traditional neurofeedback approaches into decoded neurofeedback [27]. Operation span (OSPAN) task is a complex working memory span task used to assess working memory (WM) capacity, which requires the participant to engage in a process that is irrelevant to the to-be-remembered information [28], and this task involves with encoding, maintenance, storage, and other information processing. As participants judge equations for correctness and retain the target letters, processing and storage of information happen. OSPAN task entails a higher degree of executive control and exhibits a higher level of ecological validity than traditional neuroimaging WM tasks such as the Sternberg or n-back task. In addition, the neuroimaging literature on complex WM span tasks like OSPAN task is scarce. As dlPFC is highly involved in the working memory process [29], we assumed that fNIRS signals in dlPFC may contain the information to decode working memory status.
For classification, support vector machine (SVM) algorithms are commonly used machine-learning approaches in fNIRS-based BCI studies [30]. Specifically, SVM can resolve the small sample problems because of its statistic learning theory and structural risk minimization principle [31]. The purpose of this study was to evaluate the classification performance of prefrontal fNIRS signals to discriminate working memory status and to explore whether the HbO signals could predict the cognitive performance.
2. Materials and methods
2.1 Participants
Nineteen right-handed healthy adults (mean age = 22.6 ± 3.8 years) were recruited from students at Capital Medical University (Beijing, China) and participated in the study. Participants had normal or corrected-to-normal vision and had no degenerative, neurological and psychiatric conditions that are known to affect cognition (self-reported). All participants in this study performed OSPAN test, Stroop test, symbol digital modalities test (SDMT) and digit span test. Demographic characteristics and cognitive assessment data are shown in Table 1. The study was approved by the Medical Ethics committee of Capital Medical University. Written informed consent was obtained from all participants prior to study participation.
Table 1. Demographic characteristics and cognitive performance of participants.a
2.2 Experimental protocol
The participants were seated on a comfortable chair in front of a computer screen in a confined and quiet room. They were asked to be relaxed and to keep from any movement before fNIRS data acquisition.
Participants performed the OSPAN task in a similar fashion to Carlos et al. [32]. There were 3 OSPAN epochs followed by 15s response epochs and 3 baseline resting epochs during the entire OSPAN task, each epoch lasting 30s as shown in Fig. 1. During each OSPAN epoch, five arithmetic problems and five target letters were alternately presented. The arithmetic problems were presented for 4s and the letters were presented for 2s. Participants were instructed to judge equations [e.g., (12 × 3) - 4 = 32]as accurately as possible by pressing the appropriate key (i.e., “F” key denotes the true and “J” key denotes the false) once the arithmetic problem was presented and to remember the target letters in serial sequence. After each OSPAN epoch, there was a response epoch, in which five separate arrays including three candidates were presented for 3s each. Participants were asked to identify the target letter among candidates with the appropriate key. The response was regarded as incorrect once the response exceeded the allotted time.
Fig. 1. Experimental protocol for fNIRS measurement. (a) The block design paradigm for stimuli, each block repeated 3 times; (b) Progression of OSPAN block and response block. During OSPAN period, each participant was asked to judge equations as accurately as possible with the corresponding key and retain the target letters in serial sequence. During response period, participants were asked to identify the target letter by pressing the appropriate key. Abbreviations: fNIRS, functional near-infrared spectroscopy; OSPAN, operation span.
To be acquainted with the task, Participants took around 3 min to practice. After practice, they performed the OSPAN task while fNIRS data was recorded. The task was programmed in PsychToolbox (PTB) version 3.0.16, within MATLAB (Mathworks, 2013b).
2.3 fNIRS data acquisition
Relative changes in concentration of HbO and HbR in the cortex during OSPAN task were detected by a multichannel continuous-wave fNIRS system (LABNIRS, Shimadzu Corp., Kyoto, Japan) with three wavelengths (780, 805, and 830 nm). We used a probe set consisting of 16 emitters and 12 detectors, resulting in a total of 34 channels with a standard inter-optode distance of 3 cm. This channel width allows for measurement of cortical activation at a depth of 1.5 cm [18,33].The optical signals were sampled at 47.6 Hz.
All channels were placed according to the international 10–20 system and covered the frontal and parietal regions as shown in Fig. 2(a). The anatomical locations of channels in the Montreal Neurological Institute (MNI) standard brain space were determined by utilizing a 3-D digitizer (FASTRAK, Polhemus, Vermont, USA) in reference to the nasion, top center (Cz) and bilateral preauricular points.
Fig. 2. fNIRS optode information. (a) fNIRS optode placement and channel configuration with emitters shown in red circles, detectors in blue, and channel numbers listed between emitters and detectors. (b) Experimental set-up. Abbreviations: fNIRS, functional near-infrared spectroscopy; OSPAN, operation span.
2.4 fNIRS data analysis
2.4.1 Data pre-processing
HbO signals were used in the present study due to its better performance for assessing task-induced brain activation and higher signal-to-noise ratio [34,35]. Baseline correction and filtering were applied by using internal routines of NIRS_SPM software, with a wavelet-minimum description length (MDL) detrending algorithm and hemodynamic response function (hrf), respectively [36]. The Wavelet-MDL detrending algorithm removes the unknown global drift due to physiological changes such as breathing, cardiac, and vasomotion errors or other experimental errors such as movement artifacts and instrumental instability by decomposing NIRS measurements into global trends, hemodynamic signals and uncorrelated noise components [37].
2.4.2 General linear model analysis
General Linear Model (GLM) was used to analyze the temporal variational patterns of HbO signals associated with two regressors, encoding along with maintenance (OSPAN epoch for 30 s) and retrieval along with maintenance (response epoch for 15 s) by NIRS-SPM software. Group cortical activation maps on OSPAN and response conditions were both estimated for 34 channels. The T statistic was calculated for every channel and interpolated T-maps below the threshold (the corrected p-value = 0.05) were superimposed on brain template to construct 3-D brain activation maps. Lipschitz-Killing curvature (LKC)-based expected Euler characteristics was selected for the p-value correction in this group analysis [38].
2.4.3 Feature extraction
Based on previous studies [11,24,39–45], eleven features extracted from the time-courses that were standardized to zero mean and unit variance for OSPAN, response and rest trials were as follows: signal mean (SM), signal slope (SS), signal peak (SP), signal standard deviation (SSD), signal covariance (SC), signal root mean square (SRMS), signal kurtosis (SK), signal skewness (SSk), signal maximum power (SMP), signal frequency (SF) corresponding to maximum power, and signal sample entropy (SSE). For this, mean, max, regress, std, var, rms, kurtosis and skewness functions available in MATLAB (Mathworks, 2013b) were used. In addition, features including SMP, SF and SSE were calculated with custom functions based on MATLAB environment. These features were calculated for 34 channels. However, only averaged features of channel 6 (left dorsolateral prefrontal cortex (dlPFC): Brodmann area (BA) 46) and 30 (right dlPFC: BA 9) activated during both OSPAN and response epochs were selected for classification.
2.4.4 Feature selection
To eliminate the effect of feature range differences on classification performance, each feature vector was standardized by removing the mean and scaling to unit variance. To reduce those redundant features and computational cost, the feature selection technique was applied to identify meaningful features.
Feature selection can be defined as a process to find those features describing datasets as well as possible [46]. The Pearson’s correlation coefficient and mutual information (MI) criteria were used to obtain the most relevant features. They estimate the linear and non-linear relationship of each feature with the output class, respectively [47,48].
2.4.5 Feature classification
Samples with selected features were classified with SVM that can be used as a linear as well as a non-linear classifier. SVM is a discriminative classifier which creates the optimum hyperplane by maximizing the distance between the training data of different classes [11,49]. SVM has been widely used in various fNIRS-based BCI studies for classification and achieved high performance [11,40,50]. The linear, polynomial and Gaussian kernels were used, and regularization parameters, polynomial parameters and Gaussian parameters were determined by a grid search and 5-fold cross-validation to select models and alleviative overfitting [51]. 5-fold cross-validation was also utilized to determine the classification accuracies for three conditions: OSPAN vs. response, OSPAN vs. rest, and response vs. rest. The average mean accuracy for each condition was taken.
To ascertain the performance of the SVM classifier, the true positive rate (TPR) and false positive rate (FPR) were calculated for binary classification as follows [52]:
where TP, TN, FP, and FN denote true positive, true negative, false positive, and false negative, respectively. The four values mentioned were obtained from the confusion matrix. Then FPR and TPR for each condition were used to draw the receiver operating characteristic (ROC) curve of the binary SVM classifier and the area under the curve (AUC) was calculated. In addition, we added a separate data set from fifteen healthy adults when they performed the OSPAN task for the testing of machine leaning models to test whether there existed an overfitting phenomenon. The code implemented for analysis was developed in MATLAB (Mathworks, 2013b) by applying functions of the Statistical and Machine Learning Toolbox.2.4.6 Prediction of cognitive performance using relevance vector regression
To investigate the relationship between neural signals during OSPAN and response periods and cognitive performance of OSPAN test, Stroop test, SDMT and digit span test, the relevance vector regression (RVR) algorithm was utilized. RVR is a sparse kernel regression algorithm set in a probabilistic Bayesian framework, where RVR introduces a zero-mean Gaussian prior for the model weights governed by a set of hyperparameters [53,54]. It allows the quantitative prediction of interested variables without the discrete categorical decision.
Specifically, the RVR algorithm took the original eleven feature sets of OSPAN and response periods as input vectors both separately and together and the performance on four cognitive tests as targets that were involved with WM, selective attention, processing speed, etc. To assess the accuracy of the prediction, the Pearson correlation coefficient (CORR), mean squared error (MSE) and norm MSE between the targets and predictions were computed [55,56].
3. Results
3.1 Cortical activation maps on OSPAN and response conditions
For the OSPAN condition involved with encoding as well as maintenance, we found signifcant task related activations (i.e., HbO increase) in both hemispheres for channel 6 (left dlPFC: BA 46), 10 (left dlPFC: BA9), 30 (right dlPFC: BA9), and 27 (right Broca’s area: BA45), which are listed from small to large according to the statistic T. For the response condition involved with retrieval as well as maintenance, we also found task related activations in both hemispheres for channel 6 (left dlPFC: BA46), 29 (right dlPFC: BA9), 30 (right dlPFC: BA9), 3 (left Frontopolar area: BA10), and 9 (left Broca’s area: BA45). These results indicated that channel 6 (left dlPFC: BA46) and 30 (right dlPFC: BA9) were both activated on OSPAN and response conditions. Some differences, however, existed between them. For response periods, activated regions contained frontopolar area besides Broca’s area in the left hemisphere (See Fig. 3).
Fig. 3. The group brain activation maps (T-maps) of HbO signals on OSPAN and response conditions (the corrected p-value < 0.05). For the OSPAN condition involved with encoding as well as maintenance, significant cortical activations were observed in both hemispheres for dlPFC and the right Broca’s area (Z-score > 3.57). For the response condition involved with retrieval as well as maintenance, activated regions contained dlPFC of both hemispheres and the left frontopolar area besides Broca’s area (Z-score > 3.41). Abbreviations: dlPFC, dorsolateral prefrontal cortex; HbO, oxygenated hemoglobin; OSPAN, operation span.
3.2 Classification performance
To determine the classification accuracies of three SVM classifiers (i.e., linear, polynomial and Gaussian) for three conditions (i.e., OSPAN vs. response, OSPAN vs. rest, and response vs. rest), we employed 5-fold cross validation to obtain the mean results. Other performance parameters i.e., sensitivity and specificity were also calculated. We compared the classification performance of the optimized features using the Pearson’s correlation coefficient and MI criteria and the original eleven features.
Based on original features, all SVM classifiers performed with high accuracies above 88.6% for OSPAN vs. response and OSPAN vs. rest conditions. However, the accuracy performance of all three kernels in differentiating response from rest decreased but above 70.0%. After applying feature selection technique, three SVM classifiers were able to differentiate OSPAN signals from response signals and OSPAN signals from rest signals with far higher accuracies above 91.2% compared to that obtained by using original features, indicating that our feature selection method worked well. Similarly, classification accuracies of all SVM classifiers for response vs. rest condition were evidently lower than the other two conditions (See Table 2). The results of the testing of machine leaning models were shown in Table 3, which proved that our models were robust and effective.
Table 2. Classification performance of three SVM classifiers for three conditions when selected and eleven original features were input, respectively.a
Table 3. Classification performance of three SVM classifiers for three conditions when selected features of the testing set are input, respectively.a
Figure 4 shows the grand-averaged hemodynamic response across all participants at channel 6 and 30 that were both activated on OSPAN and response conditions, for OSPAN, response and rest phases during OSPAN task, respectively. The hemodynamic responses evoked by different mental states were significantly different, especially for OSPAN vs. response and OSPAN vs. rest conditions, making it possible to differentiate two different mental states with considerable accuracy. Furthermore, small standard deviation computed across all participants for corresponding HbO response demonstrate the high reliability of our results. The ROC curves of three SVM classifiers for OSPAN vs. response and OSPAN vs. rest conditions after applying feature selection technique were drawn for each binary classification using the averaged TPR and FPR for 5-runs of 5-fold cross-validation (See Fig. 5).
Fig. 4. Grand averages of the hemodynamic responses recorded during OSPAN, response, and rest periods for channel 6 and 30. The shade areas denote standard deviation computed across nineteen participants for corresponding HbO response. Abbreviations: HbO, oxygenated hemoglobin; OSPAN, operation span.
Fig. 5. ROC curves of three SVM classifiers for OSPAN vs. response and OSPAN vs. rest conditions over all subjects. (a) ROC curves of SVM classifiers with linear, polynomial and Gaussian kernels for OSPAN vs. response condition. (b) ROC curves of SVM classifiers with linear, polynomial and Gaussian kernels for OSPAN vs. rest condition. Abbreviations: AUC, area under the curve; G: Gaussian; L: linear; OSPAN, operation span; P: polynomial; ROC, receiver operating characteristic; SVM, support vector machine.
3.3 Prediction results of RVR
The mean behavioral performance of four cognitive tests for nineteen participants were shown in Table 1. We relegated the results of predictions with OSPAN or response features alone to the Supplement 1 (Table S1 and S2). The combination of OSPAN and response features obtained the best predictive performance and the results were reported. Specifically, in 8 of the 10 cognitive testing scores, the predicted scores were correlated highly with the actual score (Letter recognition (LR): CORR = 0.85, p < 0.01; Letter recognition reaction time (LRRT): CORR = 0.52, p < 0.05; Arithmetic verification (AV): CORR = 0.61, p < 0.01; Arithmetic verification reaction time (AVRT): CORR = 0.52, p < 0.05; Stroop A RT: CORR = 0.49, p < 0.05; Stroop C RT: CORR = 0.68, p < 0.01; DSF: CORR = 0.66, p < 0.01; DSB: CORR = 0.77, p < 0.01) (See Table 4). Figure 6 illustrates the corresponding scatter plots of each significant correlation score predicted from the RVR against the actual scores. We presented the scatter plots based on OSPAN or response features alone in the Supplement 1 (Figure S1 and S2).
Fig. 6. Scatter plots showing the correlation between actual and predicted cognitive performance for all participants using RVR based on combination of OSPAN and response features. (a) LR in OSPAN test; (b) LRRT in OSPAN test; (c) AV in OSPAN test; (d) AVRT in OSPAN test; (e) Stroop A RT in Stroop test; (f) Stroop C RT in Stroop test; (g) DSF in digit span test; (h) DSB in digit span test. Abbreviations: AV, arithmetic verification; AVRT, arithmetic verification reaction time; CORR, Pearson correlation coefficient; DSB, digit span backward; DSF, digit span forward; LR, letter recognition; LRRT, letter recognition reaction time; OSPAN, operation span; RT, reaction time; RVR, relevance vector regression. *p < 0.05, **p < 0.01.
Table 4. RVR predictions of cognitive performance based on combination of OSPAN and response features.a
4. Discussion
In the present study of 19 healthy adults, we utilized GLM to evaluate group cortical activation during two different periods of the OSPAN task. Then, the recorded HbO data from channel 6 (left dlPFC, BA 46) and 30 (right dlPFC, BA 9) that were activated during both OSPAN and response epochs were selected for further feature extraction, selection and SVM-based classification. Lastly, using RVR algorithm, we predicted multidomain cognitive performance based on multidomain features of dlPFC from OSPAN and response epochs both separately and together in order to investigate whether the objective neuroimaging technique (i.e., fNIRS) was able to inform the individual cognition.
4.1 Validity of fNIRS-based estimation of OSPAN-related cortical activities using GLM
We have chosen the OSPAN task because it entails a higher degree of executive control and exhibits a higher level of ecological validity than traditional neuroimaging WM tasks. In addition, the neuroimaging literature on complex WM span tasks like OSPAN task is scarce. In our knowledge, this is the first study examining cortical activities during OSPAN task based on fNIRS.
Compared to two previous fMRI studies [32,57], the present results showed similar activation pattern during OSPAN task. More precisely, cortical regions activated are bilateral dlPFC (BA 9 and 46), Broca’s area (BA 45) and left frontopolar area (BA10). As mentioned in results section, bilateral dlPFC and either left or right Broca’s area exhibited robust activation during both analysis conditions (i.e., OSPAN and response conditions related to encoding embedded with active maintenance and retrieval embedded with maintenance, respectively). The left frontopolar area was only activated for response condition. Recently, Chein et al. examined the general mechanisms during encoding and maintenance and they found that cortical areas activated during encoding and maintenance is typically associated with WM [58]. Furthermore, the dlPFC (BA 9/46) and Broca’s area (BA 44 and 45) were engaged during encoding and retrieval phases of the WM task [59]. Similarly, the left anterior middle frontal gyrus (BA 10/46) was activated during the retrieval phases of the WM, which converges with our findings. In other words, the fNIRS-based group activation results of two conditions during the OSPAN task were reliable and consistent with previous fMRI findings.
4.2 Pre-processing of fNIRS signals
Although fNIRS systems can record two hemodynamic signals (i.e., HbO and HbR) from the cortical surface in a range of contexts and populations, there is no an agreement or guidelines on the pre-processing of fNIRS data [60]. In fact, the fNIRS signals comprises different sources of noise. Most problematic seems to be the superficial physiological noise that arises from the physiological fluctuation of the blood supply system in the scalp and the skull, and motion artifacts caused by movements of the head [61,62].
The wavelet transform (WT) is a powerful mathematical tool through expanding a time series into time-frequency space with time and frequency localized basis functions and is used in fNIRS signal detrending [63] and motion artifact removal [64,65]. The Wavelet-based method has been proven to be used for fNIRS global physiological noise removal [62]. In our paper, we used the WT-based Wavelet-MDL detrending algorithm to remove the unknown global drift due to physiological changes such as breathing, cardiac, and vasomotion errors or other experimental errors such as movement artifacts and instrumental instability by decomposing fNIRS measurements into global trends, hemodynamic signals and uncorrelated noise components [36,37] and to address the movement artifacts. The Wavelet-MDL detrending algorithm is robust and widely used in fNIRS signals for data preprocessing [66,67]. For the physiological effect on fNIRS signals, we know the short source-detector distance channel (sSD channel)-based correction is an effective method to reduce the influence of physiological noise through a linear regression model [62,68,69]. In the case that short separation correction technique is unavailable, some other methods have been developed to remove the fNIRS physiological noise, such as the band-pass filtering, principal component analysis (PCA), independent component analysis (ICA) and Wavelet-based method [62,69]. However, the band-pass filtering may destroy the experimental effects of interest if the frequency band of the noise overlaps with that of the neural activity. The performance of PCA and ICA highly depends on the noise component identification procedure, which often needs manual selection which requires rich experience from the user. Consequently, Wavelet-based method would be a better choice among them. In addition, the spline interpolation and wavelet analysis are recommended to minimize the impact of motion artifacts on fNIRS data compared to other motion correction techniques [70]. The performance of spline interpolation depends on the method of artifact detection and the technique may not improve the signal if the artifacts are difficult to detect. However, the wavelet filtering can work well with different forms of motion artifacts such as low-frequency, low-amplitude, HRF-correlated artifacts even those that are relatively subtle [61]. Furthermore, we have checked carefully our raw fNIRS signals, there exists almost no large drops in the courses. Hence, we applied wavelet filtering to correct the motion artifacts here, rather than the spline interpolation.
4.3 Classification performance compared to previous fNIRS-based BCI applications
Before classification, we ran the GLM for OSPAN vs. rest and response vs. rest conditions to identify the commonly activated channels, which mainly aimed to locate brain regions mostly related to the cognitive task, then we extracted HbO features from these channels for classification. GLM measures the temporal variational pattern of signals rather than their magnitude [36]. However, we observed that the temporal variational patterns of OSPAN and response periods from channels 6 and 30 were similar (See Fig. 4), which indicates it may be very difficult to differentiate OSPAN signals from response signals via beta values for OSPAN vs. response condition. The SVM classifier can be used for both linear and non-linear datasets by kernel functions to work favorably in fNIRS-BCI studies for classification purposes [11], which is suitable for our research objective.
The majority of previous fNIRS-based BCI studies that detected high-level cognitive tasks have attained the offline classification accuracies ranging from 71.0% to 82.8% [20–22,71–76]. These tasks mainly included mental arithmetic, verbal fluency task, Stroop task and n-back task. In the present study, we achieved impressive accuracies of at least 91.2% for OSPAN vs. response condition and that of at least 94.7% for OSPAN vs. rest condition when fed with selected features as reported in Table 2. There was a significant improvement in classification performance relative to other studies of decoding two different mental states. In case of differentiating response from rest, the accuracy performance of all three kernels decreased but above 70.0% that is defined as the threshold acceptable for practical communication with BCIs [77]. In addition, we have run the analysis using HbR that tends to have better sensitivity to the brain for classification. The results show features of HbR obtain the similar accuracies for OSPAN vs. response and OSPAN vs. rest compared to that of HbO. However, the classification accuracy of three kernels in differentiating response signals from rest are significantly lower than that of HbO (a maximum of 50.9%) (See Table S3 in the Supplement 1).
The HbO-based classifcation accuracy may attribute to the feature selection technique on the basis of the combination of Pearson’s correlation coefcient and MI criteria, which has improved the classifcation accuracy, sensitivity and specifcity in comparison to not applying this phase. Linear correlation helped to determine relevant features with linear correlation to the outputs but it could not detect the arbitrary relation while MI worked [78]. Thus, they were complementary and yielded efficient classification. Some previous fNIRS based BCI studies that experienced feature selection mainly used the Fisher criterion and MI/joint MI to evaluate multidimensional feature subsets but not reported corresponding results for original features [21,22,42,73,75,76]. In line with our findings, a Multi-layer-perceptron (MLP) neural network (NN) trained with selected feature sets based on Pearson’s correlation coefficient led to better classification results for the Wisconsin Breast Cancer data [79]. Moreover, Fatemeh et al. revealed that a least squares SVM (LSSVM) with MI or linear correlation coefficient-based feature selection also performed better than that without this phase in computational cost and detection accuracy [47]. In addition, high-dimensional features, small samples, high regularization parameter and kernel parameters may lead to the SVM classifier’s overfitting in classification. A routine check for recognizing overfitting is to obtain the loss and accuracy on the training and testing sets [80]. In our paper, the training set may be not enough, which derived from fNIRS signals when 19 participants performed the OSPAN task, totally including 57 samples for each mental state (i.e., OSPAN, Response and rest periods). Consequently, by testing the models with the never-seen-before data from 15 participants, we tested whether the models existed overfitting problem and found that the models performed comparably on the test data set (See Tables 2 and 3), which suggested that our models were effective and there seemed to be no overfitting. Furthermore, regularization parameters, polynomial parameters and Gaussian parameters were determined by a grid search and 5-fold cross-validation to select models and alleviative overfitting as noted in the method section [51]. Although the k-fold cross-validation is the most popular cross-validation procedure due to its mild computational cost, the best value of k remains widely open. For model selection, it is often reported that the optimal k is between 5 and 10 [81]. Hence, we chose 5-fold cross-validation because of our small datasets and less computational cost.
Collectively, we attained decent accuracies in terms of classifying two different mental states, which may attribute to the following three reasons: 1) the promising potential of complex working memory span task (i.e., OSPAN task) for fNIRS-based BCIs. 2) effective feature selection technique that removed information redundancy and improved the classification accuracy as well as decreased the likelihood of an SVM classifier’s overfitting in classification by obtaining effective features. 3) the cross-validation we used to select models and tune parameters such as regularization parameter and kernel parameters, which is widely used to alleviative overfitting. As for the response vs. rest condition, the classification performance was likely affected by the delayed hemodynamic response from response periods, which led to overlapped information during response and rest periods.
4.4 Prediction of cognitive performance from dlPFC activity
Current non-invasive neuroimaging modalities have provided a new approach to decode the informational contents of neurocognitive representations through ongoing brain activity. WM refers to the temporary representation of information and enables individuals to focus on a desired goal and to override the automatic responses through cognitively salient processes, thus making it critical for our everyday tasks [82,83]. The dlPFC is a region of the prefrontal cortex typically associated with WM and selective attention and considered as the crucial node supporting WM [83]. Blumenfeld and Ranganath investigated the relationship between dlPFC activity during WM tasks and subsequent long-term memory test (LTM) [84]. They indicated that signal changes of dlPFC were predictive of LTM performance. To date, there have been no fNIRS studies that explore whether the brain hemodynamic responses from dlPFC contain meaningful information suitable for predicting cognitive performance using machine learning technology in order to investigate further the association between neural signals and individual cognition.
All participants in this study performed OSPAN test, Stroop test, SDMT and digit span test. We applied RVR with no free parameter to the multidomain features derived from fNIRS signals during OSPAN task, which allowed quantitative prediction of cognitive performance. The OSPAN and response features were put into RVR algorithm both separately and together to find the best prediction results. More precisely, in 8 of the 10 cognitive testing scores, the predicted scores were correlated highly with the actual scores for combination of OSPAN and response features. Consequently, we suggested that the contents of the measured fNIRS signals from dlPFC captured signatures of cognitive performance associated with WM and selective attention. Furthermore, in line with the controlled attention theory of WM [85], our neural activities derived from OSPAN task that measured individual’s working memory capacity (WMC) could indeed be used to predict the control of attention. In the future, the characteristics of hemodynamic response due to WM task instead of behavioral performance may predict workload capacity, which refers to the measurement of the efficiency of processing [86].
4.5 Limitations
There are several limitations in the present study. Although we used a separate data set from fifteen healthy adults when they performed the OSPAN task to test the SVM models and results suggest that there seems to be no overfitting phenomenon, we believe that these SVM models would have better generalization ability and our classification results would be more acceptable with a lager sample size. Moreover, we had just interrogated the constrained parietal regions due to the limited optodes, which are crucial to WM and believed to store the contents in the focus of attention [32,87]. In future studies, the more interrogation area of parietal regions and its role in cognitive processes and value in mental states decoding warrant further exploration.
5. Conclusions
Mental state decoding-based approach provides great promise in developing BCI systems, predicting the cognitive performance and understanding the perceptual and cognitive mechanisms from brain activity. Hence, we explored the utility of training SVM classifers with three kernels by fNIRS measurements during cognitive task that invoked WM. The highest accuracies achieved were 97.4% and 94.7% for OSPAN vs. rest and OSPAN vs. response respectively, which were significantly higher than previous fNIRS-based BCI studies that detected high-level cognitive tasks. Furthermore, we demonstrated the objective neuroimaging technique (i.e., fNIRS) was able to predict the individual cognition performance through RVR algorithm. Collectively, our results are encouraging and show a promising potential of mental state decoding for the design of BCI, prediction of cognitive behaviour and investigation of cognitive mechanism.
Funding
Science and Technology Development Center, Ministry of Education (2018A03004); National Natural Science Foundation of China (31600933, 61701323, 81771909).
Acknowledgments
The authors would like to thank all volunteers who participated in the study and the staff of Capital Medical University, Beijing, China for their selfless and valuable assistance.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. J. J. Shih, D. J. Krusienski, and J. R. Wolpaw, “Brain-computer interfaces in medicine,” Mayo Clin. Proc. 87(3), 268–279 (2012). [CrossRef]
2. J. D. Haynes and G. Rees, “Decoding mental states from brain activity in humans,” Nat. Rev. Neurosci. 7(7), 523–534 (2006). [CrossRef]
3. A. S. Greene, S. Gao, D. Scheinost, and R. T. Constable, “Task-induced brain state manipulation improves prediction of individual traits,” Nat. Commun. 9(1), 2807 (2018). [CrossRef]
4. G. Maleki, A. Zabihzadeh, M. J. Richman, Z. Demetrovics, and F. Mohammadnejad, “Decoding and reasoning mental states in major depression and social anxiety disorder,” BMC Psychiatry 20(1), 463 (2020). [CrossRef]
5. L. R. Hochberg, D. Bacher, B. Jarosiewicz, N. Y. Masse, J. D. Simeral, J. Vogel, S. Haddadin, J. Liu, S. S. Cash, P. van der Smagt, and J. P. Donoghue, “Reach and grasp by people with tetraplegia using a neurally controlled robotic arm,” Nature 485(7398), 372–375 (2012). [CrossRef]
6. A. Abdalmalak, D. Milej, L. Norton, D. B. Debicki, T. Gofton, M. Diop, A. M. Owen, and K. S. Lawrence, “Single-session communication with a locked-in patient by functional near-infrared spectroscopy,” Neurophotonics 4, 040501 (2017). [CrossRef]
7. G. Gallegos-Ayala, A. Furdea, K. Takano, C. A. Ruf, H. Flor, and N. Birbaumer, “Brain communication in a completely locked-in patient using bedside near-infrared spectroscopy,” Neurology 82(21), 1930–1932 (2014). [CrossRef]
8. S. Waldert, “Invasive vs. non-invasive neuronal signals for brain-machine interfaces: will one prevail?” Front. Neurosci. 10, 295 (2016). [CrossRef]
9. I. Rejer, “Genetic algorithm with aggressive mutation for feature selection in BCI feature space,” Pattern Anal. Applic. 18(3), 485–492 (2015). [CrossRef]
10. B. Sorger, B. Dahmen, J. Reithler, O. Gosseries, A. Maudoux, S. Laureys, and R. Goebel, “Another kind of ‘BOLD Response': answering multiple-choice questions via online decoded single-trial brain signals,” Prog Brain Res 177, 275–292 (2009). [CrossRef]
11. S. B. Erdogan, E. Ozsarfati, B. Dilek, K. S. Kadak, L. Hanoglu, and A. Akin, “Classification of motor imagery and execution signals with population-level feature sets: implications for probe design in fNIRS based BCI,” J. Neural Eng. 16(2), 026029 (2019). [CrossRef]
12. S. Waldert, H. Preissl, E. Demandt, C. Braun, N. Birbaumer, A. Aertsen, and C. Mehring, “Hand movement direction decoded from MEG and EEG,” J. Neurosci. 28(4), 1000–1008 (2008). [CrossRef]
13. U. Chaudhary, N. Birbaumer, and M. R. Curado, “Brain-Machine Interface (BMI) in paralysis,” Ann. Phys. Rehabil. Med. 58(1), 9–13 (2015). [CrossRef]
14. B. He, L. Yang, C. Wilke, and H. Yuan, “Electrophysiological imaging of brain activity and connectivity-challenges and opportunities,” IEEE Trans. Biomed. Eng. 58(7), 1918–1931 (2011). [CrossRef]
15. V. Mihajlovic, S. Patki, and B. Grundlehner, “The impact of head movements on EEG and contact impedance: an adaptive filtering solution for motion artifact reduction,” Ann. Int. Conf. IEEE Eng. Med. Biol. Soc.2014, 5064–5067 (2014).
16. A. Villringer and B. Chance, “Non-invasive optical spectroscopy and imaging of human brain function,” Trends Neurosci. 20(10), 435–442 (1997). [CrossRef]
17. M. M. Monti, A. M. Owen, and S. Laureys, “Willful modulation of brain activity in disorders of consciousness REPLY,” N. Engl. J. Med. 362(7), 579–589 (2010). [CrossRef]
18. G. Strangman, J. P. Culver, J. H. Thompson, and D. A. Boas, “A quantitative comparison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation,” NeuroImage 17(2), 719–731 (2002). [CrossRef]
19. L. R. Trambaiolli, C. E. Biazoli, A. M. Cravo, T. H. Falk, and J. R. Sato, “Functional near-infrared spectroscopy-based affective neurofeedback: feedback effect, illiteracy phenomena, and whole-connectivity profiles,” Neurophotonics 5(03), 1 (2018). [CrossRef]
20. A. Faress and T. Chau, “Towards a multimodal brain-computer interface: combining fNIRS and fTCD measurements to enable higher classification accuracy,” NeuroImage 77, 186–194 (2013). [CrossRef]
21. C. Herff, D. Heger, O. Fortmann, J. Hennrich, F. Putze, and T. Schultz, “Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS,” Front. Hum. Neurosci. 7, 935 (2014). [CrossRef]
22. L. C. Schudlo and T. Chau, “Single-trial classification of near-infrared spectroscopy signals arising from multiple cortical regions,” Behav. Brain Res. 290, 131–142 (2015). [CrossRef]
23. T. O. Zander and C. Kothe, “Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general,” Journal of Neural Engineering 8, 025005 (2011). [CrossRef]
24. M. J. Khan and K. S. Hong, “Passive BCI based on drowsiness detection: an fNIRS study,” Biomed. Opt. Express 6(10), 4063–4078 (2015). [CrossRef]
25. M. J. Boring, K. Ridgeway, M. Shvartsman, and T. R. Jonker, “Continuous decoding of cognitive load from electroencephalography reveals task-general and task-specific correlates,” J. Neural Eng. 17(5), 056016 (2020). [CrossRef]
26. X. S. Hu, K. S. Hong, and S. S. Ge, “fNIRS-based online deception decoding,” J. Neural Eng. 9(2), 026012 (2012). [CrossRef]
27. V. Taschereau-Dumouchel, A. Cortese, H. Lau, and M. Kawato, “Conducting decoded neurofeedback studies,” Social Cognitive and Affective Neuroscience 2020, nsaa063 (2020). [CrossRef]
28. M. L. Turner and R. W. Engle, “Is working memory capacity task dependent?” Journal of Memory and Language 28(2), 127–154 (1989). [CrossRef]
29. A. K. Barbey, M. Koenigs, and J. Grafman, “Dorsolateral prefrontal contributions to human working memory,” Cortex 49(5), 1195–1205 (2013). [CrossRef]
30. N. Naseer and K. S. Hong, “fNIRS-based brain-computer interfaces: a review,” Front. Hum. Neurosci. 9, 3 (2015). [CrossRef]
31. C. Cortes and V. Vapnik, “Support-Vector Networks,” Mach Learn 20, 273–297 (1995). [CrossRef]
32. C. C. Faraco, N. Unsworth, J. Langley, D. Terry, K. Li, D. Zhang, T. Liu, and L. S. Miller, “Complex span tasks and hippocampal recruitment during working memory,” NeuroImage 55(2), 773–787 (2011). [CrossRef]
33. B. D. Kussman, C. M. Aasted, M. A. Yucel, S. C. Steele, M. E. Alexander, D. A. Boas, D. Borsook, and L. Becerra, “Capturing pain in the cortex during general anesthesia: near infrared spectroscopy measures in patients undergoing catheter ablation of arrhythmias,” PLoS One 11(7), e0158975 (2016). [CrossRef]
34. M. Mihara, I. Miyai, N. Hattori, M. Hatakenaka, H. Yagura, T. Kawano, M. Okibayashi, N. Danjo, A. Ishikawa, Y. Inoue, and K. Kubota, “Neurofeedback using real-time near-infrared spectroscopy enhances motor imagery related cortical activation,” PLoS One 7, 21–30 (2012). [CrossRef]
35. N. Naseer and K. S. Hong, “Classification of functional near-infrared spectroscopy signals corresponding to the right- and left-wrist motor imagery for development of a brain-computer interface,” Neurosci Lett 553, 84–89 (2013). [CrossRef]
36. J. Chul, S. Tak, K. E. Jang, J. Jung, and J. Jang, “NIRS-SPM: Statistical parametric mapping for near-infrared spectroscopy,” NeuroImage 44(2), 428–447 (2009). [CrossRef]
37. F. S. Azar, K. E. Jang, X. Intes, S. Tak, J. Jang, J. Jung, and J. C. Ye, “Wavelet-MDL based detrending method for near infrared spectroscopy (NIRS),” in Multimodal Biomedical Imaging III, (2008).
38. H. Li, S. Tak, and J. C. Ye, “Lipschitz-Killing curvature based expected Euler characteristics for p-value correction in fNIRS,” J Neurosci Meth 204(1), 61–67 (2012). [CrossRef]
39. U. Chaudhary, B. Xia, S. Silvoni, L. G. Cohen, and N. Birbaumer, “Brain-Computer Interface-Based Communication in the Completely Locked-In State,” PLoS Biol. 15(1), e1002593 (2017). [CrossRef]
40. N. K. Qureshi, N. Naseer, F. M. Noori, H. Nazeer, R. A. Khan, and S. Saleem, “Enhancing Classification Performance of Functional Near-Infrared Spectroscopy Brain-Computer Interface Using Adaptive Estimation of General Linear Model Coefficients,” Front. Neurorobot. 11, 33 (2017). [CrossRef]
41. T. Trakoolwilaiwan, B. Behboodi, J. Lee, K. Kim, and J. W. Choi, “Convolutional neural network for high-accuracy functional near-infrared spectroscopy in a brain-computer interface: three-class classification of rest, right-, and left-hand motor execution,” Neurophotonics 5(01), 1 (2017). [CrossRef]
42. R. Fernandez Rojas, X. Huang, and K. L. Ou, “A machine learning approach for the identification of a biomarker of human pain using fNIRS,” Sci. Rep. 9(1), 5645 (2019). [CrossRef]
43. C. G. Li, M. Su, J. C. Xu, H. D. Jin, and L. N. Sun, “A Between-subject fNIRS-BCI study on detecting self-regulated intention during walking,” IEEE Trans. Neural Syst. Rehabil. Eng. 28(2), 531–540 (2020). [CrossRef]
44. L. Xu, Q. Hua, J. Yu, and J. Li, “Classification of autism spectrum disorder based on sample entropy of spontaneous functional near infra-red spectroscopy signal,” Clin. Neurophysiol. 131(6), 1365–1374 (2020). [CrossRef]
45. R. A. Khan, N. Naseer, N. K. Qureshi, F. M. Noori, H. Nazeer, and M. U. Khan, “fNIRS-based Neurorobotic Interface for gait rehabilitation,” Journal of Neuroengineering and Rehabilitation 15 (2018).
46. J. Q. Gan, B. A. S. Hasan, and C. S. L. Tsui, “A filter-dominating hybrid sequential forward floating search method for feature subset selection in high-dimensional space,” Int. J. Mach. Learn. & Cyber. 5(3), 413–423 (2014). [CrossRef]
47. F. Amiri, M. R. Yousefi, C. Lucas, A. Shakery, and N. Yazdani, “Mutual information-based feature selection for intrusion detection systems,” J Netw Comput Appl 34(4), 1184–1199 (2011). [CrossRef]
48. S. Mohammadi, H. Mirvaziri, and M. Ghazizadeh-Ahsaee, “Multivariate correlation coefficient and mutual information-based feature selection in intrusion detection,” Information Security Journal: A Global Perspective 26(5), 229–239 (2017). [CrossRef]
49. J. L. Semmlow and B. Griffel, Biosignal and Medical Image Processing, 592 (CRC Press, 2014).
50. T. Nagasawa, T. Sato, I. Nambu, and Y. Wada, “fNIRS-GANs: data augmentation using generative adversarial networks for classifying motor tasks from functional near-infrared spectroscopy,” J. Neural Eng. 17(1), 016068 (2020). [CrossRef]
51. Q. Y. Wang, Y. Lu, X. K. Zhang, and J. Hahn, “Region of interest selection for functional features,” Neurocomputing 422, 235–244 (2021). [CrossRef]
52. T. Fawcett, “An introduction to ROC analysis,” Pattern Recogn Lett 27(8), 861–874 (2006). [CrossRef]
53. M. E. Tipping, “Sparse Bayesian learning and the relevance vector machine,” J Mach Learn Res 1, 211–244 (2001).
54. J. Schrouff, M. J. Rosa, J. M. Rondina, A. F. Marquand, C. Chu, J. Ashburner, C. Phillips, J. Richiardi, and J. Mourao-Miranda, “PRoNTo: Pattern Recognition for Neuroimaging Toolbox,” Neuroinform 11(3), 319–337 (2013). [CrossRef]
55. C. Zhao, Y. Liang, T. Chen, Y. Zhong, X. Li, J. Wei, C. Li, and X. Zhang, “Prediction of cognitive performance in old age from spatial probability maps of white matter lesions,” Aging 12(6), 4822–4835 (2020). [CrossRef]
56. J. Zhu, D. M. Zhu, C. Zhang, Y. Wang, Y. Yang, and Y. Yu, “Quantitative prediction of individual cognitive flexibility using structural MRI,” Brain Imaging Behav 13(3), 781–788 (2019). [CrossRef]
57. H. Kondo, M. Morishita, N. Osaka, M. Osaka, H. Fukuyama, and H. Shibasaki, “Functional roles of the cingulo-frontal network in performance on working memory,” NeuroImage 21(1), 2–14 (2004). [CrossRef]
58. J. M. Chein, A. B. Moore, and A. R. A. Conway, “Domain-general mechanisms of complex working memory span,” NeuroImage 54(1), 550–559 (2011). [CrossRef]
59. C. Ranganath, M. K. Johnson, and M. D’Esposito, “Prefrontal activity associated with working memory and episodic long-term memory,” Neuropsychologia 41(3), 378–389 (2003). [CrossRef]
60. P. Pinti, I. Tachtsidis, A. Hamilton, J. Hirsch, C. Aichelburg, S. Gilbert, and P. W. Burgess, “The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience,” Ann. N.Y. Acad. Sci. 1464(1), 5–29 (2020). [CrossRef]
61. S. Brigadoi, L. Ceccherini, S. Cutini, F. Scarpa, P. Scatturin, J. Selb, L. Gagnon, D. A. Boas, and R. J. Cooper, “Motion artifacts in functional near-infrared spectroscopy: A comparison of motion correction techniques applied to real cognitive data,” NeuroImage 85, 181–191 (2014). [CrossRef]
62. L. Duan, Z. Zhao, Y. Lin, X. Wu, Y. Luo, and P. Xu, “Wavelet-based method for removing global physiological noise in functional near-infrared spectroscopy,” Biomed. Opt. Express 9(8), 3805–3820 (2018). [CrossRef]
63. K. E. Jang, S. Tak, J. Jung, J. Jang, Y. Jeong, and J. C. Ye, “Wavelet minimum description length detrending for near-infrared spectroscopy,” J. Biomed. Opt. 14(3), 034004 (2009). [CrossRef]
64. B. Molavi and G. A. Dumont, “Wavelet-based motion artifact removal for functional near-infrared spectroscopy,” Physiol. Meas. 33(2), 259–270 (2012). [CrossRef]
65. A. M. Chiarelli, E. L. Maclin, M. Fabiani, and G. Gratton, “A kurtosis-based wavelet algorithm for motion artifact correction of fNIRS data,” NeuroImage 112, 128–137 (2015). [CrossRef]
66. M. Muthalib, R. Re, L. Zucchelli, S. Perrey, D. Contini, M. Caffini, L. Spinelli, G. Kerr, V. Quaresima, M. Ferrari, and A. Torricelli, “Effects of Increasing Neuromuscular Electrical Stimulation Current Intensity on Cortical Sensorimotor Network Activation: A Time Domain fNIRS Study,” Plos One 10(7), e0131951 (2015). [CrossRef]
67. Q. Y. Sun, X. R. Wang, B. X. Huang, J. C. J. Sun, J. H. Li, H. W. Zhuang, and G. X. Xiong, “Cortical activation patterns of different masking noises and correlation with their masking efficacy, determined by functional near-infrared spectroscopy,” Front. Hum. Neurosci. 14, 149 (2020). [CrossRef]
68. I. Tachtsidis and F. Scholkmann, “False positives and false negatives in functional near-infrared spectroscopy: issues, challenges, and the way forward,” Neurophotonics 3(3), 031405 (2016). [CrossRef]
69. H. Santosa, X. T. Zhai, F. Fishburn, P. J. Sparto, and T. J. Huppert, “Quantitative comparison of correction techniques for removing systemic physiological signal in functional near-infrared spectroscopy studies,” Neurophotonics 7(03), 035009 (2020). [CrossRef]
70. R. J. Cooper, J. Seib, L. Gagnon, D. Phillip, H. W. Schytz, H. K. Iversen, M. Ashina, and D. A. Boas, “A systematic comparison of motion artifact correction techniques for functional near-infrared spectroscopy,” Front. Neurosci. 6, 147 (2012). [CrossRef]
71. G. Bauernfeind, R. Scherer, G. Pfurtscheller, and C. Neuper, “Single-trial classification of antagonistic oxyhemoglobin responses during mental arithmetic,” Med Biol Eng Comput 49(9), 979–984 (2011). [CrossRef]
72. C. Herff, D. Heger, F. Putze, J. Hennrich, O. Fortmann, and T. Schultz, “Classification of mental tasks in the prefrontal cortex using fNIRS,” Annu Int Conf IEEE Eng Med Biol Soc 2013, 2160–2163 (2013). [CrossRef]
73. H. J. Hwang, J. H. Lim, D. W. Kim, and C. H. Im, “Evaluation of various mental task combinations for near-infrared spectroscopy-based brain-computer interfaces,” J. Biomed. Opt 19(7), 077005 (2014). [CrossRef]
74. S. D. Power, T. H. Falk, and T. Chau, “Classification of prefrontal activity due to mental arithmetic and music imagery using hidden Markov models and frequency domain near-infrared spectroscopy,” J. Neural Eng. 7(2), 026002 (2010). [CrossRef]
75. S. D. Power, A. Kushki, and T. Chau, “Towards a system-paced near-infrared spectroscopy brain-computer interface: differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state,” J. Neural Eng. 8(6), 066004 (2011). [CrossRef]
76. L. C. Schudlo, S. D. Power, and T. Chau, “Dynamic topographical pattern classification of multichannel prefrontal NIRS signals,” J. Neural Eng. 10(4), 046018 (2013). [CrossRef]
77. F. Tan, X. Z. Fu, Y. P. Zhang, and A. G. Bourgeois, “A genetic algorithm-based method for feature subset selection,” Soft Comput 12(2), 111–120 (2007). [CrossRef]
78. R. Battiti, “Using Mutual information for selecting features in supervised neural-net learning,” IEEE Trans. Neural Netw. 5(4), 537–550 (1994). [CrossRef]
79. F. Coelho, A. P. Braga, and M. Verleysen, “Multi-objective semi-supervised feature selection and model selection based on Pearson's correlation coefficient,” Iberoamerican congress on pattern recognition; CIARP 2010, 509–516 (2010).
80. R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, “Convolutional neural networks: an overview and application in radiology,” Insights Imaging 9(4), 611–629 (2018). [CrossRef]
81. S. Arlot and A. Celisse, “A survey of cross-validation procedures for model selection,” Statist. Surv. 4, 40–79 (2010). [CrossRef]
82. N. Unsworth and R. W. Engle, “The nature of individual differences in working memory capacity: active maintenance in primary memory and controlled search from secondary memory,” Psychol Rev 114(1), 104–132 (2007). [CrossRef]
83. C. E. Curtis and M. D’Esposito, “Persistent activity in the prefrontal cortex during working memory,” Trends Cogn Sci 7(9), 415–423 (2003). [CrossRef]
84. R. S. Blumenfeld and C. Ranganath, “Dorsolateral prefrontal cortex promotes long-term memory formation through its role in working memory organization,” J Neurosci 26(3), 916–925 (2006). [CrossRef]
85. R. W. Engle and M. J. Kane, “Executive attention, working memory capacity, and a two-factor theory of cognitive control,” Psychology of Learning and Motivation 44, 145–199 (2003). [CrossRef]
86. J. C. Yu, T. Y. Chang, and C. T. Yang, “Working memory capacity predicts workload capacity,” Visual Cognition 22(8), 1046–1049 (2014). [CrossRef]
87. F. Edin, T. Klingberg, P. Johansson, F. Mcnab, J. Tegner, and A. Compte, “Mechanism for top-down control of working memory capacity,” P. Natl. Acad. Sci. USA 106(16), 6802–6807 (2009). [CrossRef]
