Open Access
Add to BibSonomy
Abstract
One of the primary objectives of the brain-computer interface (BCI) is to obtain a command with higher classification accuracy within the shortest possible time duration. Therefore, this study evaluates several stimulation durations to propose a duration that can yield the highest classification accuracy. Furthermore, this study aims to address the inherent delay in the hemodynamic responses (HRs) for the command generation time. To this end, HRs in the sensorimotor cortex were evaluated for the functional near-infrared spectroscopy (fNIRS)-based BCI. To evoke brain activity, right-hand-index finger poking and tapping tasks were used. In this study, six different stimulation durations (i.e., 1, 3, 5, 7, 10, and 15 s) were tested on 10 healthy male subjects. Upon stimulation, different temporal features and multiple time windows were utilized to extract temporal features. The extracted features were then classified using linear discriminant analysis. The classification results using the main HR showed that a 5 s stimulation duration could yield the highest classification accuracy, i.e., 74%, with a combination of the mean and maximum value features. However, the results were not significantly different from the classification accuracy obtained using the 15 s stimulation. To further validate the results, a classification using the initial dip was performed. The results obtained endorsed the finding with an average classification accuracy of 73.5% using the features of minimum peak and skewness in the 5 s window. The results based on classification using the initial dip for 5 s were significantly different from all other tested stimulation durations (p < 0.05) for all feature combinations. Moreover, from the visual inspection of the HRs, it is observed that the initial dip occurred as soon as the task started, but the main HR had a delay of more than 2 s. Another interesting finding is that impulsive stimulation in the sensorimotor cortex can result in the generation of a clearer initial dip phenomenon. The results reveal that the command for the fNIRS-based BCI can be generated using the 5 s stimulation duration. In conclusion, the use of the initial dip can reduce the time taken for the generation of commands and can be used to achieve a higher classification accuracy for the fNIRS-BCI within a 5 s task duration rather than relying on longer durations.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
A brain-computer interface (BCI) is an encoding/decoding device that acts as a bridge between the brain and peripheral devices and is guided by the brain to direct certain external activities [1]. BCI is also referred to as brain-machine interface, or mind-machine interface, or even direct neural interface. Since 2000, researchers have focused on this field, facilitating the development of several prototype systems. Diverse BCI techniques have been used by healthy subjects for research as well as by patients. One of the primary goals of BCI research is to achieve higher classification accuracy with the lowest possible time duration. Therefore, the stimulation duration to evoke specific neuronal activity needs to be minimized. In this paper, we propose the most favorable stimulation duration for functional near-infrared spectroscopy (fNIRS)-based BCI, through which the highest classification accuracy can be obtained.
A variety of techniques exist in acquiring brain signals. These include electroencephalography (EEG) [2–5], fNIRS [6–10], functional magnetic resonance imaging (fMRI) [11–14], and magnetoencephalography (MEG) [15,16], to name a few. fNIRS is a non-invasive imaging technique that uses near-infrared light within a range of 650-1,000 nm to measure the regional brain blood flow [17–19]. Oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) are the two light-absorbing blood chromophores [20] measured by fNIRS. The local concentration changes of HbO/HbR, referred to as the hemodynamic response (HR), reflect the existence of neuronal activities in the local area, and the increase of HbO indicates the consumption of oxygen for those neuronal activities [21].
fNIRS has incredible application potential as an appropriate neuroimaging instrument, which includes neurodevelopment analysis in society [22], discernment and perception [23], psychiatric condition monitoring [24], language experiment research [25], stroke, and brain damage [26], clinical and network imaging [27], and BCIs [28]. Additionally, fNIRS provides preferences, such as flexibility and minimal effort, and lower susceptibility to motion artifacts than fMRI, EEG, and positron emission tomography [29]. Furthermore, fNIRS has good time resolution and intermediate spatial resolution. Improvements in spatial and temporal resolution are also in progress. Such techniques, including bundled optodes and initial dip detection, are among them [30,31]. Initial dip detection can decrease the time required for the generation of BCI commands [32]. The work on prediction algorithms is also one of the major focus in the field of initial dip and can open several new avenues in future for quick generation of command [33–35]. For the existence of an initial dip, interested readers can refer to Hong and Zafar [36].
The existing fNIRS studies in the literature have reported various stimulation durations and different types of tasks. The stimulation duration used for the same purpose was not the same across studies. Also, some researchers used different stimulation durations when brain areas were different. Mental arithmetic, mental counting, puzzle-solving, etc., are typical stimulation types for the prefrontal cortex. Thermal stimulation [37,38], electric stimulation [39,40], painful pressures [41], and poking [42] have been applied in the studies for the sensory cortex. Finger tapping for the motor cortex and checkerboard for the visual cortex [43–45] are the most common stimulation types.
Several fMRI studies have addressed the link between blood-oxygen-level-dependent (BOLD) signals and the stimulation duration [46]. Studies on the primary visual cortex indicate that for stimuli below 3-4 s, the BOLD signals are non-linear. After these durations, they are linear because the profile follows a particular mathematical relation [47,48]. Similarly, research on the primary auditory cortex [49] showed a discrepancy in the BOLD signal pattern for stimuli below 6 s and the BOLD signal activity for stimuli greater than this interval. A recent study by Lewis et al. [50] examined the dependence of the BOLD timing on the human visual pathway in the subcortical and cortical regions. This study showed that the BOLD response in both the subcortical and cortical areas differed by changing the stimulation duration. The authors found that the BOLD response in the subcortical areas was quicker than in the cortical areas.
Some studies have attempted to vary the types of stimulation in fNIRS and to check their effects on the HR signals. The effect of different intensities of noise stimuli on the HR has been demonstrated in a previous study [51]. The noise was modulated at four different intensity levels. The study indicated that the fNIRS response depended significantly on the sound intensity, i.e., the greater the sound intensity, the greater the change in concentration. The effects of checkerboards on the fNIRS responses were recorded in another study [52]: The variability in the HR response was verified by changing the checkerboard angles. To achieve this, 1, 2, 5, 9, and 18 deg-checking squares were used to adjust the visual angles of the squares in the checkerboard task. The findings showed that the most significant activation was observed for the 1 deg stimulus. The reversed, on/off, and static checkerboards were also examined. Their findings showed that the greatest activation among the three tasks was the result of pattern reversal stimulation. Likewise, the influence of a varying anagram task on the prefrontal cortex was examined [53]. There have also been studies on the occipital and temporal responses to stimulus reproduction in infants [54]. However, in fNIRS research, the effect of the stimulation duration on the HR signal from different brain cortices has not been well documented. Various stimulation periods have also been used for BCI in the past; however, no standardized stimulation duration has been determined considering that various stimulation durations have been used.
In the authors’ previous paper [55], we have investigated six different stimulation durations (1, 3, 5, 7, 10, and 15 s) that would give the highest peak in the visual, motor, and somatosensory cortices, using checkerboard, tapping, and poking tasks. Three findings include: The stimulation durations giving the highest peak HbO value for the checkerboard, tapping, and poking tasks were 7, 10, and 5 s, respectively. The most petite stimulation duration generating a hemodynamic response was 1s, which was the same for all three tasks. Interestingly, for poking task, for the average results the initial dip occurred with 1 s stimulation duration but did not occur with other stimulation durations beyond 1 s.
On top of the previous work, this paper pursues finding the best stimulation duration for classification between two different brain regions (motor cortex and somatosensory cortex, specifically, tapping vs. poking). For this purpose, the data from the same brain area (i.e., the sensorimotor cortex) reported in [55] are used. The objective is to determine the most suitable stimulation duration to achieve the highest classification accuracies.
2. Methods
2.1 Participants
The experiment was carried out on 10 male participants (mean age: 27.2 ± 4.5 years). All the participants had normal or corrected-to-normal sight and were checked to have the dominant right hand (the hand dominance was checked using Edinburgh Handedness Test) to eliminate the possible effects of hand dominance to the cortical activity in sensorimotor control [56]. None of the participants had any history of neurological or visual disabilities. All participants received a full explanation and gave written consent prior to the experiment. After obtaining permission from the Institutional Review Board of Pusan National University, this study was conducted in compliance with the recent proclamation of Helsinki [57].
2.2 Stimulation design
Two different tasks were performed by the subjects in the current study. These were the right-hand index finger tapping (it will be simply called tapping) and right-hand index finger poking (simply, poking). Six different stimulation durations (Δt = 1, 3, 5, 7, 10, and 15 s) were tested to measure the influence of a stimulation duration on the HR. The subjects were advised to perform the task on the screen during the experiment. For the tapping task, the subjects were asked to tap their right-hand index finger as quickly as possible. In the poking task, the subjects were poked on the backside of the right-hand index finger at random positions with a frequency of 2 Hz. Snapshots of both tasks are given in Fig. 1.
2.3 Experimental paradigm
The assignment of tasks during the experiment was random, i.e., during the experiment, the subjects were advised to perform the current task unaware of what task would appear next on the screen. In total, 12 random trials were conducted, which consisted of six trials for each task. The left sensorimotor cortex was examined during these two tasks. A comfortable chair was provided for the participants, and they were instructed to avoid body movements during the experiment as much as possible. The experiments were conducted in a dark room, and to reduce the effect of external noise, the room was kept as quiet as possible.
The experimental paradigm with the initial rest, inter-stimulus interval, and the final rest is shown in Fig. 2. There was a 20 s inter-stimulation interval for all stimulation durations, whereas the pre-and post-rest periods were 60 s and 30 s, respectively. During all the rest periods, a black screen was displayed. Visual displays were given for both tasks on a computer screen placed in front of the subjects. The subjects were instructed to keep their eyes open throughout the experiment.
2.4 Optode configuration
To acquire the brain signals, two detectors and eight emitters were positioned over the left sensorimotor cortex area of the brain. Figure 3 shows the optode configuration in the region of interest. The source-detector separation for most channels (except Channels 5 and 9) was kept between 3 and 3.5 cm [58]. For the precise placement of the optodes on the left sensorimotor cortex area of the brain, the C3 area in the brain was chosen as a reference point. The selection of the reference point was in accordance with the International 10-20 System for electrode placement.
2.5 Data acquisition
For both tasks (tapping and poking), the signals were sampled at a sampling frequency of 15.625 Hz from the left sensorimotor cortex. For the acquisition of fNIRS data, a frequency-domain fNIRS system (Imagent, ISS Medical Inc., IL, USA) was used. Raw intensity data were acquired using the ISS Imagent data acquisition and analysis software (ISS-Boxy). To calculate the concentration changes of HbO (ΔHbO) and HbR (ΔHbR), two wavelengths (690 nm and 830 nm) were used by the system. The modified Beer-Lambert law was used to convert the raw intensity data into ΔHbO and ΔHbR [59]. In the case of the 690 nm wavelength, the extinction coefficients were 0.95 mM-1 cm-1 and 4.93 mM-1 cm-1 for ΔHbO and ΔHbR, respectively, and for the wavelength of 830 nm, the values were 2.135 mM-1 cm-1 and 1.791 mM-1 cm-1, respectively [60].
2.6 Signal preprocessing
The data (ΔHbO & ΔHbR) were preprocessed, after the acquisition, to eliminate physiological noise contamination. To avoid motion artifacts, the subjects were instructed not to move during the experiment. To eliminate cardiac signals (≈1 Hz), respiratory signals (0.2 ∼ 0.4 Hz), and low-frequency drift signals, a 4th-order Butterworth bandpass filter was applied with a low-pass cutoff frequency at 0.15 Hz [61]. The frequency of the high-pass filter was chosen based on the longest possible time period of trials. Owing to the differences in the time period of a trial, the cutoff frequency for each of the six stimulation durations was different. The cutoff frequency was set to 0.028 Hz (1/35 s = 0.028 Hz) for the 15 s task, as the trial period was 35 s. Similarly, for the 1 s (trial period = 21 s), 3 s (trial period = 23 s), 5 s (trial period = 25 s), 7 s (trial period = 27 s), and 10 s (trial period = 30 s) stimulations, the cutoff frequencies were set to 0.047, 0.043, 0.04, 0.037, and 0.033 Hz, respectively.
In fNIRS, the existence of neuronal activation is determined by the t-statistics analysis of the measured HbO data with the desired hemodynamic response function (dHRF), which is expected to occur upon neuronal activity. The dHRF was generated using the convolution integral of a canonical hemodynamic response function (cHRF) with the stimulation duration in each task [60]. The shapes of dHRFs vary with different stimulation durations. The cHRF adopted in this study took the form of a three-gamma function, which characterizes the initial dip period, the positive activation period, and the final undershoot period in time series. The cHRF plays a vital role in the statistical analyses, as its shape may vary among various cortices (motor, sensory, visual, prefrontal, etc.). In this paper, the same cHRF was used for both motor and sensory cortices. The dHRFs generated for each stimulation duration are shown in Fig. 4.
Fig. 4. Hemodynamic responses anticipated from six different stimulation durations: Computed using three gamma functions [55].
The vector phase diagram with dual threshold circles in Fig. 5 was used to detect the initial dip. To generate the vector phase diagram, four vector components (ΔHbO, ΔHbR, concentration changes of cerebral oxygen exchange (ΔCOE), and total hemoglobin (ΔHbT)) were used as indices. The conventional HR occurs in the shaded areas of Phases 7 and 8, whereas the shaded area of Phases 3, 4, and 5 in blue color indicate the initial dip regions.
Fig. 5. A vector phase diagram with dual threshold circles depicting the HR region (yellow) and the initial dip region (blue) [57].
2.7 Feature extraction and classification
After preprocessing the data, eight temporal features of individual trials (for each stimulation duration and each task) were extracted. They were the mean of the main HR (from 4 s to Δt + extra time), maximum (a peak between 0 and Δt + extra time), skewness, kurtosis, increasing slope (from 2 s to Δt + extra time), increasing mean (from 2 s to Δt + extra time), decreasing slope (from 7 s to Δt + extra time, except for 10 s and 15 s), and decreasing mean (from 7 s to Δt + extra time, except for 10 and 15 s) were extracted for all subjects. For classification, For the various methods exist like linear discriminant analysis (LDA), neural networks etc. [62–64]. In this study, paired features were used in the LDA, making a total of 28 different feature combinations (i.e., two out of eight features) [65].
It is noted that different window sizes were used to extract the temporal features due to different stimulation durations. For the main HR of 1 s stimulation, a 4∼10 s window was used to calculate the overall mean, skewness, and kurtosis; whereas the increasing slope and increasing mean were obtained from a 2 to 7 s window, and the decreasing slope and decreasing mean were calculated from a 7 to 10 s window. For the 3 s stimulation, a 4∼13 s window was used to compute the overall mean, skewness, and kurtosis; the increasing slope and increasing mean from 2 to 7 s; and the decreasing slope and decreasing mean from 7 to 13 s. For the 5 s stimulation, a 4∼13 s window for the 5 s stimulation was used for mean, skewness, and kurtosis; the increasing slope and mean from 2 to 8 s; and the decreasing slope and decreasing mean from 7 to 13 s. For the 7 s stimulation, a 4∼15 s window was used to get the mean, skewness, and kurtosis; the increasing slope and increasing mean from 2 to 7 s; and the decreasing slope and decreasing mean from 7 to 15 s. For the 10 s stimulations, a 4∼14 s window was used for the mean, skewness, and kurtosis; the increasing slope and increasing mean from 2 to 11 s; and the decreasing slope and decreasing mean from 11 to 14 s. Finally, for the 15 s stimulations, a 4∼18 s window was used for the overall mean, skewness, and kurtosis; the increasing slope and increasing mean form from 2 to 10 s; and the decreasing slope and decreasing mean from 15 to 18 s. Six-fold cross-validation was performed to determine the classification accuracy.
For the initial dip part, three different window sizes were used, i.e., 0∼1 s, 0∼2.5 s, and 0∼4 s. The computed temporal features were the mean, minimum, slope, skewness, and kurtosis. These features were also used in pairs, making a total of 10 different feature combinations for classification. Like the main HR case, LDA and six-fold cross-validation were used for classification.
2.8 Statistical analysis
The t-value, p-value, and the mean of the averaged ΔHbO (over all trials) were used to identify the most active channels. In this study, tcrt was selected distinctively for each stimulus according to the degree of freedom and statistical tables. The tcrt was set to 1.6495 for the 1 s task, considering that the trial period = 21 s (1 s task + 20 s rest), the number of data points N = 21 × 15.625 = 328.125, and N-1 = 327.125. Similarly, for the 3 s (trial period = 23 s), 5 s (trial period = 25 s), 7 s (trial period = 27 s), 10 s (trial period = 30 s), and 15 s (trial period = 30 s) stimulations, the tcrt values were set to 1.6491, 1.6488, 1.6485, 1.6481, and 1.6477 respectively. For the one-tailed t-test, the statistical significance level was set to 0.05. To calculate t-values, the built-in function robustfit in MATLAB was used. A channel with p < 0.05 and t > tcrt indicates that the channel was activated.
3. Results
3.1 Comparison of brain activation
The averages of ΔHbO concentration changes evoked by six different stimulation durations for both tasks are shown in Fig. 6. All the stimulation durations show activation. The shorter the duration is, the higher the variance of individual subjects is. But, the average responses across the subjects are distinguishable. It is noted that after the start of the stimulation, most HRs achieved their peaks between 5 and 17 s.
Fig. 6. Comparison of the averaged HbOs over the activated channels of all subjects: (a) 1 s, (b) 3 s, (c) 5 s, (d) 7 s, (e) 10 s, and (f) 15 s tasks.
Furthermore, it was observed that the HR of the shorter stimulation duration, in comparison with the longer stimulation duration in the area under observation, was narrowly spread and occurred faster. The peaks of the mean ΔHbO responses for the 1 s tapping and poking tasks appeared at around 5.3 s and 6 s, respectively.
3.2 Classification with the main hemodynamic response
The two-class classification of both tasks was performed for the HRs by utilizing different feature combinations. The obtained classification accuracies for 1, 3, 5, 7, 10, and 15 s were averaged. The averaged classification accuracies obtained over all subjects for each feature combination are shown in Table 1. The average classification accuracies achieved from the 5 s and 15 s stimulation durations were almost close to 74%. The overall classification accuracies were significantly higher than the chance level (i.e., 50%).
Table 1. Comparison of averaged classification accuracies (for main hemodynamics) of all subjects for all features (mean, maximum, skewness, kurtosis, increasing mean, increasing slope, decreasing mean, and decreasing slope) in pairs.a
3.3 Classification using initial dip
First, using the MATLAB function robustfit, the t-values were computed to confirm whether the channels showing initial dips were active. This function compares the averaged ΔHbO of each channel with the designed HR and yields t- and p-values. Dual thresholds-based vector phase analysis was used to detect the initial dip. In the vector phase analysis, the t-values of the initial dip-detected channels were higher than tcrt. Figure 7 is an example of the vector phase plots of the trials showing an initial dip for all the stimulation durations (Subject 1).
Fig. 7. Comparison of the trajectories of six different task durations followed by 10 s rest for subject 1.
The averaged classification accuracies over all the subjects for each feature combination for three different window sizes are shown in Table 2. Notably, the 5 s stimulation duration (like the main HR case) showed high classification accuracies even from the windows of initial dip: The accuracies were 72.1, 73.5, and 70.1 for the 1 s, 2.5 s, and 4 s windows, respectively. These accuracies were relatively high than the chance level of the two-class classification. From Table 2, it is observed that the initial dip-based classification performed well for all three windows. Overall, it is observed that the 5 s stimulation duration yielded the best classification accuracy in both the initial dip and the main HR periods.
Table 2. Averaged classification accuracies (for initial dip) of all subjects for all features: mean, slope, minimum (min), kurtosis (kurt), skewness (skew).
4. Discussions
This study aimed to evaluate the variability in the temporal characteristics of HR (i.e., ΔHbO) in the brain’s sensorimotor cortex. Two different tasks (i.e., the right-hand index finger tapping and poking) were utilized to evoke the neural activity. For classification, an LDA classification was used for the tasks performed for the six different stimulation durations. The objective is to propose a suitable stimulation duration that can yield the best classification accuracy. One of the core research issues of BCI is to shorten the command generation time, which is related to the stimulation duration. Therefore, we focused on shorter durations rather than the conventionally used stimulation durations (i.e., 30 s and 60 s of mental tasks for the prefrontal cortex). To the best of our knowledge, this is the first study in fNIRS, which explores a classification-based stimulation duration. As a result, the most suitable stimulation duration was suggested.
The focus of the current study lies primarily on the two crucial brain areas, the motor and somatosensory cortices (collectively, the sensorimotor cortex) [66,67]. From the visual inspection of the HRs, it is seen that the peak values are different over the stimulation durations. An increase in stimulation duration does not guarantee an increase in peak value [55]. The decrease in the peak value after a certain stimulation duration may be deduced from the hypothesis that the neural activity becomes neutral if the stimulation duration gets longer, and therefore the strength of the hemodynamic signal decreases. In the poking task, because of repetitive poking in longer stimulation intervals, the finger can become numb to any stimulation. These two observations make reasonable grounds for preferring a shorter stimulation duration over the longer one.
Classification using LDA is one of the most common techniques in fNIRS-based BCIs [68,69]. Initially, the classification was performed for the main HR. Considering the guidelines from the previous studies [70], the window size ranged from 4 s to the end of the main hemodynamic response. Using these windows, eight different temporal features were extracted for each stimulation duration, yielding a total of 28 different feature sets. The averaged classification accuracies for 28 feature sets are shown in Table 1. The classification accuracies of the individual subjects were all higher than the chance level (i.e., 50%).
The work of Power et al. [71] achieved an average classification accuracy of 56.2% for two tasks (mental arithmetic and mental singing). In a recent work [72], the left-hand and right-hand index finger tapping were classified using LDA: The classification accuracies obtained using temporal features were around 70%. The studies in [73,74] have reported classification accuracies above 90% for task vs. rest classification. Another study [75] on binary classification reported an average classification accuracy of 77.4%. In most of these studies, task vs. rest classification was performed. In the current work, tapping and poking tasks (for six stimulation durations) were classified. Interestingly, the two task durations (i.e., 5 s and 15 s) outperformed all other stimulation durations, and yielded high classification accuracies above 70% in most feature combinations.
To shorten the BCI time, the initial dip classification was pursued besides the conventional HR-based classification (note that early fNIRS studies used a large window size; for instance, 0∼10, 0∼15, 0∼17, 0∼20 s, etc. [76–79]). For initial dip classification, we designed three different window sizes (0∼1, 0∼2.5, and 0∼4 s) and evaluated five different features (mean, slope, minimum, skewness, and kurtosis). Using the combination of signal mean and minimum value with the window size of 0∼4 s, an average classification accuracy of 73.97% was achieved, which was the best in the case of the initial dip. According to the previous studies [80–82], the initial dip peaks occur at approximately 2 s and are complete at approximately 4 s. From the classification results based on the initial dip (Table 2), the 5 s stimulation duration yields the best classification accuracy in the initial dip period too. Comparing the accuracies both the main HR and the initial dip, it is concluded that the best stimulation duration in classifying tapping and poking tasks is 5 s. This finding may lead to one step toward a real-time BCI with no need to wait until the main HR occurs. Another thing to note is that the initial dip phenomenon is specific to a brain region. If we wait for the main HR to occur, a wider brain region will be activated in time, resulting in lower classification accuracy.
Impulsive stimulation in the somatosensory cortex may magnify the initial dip. Subsequently, the classification accuracy may improve for a shorter stimulation interval. In some trials longer than 2 s, no initial dip was observed. The literature says that several factors can contribute to this. For example, caffeine decreases the likelihood of determining the initial dip [83,84]. Additionally, individual differences can contribute to the variance in classification accuracy among the subjects [85]. The phenomena of the initial dip for classification have been utilized previously in different brain areas. Our results show that initial dips in the sensorimotor cortex region could be successfully detected by vector phase analysis. During 2∼2.5 s, the initial dip peaking was observed, which conforms to the previous fNIRS and fMRI studies [86]. The completion time of the initial dip period varies from 3.5 to 4 s. The magnitude of the initial dip may vary in different brain regions. Therefore, further evaluation is necessary to fully examine the nature of differentiating the initial dips in different brain regions. The limitations of this study are the following: First, the temporal features of only HbO signals were used. To further improve the initial dip-based classification accuracy, the HbR trend (or COE and HbT) should be simultaneously evaluated. Second, for the preprocessing, a band-pass filter was used for filtration. Third, a modern technique in handling motion artifacts can be implemented as well. Alongside, advanced control techniques [87, 88] can further validate the results of the current study. Fourth, the insufficient number of subjects was also a limitation of this study. The results would be more acceptable if experiments were performed with a larger number of subjects. Fifth, only a linear classifier (LDA) was investigated in this study. More advanced classifiers [89–91] can improve classification accuracies. Sixth, in this work, the relative changes in the hemodynamic response were examined. The analysis of the absolute concentration values may result in another new finding. Furthermore, the focus of the current study is only limited to the sensorimotor cortex area of the brain; however, more interesting tasks such as the checkerboard, puzzle-solving, mental asthmatics, noise, and touching can be utilized in future studies. For future work, other brain areas need to be explored to validate the claim of this study further.
5. Conclusion
In this study, the stimulation duration that can yield the highest classification accuracy was recommended for fNIRS-based BCI. In association with the sensorimotor cortex, two different tasks, namely, right-hand index finger tapping and poking, were performed. Classification was performed using the main hemodynamic response signal and initial dip. Stimulation durations of 5 s and 15 s yielded the highest classification accuracies using the main HR for classification. The 5 s stimulation duration yielded the best classification accuracy when it came to classification using the initial dip. The results of the current study are empowering and indicate a significant potential for the reduction of the stimulation duration and use of the initial dip for fNIRS-based BCI up to 5 s, leading to a noteworthy improvement in the temporal resolution.
Funding
National Research Foundation of Korea (NRF-2020R1A2B5B03096000); Ministry of Science and ICT, South Korea (NRF-2020R1A2B5B03096000).
Acknowledgments
The authors would like to thank all volunteers who participated in the study.
Disclosures
The authors declare no conflict of interest.
Data availability
The datasets generated for this study can be made available on request to the corresponding author.
References
1. J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, T. M., and Vaughn, “Brain–computer interfaces for communication and control,” Clin. Neurophysiol. 113(6), 767–791 (2002). [CrossRef]
2. C. T. Lin, J. T. King, C. H. Chuang, W. P. Ding, W. Y. Chuang, L. D. Liao, and Y. K. Wang, “Exploring the brain responses to driving fatigue through simultaneous EEG and fNIRS measurements,” Int. J. Neur. Syst. 30(01), 1950018 (2020). [CrossRef]
3. C. Porcaro, S. D. Mayhew, M. Marino, D. Mantini, and A. P. Bagshaw, “Characterization of hemodynamic activity in resting state networks by fractal analysis,” Int. J. Neur. Syst. 30(12), 2050061 (2020). [CrossRef]
4. F. Laport, A. Dapena, P. M. Castro, F. J. Vazquez-Araujo, and D. Iglesia, “A prototype of EEG system for IoT,” Int. J. Neur. Syst. 30(07), 2050018 (2020). [CrossRef]
5. R. J. Deligani, S. B. Borgheai, J. McLinden, and Y. Shahriari, “Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework,” Biomed. Opt. Express 12(3), 1635–1650 (2021). [CrossRef]
6. N. Naseer and K.-S. Hong, “fNIRS-based brain-computer interfaces: a review,” Front. Hum. Neurosci. 9, 172 (2015). [CrossRef]
7. K.-S. Hong and M. A. Yaqub, “Application of functional near-infrared spectroscopy in the healthcare industry: A review,” J. Innov. Opt. Health Sci. 12(06), 1930012 (2019). [CrossRef]
8. L. Duan and X. Q. Mai, “Spectral clustering-based resting-state network detection approach for functional near-infrared spectroscopy,” Biomed. Opt. Express 12(3), 1635–1650 (2021). [CrossRef]
9. U. Asgher, M. J. Khan, M. H. A. Nizami, K. Khalil, R. Ahmad, Y. Ayaz, and N. Naseer, “Motor training using mental workload (MWL) with an assistive soft exoskeleton system: a functional near-infrared spectroscopy (fNIRS) study for brain–machine interface (BMI),” Front. Neurorobot. 15, 605751 (2021). [CrossRef]
10. J. W. Yu, S. H. Lim, B. Kim, E. Kim, K. Kim, S. Park, Y. Byun, J. Sakong, and J. W. Choi, “Prefrontal functional connectivity analysis of cognitive decline for early diagnosis of mild cognitive impairment: a functional near-infrared spectroscopy study,” Biomed. Opt. Express 11(4), 1725–1741 (2020). [CrossRef]
11. D. Vidaurre, “A new model for simultaneous dimensionality reduction and time-varying functional connectivity estimation,” PLoS Comput. Biol. 17(4), e1008580 (2021). [CrossRef]
12. F. Fang, T. Potter, T. Nguyen, and Y. C. Zhang, “Dynamic reorganization of the cortical functional brain network in affective processing and cognitive reappraisal,” Int. J. Neur. Syst. 30(10), 2050051 (2020). [CrossRef]
13. C. C. Lau, K. Yuan, P. Wong, W. C. W. Chu, T. W. Leung, W. W. Wong, and R. K. Y. Tong, “Modulation of functional connectivity and low-frequency fluctuations after brain-computer interface-guided robot hand training in chronic stroke: a six-month follow-up study,” Front. Hum. Neurosci. 14, 614 (2020). [CrossRef]
14. D. J. Davies, K. M. Yakoub, Z. J. Su, M. Clancy, M. Forcione, S. J. E. Lucas, H. Dehghani, and A. Belli, “The Valsalva maneuver: an indispensable physiological tool to differentiate intra versus extracranial near-infrared signal,” Biomed. Opt. Express 11(4), 1712–1724 (2020). [CrossRef]
15. H. L. Halme and L. Parkkonen, “Across-subject offline decoding of motor imagery from MEG and EEG,” Sci. Rep. 8(1), 1–12 (2018). [CrossRef]
16. A. O. Ovchinnikova, A. N. Vasilyev, I. P. Zubarev, B. L. Kozyrskiy, and S. L. Shishkin, “MEG-based detection of voluntary eye fixations used to control a computer,” Front. Neurosci. 15, 619591 (2021). [CrossRef]
17. A. Pellicer and M. del Carmen Bravo, “Near-infrared spectroscopy: A methodology-focused review,” Semin. Fetal Neonatal Med. 16(1), 42–49 (2011). [CrossRef]
18. H. Santosa, M. J. Hong, S. P. Kim, and K.-S. Hong, “Noise reduction in functional near-infrared spectroscopy signals by independent component analysis,” Rev. Sci. Instrum. 84(7), 073106 (2013). [CrossRef]
19. M. Cope and D. T. Delpy, “System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination,” Med. Biol. Eng. Comput. 26(3), 289–294 (1988). [CrossRef]
20. F. Montani, A. Oliynyk, and L. Fadiga L, “Superlinear summation of information in premotor neuron pairs,” Int. J. Neur. Syst. 27(02), 1650009 (2017). [CrossRef]
21. A. Villringer, J. Planck, C. Hock, L. Schleinkofer, and U. Dirnagl, “Near infrared spectroscopy (NIRS): a new tool to study hemodynamic changes during activation of brain function in human adults,” Neurosci. Lett. 154(1-2), 101–104 (1993). [CrossRef]
22. H. Watanabe, Y. Shitara, Y. Aoki, T. Inoue, S. Tsuchida, N. Takahashi, and G. Taga, “Hemoglobin phase of oxygenation and deoxygenation in early brain development measured using fNIRS,” Proc. Natl. Acad. Sci. U. S. A. 114(9), E1737–E1744 (2017). [CrossRef]
23. S. Cutini, S. B. Moro, and S. Bisconti, “Functional near infrared optical imaging in cognitive neuroscience: an introductory review,” J. Near Infrared Spectrosc. 20(1), 75–92 (2012). [CrossRef]
24. U. Ghafoor, J. H. Lee, K.-S. Hong, S. S. Park, J. Kim, and H. R. Yoo, “Effects of acupuncture therapy on MCI patients using functional near-infrared spectroscopy,” Front. Aging Neurosci. 11, 237 (2019). [CrossRef]
25. Z. Wang, Y. Zhou, L. Chen, B. Gu, W. Yi, S. Liu, M. Xu, H. Qi, F. He, and D. Ming, “BCI monitor enhances electroencephalographic and cerebral hemodynamic activations during motor training,” IEEE Trans. Neural Syst. Rehabil. Eng. 27(4), 780–787 (2019). [CrossRef]
26. P. C. Petrantonakis and I. Kompatsiaris, “Single-trial NIRS data classification for brain–computer interfaces using graph signal processing,” IEEE Trans. Neural Syst. Rehabil. Eng. 26(9), 1700–1709 (2018). [CrossRef]
27. Y. Zheng, D. Zhang, L. Wang, Y. Wang, H. Deng, S. Zhang, D. Li, and D. Wang, “Resting-state-based spatial filtering for an fNIRS-based motor imagery brain-computer interface,” IEEE Access 7, 120603–120615 (2019). [CrossRef]
28. R. A. Khan, N. Naseer, N. K. Qureshi, F. M. Noori, H. Nazeer, and M. U. Khan, “fNIRS-based Neurorobotic Interface for gait rehabilitation,” J NeuroEngineering Rehabil 15(1), 7 (2018). [CrossRef]
29. K. Wiencke, A. Horstmann, D. Mathar, A. Villringer, and J. Neumann, “Dopamine release, diffusion and uptake: A computational model for synaptic and volume transmission,” PLoS Comput. Biol. 16(11), e1008410 (2020). [CrossRef]
30. H. D. Nguyen and K.-S. Hong, “Bundled-optode implementation for 3D imaging in functional near-infrared spectroscopy,” Biomed. Opt. Express 7(9), 3491–3507 (2016). [CrossRef]
31. U. Ghafoor, S. Kim, and K.-S. Hong, “Selectivity and longevity of peripheral-nerve and machine interfaces: a review,” Front. Neurorobot. 11, 59 (2017). [CrossRef]
32. K.-S. Hong and N. Naseer, “Reduction of delay in detecting initial dips from functional near-infrared spectroscopy signals using vector-based phase analysis,” Int. J. Neur. Syst. 26(03), 1650012 (2016). [CrossRef]
33. Y. Ghoul, “Prediction error identification method for continuous-time systems having multiple unknown time delays,” Int. J. Control Autom. Syst. 18(9), 2268–2276 (2020). [CrossRef]
34. A. Zafar and K.-S. Hong, “Reduction of onset delay in functional near-infrared spectroscopy: prediction of HbO/HbR signals,” Front. Neurorobotics 14, 10 (2020). [CrossRef]
35. F. Kamil, T. S. Hong, W. Khahsar, N. Zulkifli, and S. A. Ahmad, “An ANFIS-based optimized fuzzy-multilayer decision approach for a mobile robotic system in ever-changing environment,” Int. J. Control Autom. Syst. 17(1), 253–266 (2019). [CrossRef]
36. K.-S. Hong and A. Zafar, “Existence of initial dip for BCI: an illusion or reality,” Front. Neurorobotics 12, 69 (2018). [CrossRef]
37. S. Sorkpor, H. Ahn, L. Pollonini, and J. Do, “Cortical hemodynamic response to contact thermal stimuli in older adults with knee osteoarthritis: a functional near infrared spectroscopy pilot study,” J. Pain 20(4), S33 (2019). [CrossRef]
38. K. C. Phillips, D. Verbrigghe, A. Gabe, B. Jauquet, C. Eischer, and T. Yoon, “The influence of thermal alterations on prefrontal cortex activation and neuromuscular function during a fatiguing task,” Int. J. Environ. Res. Public Health 17(19), 7194 (2020). [CrossRef]
39. D. Yang, Y. I. Shin, and K.-S. Hong, “Systemic review on transcranial electrical stimulation parameters and EEG/fNIRS features for brain diseases,” Front. Neurosci. 15, 274 (2021). [CrossRef]
40. M. Ortiz, E. lanez, J. A. Gaxiola, D. Gutierrez, and J. M. Azorin, “Study of the functional brain connectivity and lower-limb motor imagery performance after transcranial direct current stimulation,” Int. J. Neur. Syst. 30(08), 2050038 (2020). [CrossRef]
41. M. A. Yücel, C. M. Aasted, M. P. Petkov, D. Borsook, D. A. Boas, and L. Becerra L, “Specificity of hemodynamic brain responses to painful stimuli: a functional near-infrared spectroscopy study,” Sci. Rep. 5(1), 9469 (2015). [CrossRef]
42. K.-S. Hong, M. R. Bhutta, X. Liu, and Y. I. Shin, “Classification of somatosensory cortex activities using fNIRS,” Behav. Brain Res. 333, 225–234 (2017). [CrossRef]
43. S. M. Guérin, M. A. Vincent, C. I. Karageorghis, and Y. N. Delevoye-Turrell, “Effects of motor tempo on frontal brain activity: An fNIRS Study,” NeuroImage 230, 117597 (2021). [CrossRef]
44. L. Pu, N. K. Qureshi, J. Ly, B. Zhang, F. Cong, W. C. Tang, and Z. Liang, “Therapeutic benefits of music-based synchronous finger tapping in Parkinson’s disease—an fNIRS study protocol for randomized controlled trial in Dalian, China,” Trials 21(1), 1–14 (2020). [CrossRef]
45. J. A. Noah, X. Zhang, S. Dravida, C. DiCocco, T. Suzuki, R. N. Aslin, I. Tachtsidis, and J. Hirsch, “Comparison of short-channel separation and spatial domain filtering for removal of non-neural components in functional near-infrared spectroscopy signals,” Neurophotonics 8(1), 015004 (2021). [CrossRef]
46. G. M. Boynton, S. A. Engel, G. H. Glover, and D. J. Heeger, “Linear systems analysis of functional magnetic resonance imaging in human V1,” J. Neurosci. 16(13), 4207–4221 (1996). [CrossRef]
47. A. L. Vazquez and D. C. Noll, “Nonlinear aspects of the BOLD response in functional MRI,” Neuroimage 7(2), 108–118 (1998). [CrossRef]
48. D. A. Soltysik, K. K. Peck, K. D. White, B. Crosson, and R. W. Briggs, “Comparison of hemodynamic response nonlinearity across primary cortical areas,” Neuroimage 22(3), 1117–1127 (2004). [CrossRef]
49. M. D. Robson, J. L. Dorosz, and J. C. Gore, “Measurements of the temporal fMRI response of the human auditory cortex to trains of tones,” Neuroimage 7(3), 185–198 (1998). [CrossRef]
50. L. D. Lewis, K. Setsompop, B. R. Rosen, and J. R. Polimeni, “Stimulus-dependent hemodynamic response timing across the human subcortical-cortical visual pathway identified through high spatiotemporal resolution 7T fMRI,” Neuroimage 181, 279–291 (2018). [CrossRef]
51. S. Weder, X. Zhou, M. Shoushtarian, H. Innes-Brown, and C. McKay, “Cortical processing related to intensity of a modulated noise stimulus—a functional near-infrared study,” JARO 19(3), 273–286 (2018). [CrossRef]
52. N. H. Kashou and B. M. Giacherio, “Stimulus and optode placement effects on functional near-infrared spectroscopy of visual cortex,” Neurophotonics 3(2), 025005 (2016). [CrossRef]
53. F. Tian, B. Chance, and H. Liu, “Investigation of the prefrontal cortex in response to duration-variable anagram tasks using functional near-infrared spectroscopy,” J. Biomed. Opt. 14(5), 054016 (2009). [CrossRef]
54. L. L. Emberson, G. Cannon, H. Palmeri, J. E. Richards, and R. N. Aslin, “Using fNIRS to examine occipital and temporal responses to stimulus repetition in young infants: Evidence of selective frontal cortex involvement,” Dev. Cogn. Neurosci. 23, 26–38 (2017). [CrossRef]
55. M. N. A Khan, M. R. Bhutta, and K.-S. Hong, “Task-Specific stimulation duration for fNIRS brain-computer interface,” IEEE Access 8, 89093–89105 (2020). [CrossRef]
56. K. Martin, S. Jacobs, and S. H. Frey, “Handedness-dependent and -independent cerebral asymmetries in the anterior intraparietal sulcus and ventral premotor cortex during grasp planning,” Neuroimage 57(2), 502–512 (2011). [CrossRef]
57. B. Christie, “Doctors revise declaration of Helsinki,” BMJ 321(7266), 913 (2000). [CrossRef]
58. P. Pinti, I. Tachtsidis, A. Hamilton, J. Hirsch, C. Aichelburg, S. Gilbert, and P. W. Burgess, “The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience,” Ann. N.Y. Acad. Sci. 1464(1), 5–29 (2020). [CrossRef]
59. D. T. Delpy, M. Cope, P. van der Zee, S. Arridge, S. Wray, and J. S. Wyatt, “Estimation of optical pathlength through tissue from direct time of flight measurement,” Phys. Med. Biol. 33(12), 1433–1442 (1988). [CrossRef]
60. A. Zafar and K.S. Hong, “Neuronal activation detection using vector phase analysis with dual threshold circles: A functional near-infrared spectroscopy study,” Int. J. Neur. Syst. 28(10), 1850031 (2018). [CrossRef]
61. C. Ming, S. Haro, A. M. Simmons, and J. A. Simmons, “A comprehensive computational model of animal biosonar signal processing,” PLoS Comput. Biol. 17(2), e1008677 (2021). [CrossRef]
62. H. H. Kim, J. H. Jo, Z. Teng, and D. J. Kang, “Text detection with deep neural network system based on overlapped labels and a hierarchical segmentation of feature maps,” Int. J. Control Autom. Syst. 17(6), 1599–1610 (2019). [CrossRef]
63. R. Gunaratne, J. Goncalves, I. Monteath, R. Sheh, M. Kapfer, R. Chipper, B. Robertson, R. Khan, D. Fick, and C. N. Ironside, “Wavelength weightings in machine learning for ovine joint tissue differentiation using diffuse reflectance spectroscopy (DRS),” Biomed. Opt. Express 11(9), 5122–5131 (2020). [CrossRef]
64. S. H. Kim and H. L. Choi, “Convolutional neural network for monocular vision-based multi-target tracking,” Int. J. Control Autom. Syst. 17(9), 2284–2296 (2019). [CrossRef]
65. F. M. Noori, N. Naseer, N. K. Qureshi, H. Nazeer, and R. A. Khan, “Optimal feature selection from fNIRS signals using genetic algorithms for BCI,” Neurosci. Lett. 647, 61–66 (2017). [CrossRef]
66. E. Hernandez-Martin, F. Marcano, C. Modrono, N. Janssen, and J. L. Gonzalez-Mora, “Diffuse optical tomography to measure functional changes during motor tasks: a motor imagery study,” Biomed. Opt. Express 11(11), 6049–6067 (2020). [CrossRef]
67. M. Forano and D. W. Franklin, “Timescales of motor memory formation in dual-adaptation,” PLoS Comput. Biol. 16, e1008373 (2020). [CrossRef]
68. H. Li, A. Gong, L. Zhao, W. Zhang, F. Wang, and Y. Fu, “Decoding of walking imagery and idle state using sparse representation based on fNIRS,” Comput. Intell. Neurosci. 2021, 6614112 (2021). [CrossRef]
69. K.-S. Hong, U. Ghafoor, and M. J. Khan, “Brain–machine interfaces using functional near-infrared spectroscopy: a review,” Artif. Life Robot. 25(2), 204–218 (2020). [CrossRef]
70. A. Zafar and K.-S. Hong, “Detection and classification of three-class initial dips from prefrontal cortex,” Biomed. Opt. Express 8(1), 367–383 (2017). [CrossRef]
71. S. D. Power, A. Kushki, and T. Chau, “Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: Toward a three-state NIRS-BCI,” BMC Res. Notes 5(1), 141 (2012). [CrossRef]
72. H. Nazeer, N. Naseer, R. A. Khan, F. M. Noori, N. K. Qureshi, U. S. Khan, and M. J. Khan, “Enhancing classification accuracy of fNIRS-BCI using features acquired from vector-based phase analysis,” J. Neural Eng. 17(5), 056025 (2020). [CrossRef]
73. N. Naseer, F. M. Noori, N. K. Qureshi, and K.-S. Hong, “Determining optimal feature-combination for LDA classification of functional near-infrared spectroscopy signals in brain-computer interface application,” Front. Hum. Neurosci. 10, 237 (2016). [CrossRef]
74. N. Naseer, N. K. Qureshi, F. M. Noori, and K.-S. Hong, “Analysis of different classification techniques for two-class functional near-infrared spectroscopy-based brain-computer interface,” Comput. Intell. Neurosci. 2016, 1–11 (2016). [CrossRef]
75. L. C. Schudlo and T. Chau, “Dynamic topographical pattern classification of multichannel prefrontal NIRS signals: II. Online differentiation of mental arithmetic and rest,” J. Neural Eng. 11(1), 016003 (2014). [CrossRef]
76. S. D. Power, A. Kushki, and T. Chau, “Towards a system-paced near-infrared spectroscopy brain–computer interface: differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state,” J. Neural Eng. 8(6), 066004 (2011). [CrossRef]
77. H. Santosa, M. J. Hong, and K.-S. Hong, “Lateralization of music processing with noises in the auditory cortex: an fNIRS study,” Front. Behav. Neurosci. 8, 418 (2014). [CrossRef]
78. L. C. Schudlo and T. Chau, “Towards a ternary NIRS-BCI: single-trial classification of verbal fluency task, Stroop task and unconstrained rest,” J. Neural Eng. 12(6), 066008 (2015). [CrossRef]
79. T. Gateau, G. Durantin, F. Lancelot, S. Scannella, and F. Dehais, “Real-time state estimation in a flight simulator using fNIRS,” PloS One 10(3), e0121279 (2015). [CrossRef]
80. E. Yacoub and X. Hu, “Detection of the early negative response in fMRI at 1.5 Tesla,” Magn. Reson. Med. 41(6), 1088–1092 (1999). [CrossRef]
81. T. Ernst and J. Hennig, “Observation of a fast response in functional MR,” Magn. Reson. Med. 32(1), 146–149 (1994). [CrossRef]
82. M. Watanabe, A. Bartels, J. H. Macke, Y. Murayama, and N. K. Logothetis, “Temporal jitter of the BOLD signal reveals a reliable initial dip and improved spatial resolution,” Curr. Biol. 23(21), 2146–2150 (2013). [CrossRef]
83. Y. Behzadi and T. T. Liu, “Caffeine reduces the initial dip in the visual BOLD response at 3 T,” Neuroimage 32(1), 9–15 (2006). [CrossRef]
84. R. B. Buxton, “The elusive initial dip,” Neuroimage 13(6), 953–958 (2001). [CrossRef]
85. X. S. Hu, K.-S. Hong, and S. S. Ge, “Reduction of trial-to-trial variability in functional near-infrared spectroscopy signals by accounting for resting-state functional connectivity,” J. Biomed. Opt. 18(1), 017003 (2013). [CrossRef]
86. E. Yacoub and X. Hu, “Detection of the early decrease in fMRI signal in the motor area,” Magn. Reson. Med. 45(2), 184–190 (2001). [CrossRef]
87. K.-S. Hong and P. T. Pham, “Control of axially moving systems: a review,” Int. J. Control Autom. Syst. 17(12), 2983–3008 (2019). [CrossRef]
88. A. K. Pamosoaji, M. Piao, and K.-S. Hong, “PSO-based minimum- time motion planning for multiple vehicles under acceleration and velocity limitations,” Int. J. Control Autom. Syst. 17(10), 2610–2623 (2019). [CrossRef]
89. W. J. Du, N. N. Rao, C. L. Dong, Y. C. Wang, D. C. Hu, L. L. Zhu, B. Zeng, and T. Gan, “Automatic classification of esophageal disease in gastroscopic images using an efficient channel attention deep dense convolutional neural network,” Biomed. Opt. Express 12(6), 3066–3081 (2021). [CrossRef]
90. H. Kang, S. Yang, J. Haung, and J. Oh, “Time series prediction of wastewater flow rate by bidirectional LSTM deep learning,” Int. J. Control Autom. Syst. 18(12), 3023–3030 (2020). [CrossRef]
91. Y. M. Luo, Q. Xu, R. B. Jin, M. Wu, and L. B. Liu, “Automatic detection of retinopathy with optical coherence tomography images via a semi-supervised deep learning method,” Biomed. Opt. Express 12(5), 2684–2702 (2021). [CrossRef]
