Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Deep-learning informed Kalman filtering for priori-free and real-time hemodynamics extraction in functional near-infrared spectroscopy

Open Access Open Access

Abstract

Separation of the physiological interferences and the neural hemodynamics has been a vitally important task in the realistic implementation of functional near-infrared spectroscopy (fNIRS). Although many efforts have been devoted, the established solutions to this issue additionally rely on priori information on the interferences and activation responses, such as time-frequency characteristics and spatial patterns, etc., also hindering the realization of real-time. To tackle the adversity, we herein propose a novel priori-free scheme for real-time physiological interference suppression. This method combines the robustness of deep-leaning-based interference characterization and adaptivity of Kalman filtering: a long short-term memory (LSTM) network is trained with the time-courses of the absorption perturbation baseline for interferences profiling, and successively, a Kalman filtering process is applied with reference to the noise prediction for real-time activation extraction. The proposed method is validated using both simulated dynamic data and in-vivo experiments, showing the comprehensively improved performance and promisingly appended superiority achieved in the purely data-driven way.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Functional near-infrared spectroscopy (fNIRS) is a fast-developing non-invasive neuroimaging technique, which is able to offer excellent temporal resolution, reasonable spatial resolution and quantitative hemodynamic information about both oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) [13]. As a complementary or alternative tool to functional magnetic response imaging (fMRI) and electroencephalogram (EEG), fNIRS is portable, low cost, easy to handle, and allows long-time continuous measurement with fewer constriction on subjects.

However, the measured fNIRS signals are normally contaminated by changes in the systemic physiology, i.e., cardiac pulsation, breathing cycle and low frequency oscillation. To improve reconstruction performance, many efforts have been made to suppress these interferences. As shown in Table 1, each method has distinct strengths and weaknesses in terms of efficacy and robustness. Low-pass/band-pass filtering combined with cycle averaging is the most commonly used method to suppress physiological interferences, but it cannot deal with overlapping frequency band and requires lengthy measurement time [35]. Blind source separation methods, i.e., principal component analysis (PCA) and independent component analysis (ICA), have also been implemented to address this problem, yet these methods may lead to error due to subjective judgment [6,7]. Besides, short channel calibration methods have also been developed that use measurements at short (about 5-15 mm) source-detector separation (SDS) as a reference to remove the physiological interferences [8]. Nevertheless, due to heterogeneous distributions of the interferences, it is difficult to optimally arrange the short SDS for all normal SDS channels. This adversity creates a barrier to directly implement these methods, and may require additional setting cost. Wavelet-based filtering methods have also been used to suppress the noises in fNIRS application, which need to select the reasonable configurations in implementation [9,10]. Latterly, our group proposes a tandem inversion filtering and long short-term memory (LSTM) classification scheme, which incorporates the semi-3D reconstruction aided with overlapping array and deep learning network to efficiently suppress the physiological interferences and physical noises in functional diffuse optical tomography (fDOT) [11]. Yet, many commercial fNIRS systems only provide normal SDS channels, which cannot implement this tandem scheme directly.

Tables Icon

Table 1. Mathematical and technical methods to suppress physiological interferences

In addition, Kalman-based state-space modeling methods have been proved to achieve real-time performance in fNIRS-based neuroimaging, which model the fNIRS signal as a linear combination of physiological interferences and hemodynamic response [1215]. However, these methods still have a practical shortcoming: the state-space estimation lacks reasonable physiological regressors. For example, physiological regressor is normally approximated to the expanded Fourier series or defined using auxiliary measurements, which unable to deal with the time-lag of the physiological interferences among different brain regions or need additional hardware [1618]. Actually, it is hard to propose a universal solution to physiological interferences suppression, and therefore, techniques that solve one part of the problem may eventually be incorporated into a global solution.

Nowadays, Deep-Learning (DL) methods have already shown great success in the regime of biomedical signal filtering and imaging [11,1921]. Due to the efficient feature extraction ability of the DL-based methods, it is reasonable to assume that a DL-based scheme would provide a possible avenue to model the physiological regressor efficiently and adaptively. We herein propose a DL informed Kalman filtering scheme, referred to as Kalman-LSTM, to seek a cost-effective way of suppressing the interferences in fNIRS. The approach combines advantages of a LSTM network in accurate characterization of the physiological interferences and a Kalman filter in adaptive tracing of the activation responses and ultimately aims at a prior-free and real-time implementation of fNIRS. In the present study, we conduct simulative investigations and in-vivo experiments to validate the performance of this method. The results demonstrate the sound ability of the proposed method to suppress the physiological interferences and to enhance real-time performance of fNIRS.

2. Methods

In fNIRS, the goal is to model how variations in measurements on the surface correspond to transient perturbations in optical properties within the volume under consideration. The commonly used method is the modified Beer-Lambert law (MBLL), which models the optical density changes as a linear combination of HbO and HbR [22,23]. Further, an absorption perturbation at a given channel is expressed as a linear combination from the contributions of each hemoglobin species and is given by

$$\left\{ \begin{array}{l} \delta O{D_\lambda }(k) = L \cdot DP{F_\lambda } \cdot \delta \mu_a^\lambda (k)\\ \delta {\mu_\lambda }(k) = \varepsilon_\lambda^{\textrm{HbO}} \cdot \delta {C_{\textrm{HbO}}}(k) + \varepsilon_\lambda^{\textrm{HbR}} \cdot \delta {C_{\textrm{HbR}}}(k) \end{array} \right.$$
where k is the time index; $\delta {\mu _\lambda }$ denotes the absorption perturbation at the wavelength $\lambda$; $\varepsilon _\lambda ^{\textrm{HbO}}$ and $\varepsilon _\lambda ^{\textrm{HbR}}$ are the extinction coefficients of the HbO and HbR, respectively; L is the SDS of the channel; $DP{F_\lambda }$ is the differential length factor; $\delta {C_{\textrm{HbO}}}$ and $\delta {C_{\textrm{HbR}}}$ are the concentration perturbation of HbO and HbR, respectively.

The changes induced by the physiological interferences will seriously disturb the activation reconstruction, and therefore, it is significant to strip this kind of change from total changes for obtaining high quality reconstruction. In practice, LSTM network is able to model the temporal correlations, and in light of this, as illustrated in Fig. 1, the workflow of the Kalman-LSTM method is as follows: firstly, the absorption perturbation during the baseline is reconstructed, and then this originally reconstructed perturbation is used as the training dataset to create the LSTM prediction network; secondly, in each time-point during stimulation paradigm, the physiological interferences are estimated using the created prediction network (the parameters in the network stops optimization during stimulation paradigm), which then incorporated in the Kalman filtering framework to separate the effects of the interferences from the local activation in a data-driven and real-time way.

 figure: Fig. 1.

Fig. 1. Workflow of the Kalman-LSTM method.

Download Full Size | PDF

Traditional analysis scheme in fNIRS is based on an assumption of the linear addition of hemodynamic changes: the hemodynamic changes may be expressed as a system of linear equation given in matrix form, for example, as for a specific channel at time $k$

$$y(k) = {\textbf H}(k){\mathbf \beta }(k) + e(k)$$
where $y(k)$ is the perturbed hemoglobin concentration; ${\textbf H}(k)$ is known as the design matrix including a set of explanatory variables to model the hemodynamic response, i.e., ${\textbf H}(k) = [h(k)\textrm{ }O(k)]$, with $h(k)\textrm{ }$ denoting a task-convolved canonical hemodynamic response function (cHRF), and $O(k)$ denoting the interference function; ${\mathbf \beta } = {[{\beta _h}(k)\textrm{ }{\beta _O}(k)]^T}$ is the set of level for the specific function, with ${\beta _h}(k)$ denoting the perturbed hemoglobin level, and $\textrm{ }{\beta _O}(k)$ denoting the interference level; $e(k)$ is the measurement error, which is often assumed to be independent zero-mean Gaussian distributed random variables.

In practice, cHRF is varied, and the mismatch between the empirical and real distribution of cHRF will result in estimation performance degradation. To tackle this, we model the reconstructed perturbation as a new system of linear equation, for example, as for a specific channel, the reconstructed perturbation, $\delta \mu _a^\textrm{R}(k)$, is modeled with ${\textbf H}(k) = [1\textrm{ }O(k)]$ by multiplying a vector ${\textbf X}(k) = {[\delta {\mu _a}(k)\textrm{ }{\beta _O}(k)]^T}$ plus an error term $e(k)$, i.e.

$$\delta \mu _a^\textrm{R}(k) = {\textbf H}(k){\textbf X}(k) + e(k)$$
where $\delta {\mu _a}(k)$ denotes the task-related perturbation. In this new equation, $O(k)$ is the output of the created LSTM-based prediction network, and then, the Kalman filter is used to estimate and update the model coefficients [24]. The state vector can be described as below
$${\textbf X}(k) = {\textbf {AX}}(k - 1) + {\textbf w}(k)$$
where ${\textbf w}(k)$ denotes the process noise distribution; ${\textbf A}$ is set as identity matrix based on the slowly-varying perturbation. Then, the filter performs state estimation by the iterative process
$$\left\{ \begin{array}{l} {{\hat{{\textbf X}}}^ - }(k) = {{\textbf{A}\hat{\textbf{X}}}}(k - 1)\\ {{\textbf P}^ - }(k) = {\textbf {AP}}(k - 1){{\textbf A}^T} + {\textbf Q}\\ {\textbf G}(k) = {{\textbf P}^ - }(k){\textbf H}(k){[{\textbf H}(k){{\textbf P}^ - }(k){{\textbf H}^T}(k) + R]^{ - 1}}\\ \hat{{\textbf X}}(k) = {{\hat{{\textbf X}}}^ - }(k) + {\textbf G}(k)[\delta \mu_a^\textrm{R}(k) - {\textbf H}(k){{\hat{{\textbf X}}}^ - }(k)]\\ {\textbf P}(k) = [{\textbf E} - {\textbf G}(k){\textbf H}(k)]{{\textbf P}^ - }(k) \end{array} \right.$$
where ${\textbf G}(k)$ is the Kalman gain; ${\textbf E}$ is the identity matrix, which is same as ${\textbf A}$;${\textbf P}(k)$ is the updated error covariance matrix; the superscript notation $( - )$ refers to the intermediate state and covariance predictions provided by the state update model, which are then modified by the observed data to produce the next state value. In this work, $\hat{{\textbf X}}(0)$ and ${\textbf P}(0)$ are initialized to zero vectors, and the prior estimates of the process and observation noise covariance are empirically determined, i.e., ${\textbf Q} = [0.1\textrm{ 0}\textrm{.1}]$ and $R = 0.1$. The measurements are processed through this iterative process, and the task-related perturbation is estimated in each time-point. The LSTM network is used to provide information for the modified version of the general linear model. Specifically, the perturbation during baseline is used to train the LSTM network, and the output of the LSTM network is incorporated in the Kalman filtering framework. To enrich the input information, as shown in Fig. 2, time overlapping strategy is implemented to form N-order input vectors (N = 3 in this study) in each time-point.

 figure: Fig. 2.

Fig. 2. Time overlapping strategy: ${\textbf I}(k)$ denotes the input vectors at the specific time-point during the network creation, where n is the total number of the time-points during baseline; $O(k)$ denotes the output of the LSTM network.

Download Full Size | PDF

3. Experiments

The simulative investigations and in-vivo experiments are conducted to assess the effectiveness of the proposed method in recovering the task-related absorption perturbation.

3.1 Simulative investigations

Firstly, in the simulative investigations, three assessment metrics, namely the contrast-to-noise-ratio (CNR), the relative root mean square error (rRMSE), and the dimension fidelity (DF), are defined below for quantitative assessment of the reconstructed maps

$$\begin{array}{l} CNR(k) = \frac{{mean[\delta {\mathbf \mu }_a^\textrm{T}(k)] - mean[\delta {\mathbf \mu }_a^\textrm{B}(k)]}}{{std[\delta {\mathbf \mu }_a^\textrm{B}(k)]}}\\ rRMSE(k) = \sqrt {\frac{{\textrm{||}\delta {{\mathbf \mu }_a}(k) - \delta {\mathbf \mu }_a^\textrm{G}(k)\textrm{||}_2^2}}{{\textrm{||}\delta {\mathbf \mu }_a^\textrm{G}(k)\textrm{||}_2^2}}} \\ DF(k) = \frac{{FWHM(k)}}{D},\textrm{ } \end{array}$$
where $\delta {\mathbf \mu }_a^\textrm{T}(k)$ and $\delta {\mathbf \mu }_a^\textrm{B}(k)$ denote the perturbed absorption vectors covering the activated and background region at time k, respectively; $mean$ and $std$ are the functions to calculate the average and standard deviation, respectively; $\delta {\mathbf \mu }_a^\textrm{G}(k)$ is the ground-truth perturbed absorption vectors; $FWHM(k)$ is the full-width-at-half-maximum in the specific profile of the reconstructed map, and D is the diameter of the activated region in the specific profile.

3.1.1 Setting of the simulative investigations

Herein, without loss of methodological generality, a four-layer model of cuboid shape is adopted to emulate a simplified brain geometry with the scalp-, skull-, cerebrospinal fluid- (CSF-) and cortex-layers, as illustrated in Fig. 3. It is noted that the layered planar model is practically effective due to the regionally mild curvature of the head surface, and can also be readily registered to the realistic head model, just by shifting the z-coordinates of the mesh nodes accordingly. As for the source-detector configuration, the commonly used lattice array is adopted to provide sparse multi-channel measurements with same SDS [25]. A cylindrical activated region locates in the cortex-layer with a diameter of 15 mm, a height of 12 mm and its centroid at the specific position (X = 80 mm, Y = 65 mm, Z = 32 mm).

 figure: Fig. 3.

Fig. 3. 3-Dimensional and longitudinal sectional view of the brain-emulating model.

Download Full Size | PDF

As listed in Table 2, the background absorption coefficient ($\mu _\textrm{a}^\textrm{B}$) and reduced scattering coefficient ($\mu ^{\prime}_s{^\textrm{B}}$) are set within the range of ones published in the literature [26]. Considering the low-scattering property of the CSF-layer where the diffusion equation is not applicable, Monte Carlo (MC) method (108 photon packets) is used to generate the simulated measurements, and then the perturbation is reconstructed using MBLL [27]. As for the LSTM-based prediction network, a four-layer architecture is created: input layer, hidden layer (LSTM layer), fully-connected layer and regression layer. Since LSTM network is sensitive to data scale, z-score normalization is adopted to pre-process the originally reconstructed perturbation. The adaptive moment (Adam) estimation can adaptively match parameters and learning rate with less resources, so that sparse features can be better updated. The network performs 1000 rounds of training using Adam algorithm to complete the gradient optimization, the number of hidden layer neurons is set to 250, the initial learning rate is set to 0.005, and after 125 rounds of training, the learning rate is reduced by 0.2 times.

Tables Icon

Table 2. Optical coefficient of the brain-emulating model

3.1.2 Hemodynamic response function

As shown in Fig. 4(a), the measurements were simulated for a period of 120 s at a sampling rate of 2 Hz, where a 40 s baseline was added the beginning and end of the paradigm. Task-related perturbation is originated from hemodynamic response, i.e., HbO and HbR, which depends on the stimulus function and cHRF. To simulate variety of the subject-dependent cHRF, four parameters in the cHRF model are supposed as free parameters (delay of response ${\alpha _1}$, delay of undershoot ${\alpha _2}$, dispersion of response ${\beta _1}$ and dispersion of undershoot ${\beta _2}$)

$$\left\{ \begin{array}{l} \delta {C_{\textrm{Hbo}}}(k) = {\alpha_0}[cHR{F_{\textrm{Hbo}}}(k) \otimes \sum\nolimits_{\textrm{n ={-} }\infty }^\infty {u(k)]} \\ cHR{F_{\textrm{HbO}}}(k) = \frac{{{k^{{\alpha_1} - 1}}\beta_1^{{\alpha_1}}{e^{ - {\beta_1}k}}}}{{\Gamma ({\alpha_1})}} - \frac{{{k^{{\alpha_2} - 1}}\beta_2^{{\alpha_2}}{e^{ - {\beta_2}k}}}}{{6\Gamma ({\alpha_2})}} \end{array} \right.$$
where ${\alpha _0}$ defines the amplitude of the hemoglobin perturbation; $u(k)$ is the stimulus function; $\Gamma $ represents the Gamma distribution; ${\otimes}$ denotes the convolution function. To compute the perturbed hemoglobin concentrations, cHRF is convoluted with a boxcar stimulation, where the parameters set as ${\alpha _0} = 9.2$; ${\alpha _1} = 5.5$; ${\alpha _2} = 6.5$; ${\beta _1} = 0.8$; ${\beta _2} = 0.1$; $\sigma ={-} 0.22$[11,28].

 figure: Fig. 4.

Fig. 4. Simulated hemodynamic response: (a) stimulation paradigm; (b) time courses of the HbO- and HbR- concentration perturbation induced by task and the corresponding absorption perturbation; (c) time courses of the HbO- and HbR- concentration perturbation induced by the physiological interferences (when $\textrm{A}_r^G({\textbf r}) = 1$ and $\textrm{A}_h^G({\textbf r}) = 1$) and the corresponding absorption perturbation.

Download Full Size | PDF

With $\delta {\textrm{C}_{\textrm{HbO}}}({t_i})$ and $\delta {\textrm{C}_{\textrm{HbR}}}({t_i})$ known, time courses of the perturbed absorption can be calculated based on the extinction coefficients of HbO and HbR:$\varepsilon _{785nm}^{\textrm{HbO}} = 0.732\textrm{m}{\textrm{M}^{ - 1}} \cdot \textrm{c}{\textrm{m}^{ - 1}}$; $\varepsilon _{830nm}^{\textrm{HbO}} = 0.97\textrm{m}{\textrm{M}^{ - 1}} \cdot \textrm{c}{\textrm{m}^{ - 1}}$; $\varepsilon _{785nm}^{\textrm{HbR}} = 0.974\textrm{m}{\textrm{M}^{ - 1}} \cdot \textrm{c}{\textrm{m}^{ - 1}}$;$\varepsilon _{830nm}^{\textrm{HbR}} = 0.693\textrm{m}{\textrm{M}^{ - 1}} \cdot \textrm{c}{\textrm{m}^{ - 1}}$. With the setting, the time courses of the perturbed hemoglobin concentration and the corresponding absorption in the activation region are shown in Fig. 4(b). As for the perturbation induced by physiological interferences, a multi-frequency sinusoidal function is defined to represent the effects from the respiration and heartbeat [29]. In addition, considering the possible inhomogeneity, the interference amplitudes with different frequencies are assumed to be Gaussian distributed, and have different center points, i.e.,

$$\left\{ \begin{array}{l} \delta C_{\textrm{HbO}}^\textrm{I}({\textbf r},k) = \textrm{A}_r^G({\textbf r})\delta C_{\textrm{HbO}}^r\sin (2\pi {f_r}k) + \textrm{A}_h^G({\textbf r})\delta C_{\textrm{HbO}}^h\sin (2\pi {f_h}k)\\ \delta C_{\textrm{HbR}}^\textrm{I}({\textbf r},k) = \textrm{A}_r^G({\textbf r})\delta C_{\textrm{HbR}}^r\sin (2\pi {f_r}k) + \textrm{A}_h^G({\textbf r})\delta C_{\textrm{HbR}}^h\sin (2\pi {f_h}k) \end{array} \right.$$
where ${\textbf r} = [{\mathbf \rho },z] \in {\Omega ^\textrm{B}}$ with $z \in [\textrm{45 mm, 50 mm}]$; $\delta C_{\textrm{HbO}}^r = 2uM$ and $\delta C_{\textrm{HbO}}^h = 1.1uM$ denote the perturbed HbO induced by the respiration and heartbeat, respectively, while $\delta C_{\textrm{HbR}}^r = 0.13uM$ and $\delta C_{\textrm{HbR}}^h = 0.074uM$ denote the perturbed HbR induced by the interferences; $\textrm{A}_r^G({\textbf r})$ and $\textrm{A}_h^G({\textbf r})$ denote the Gaussian amplitude distributions of respiration and heartbeat, respectively ; ${f_r}$ and ${f_h}$ are set as 1.1 Hz and 0.25 Hz, respectively. With the setting, the time courses of the perturbed hemoglobin concentration induced by the interferences and the corresponding perturbed absorption are shown in Fig. 4(c). In addition, to conform to real scenarios, a Gaussian-type noise is added to each channel, with its level proportional to the square root of the underlying intensity, achieving a dataset with the minimum signal to noise ratio of 20 dB.

3.1.3 Results

To cope with the regional average in MBLL-based imaging, the reconstruction is processed by maximum normalization to evaluate the reconstruction performance. The reconstructed perturbation maps using different methods at 785nm are illustrated in Fig. 5(a). It is obvious that the interferences are effectively suppressed using the low-pass (LP) filtering method with cut-off frequency at 0.1 Hz and the Kalman-LSTM method, but there are non-negligible artefacts using the LP method with cut-off frequency at 0.2 Hz. As shown in Fig. 5(b) and (c), this conclusion can be further proved based on the time courses of the averaged perturbation in the activation region and the quantitation comparison. It should be emphasized that space-localized and less blurry reconstruction can be obtained through Kalman-LSTM, and this real-time ability provides the possibility for online brain-computer interface (BCI). Another point to note is that Kalman-LSTM can achieve effective suppression of physiological interferences after a few time-points at the paradigm beginning.

 figure: Fig. 5.

Fig. 5. Simulative validations: (a) absorption perturbation maps at the selected time-points; (b) time courses of the averaged absorption perturbation in the activation region; (c) quantitative evaluations of the reconstructed perturbation maps.

Download Full Size | PDF

3.2 Method effectiveness under cHRF variety

In this section, we will compare the effectiveness of the method under cHRF variety during the sequential stimuli trials. As illustrated in Fig. 6(a), a new paradigm is adopted, and the time courses of HbO and HbR perturbation and the corresponding reconstructed averaged perturbation in the activation region at 785 nm are show in Fig. 6(b), where the free parameters of the adopted cHRF during different task are set as Table 3.

 figure: Fig. 6.

Fig. 6. Simulative validations under cHRF variety in sequential trials: (a) stimulation paradigm; (b) time courses of the HbO- and HbR- concentration perturbation and the corresponding reconstructed absorption perturbation in the activation region.

Download Full Size | PDF

Tables Icon

Table 3. Free parameters in the cHRF variety

The results prove that the proposed method can still estimate the perturbation accurately under cHRF variety during sequential trials. It should be noted that the performance of the LSTM model will inevitably go down over time due to error accumulation, which manifests itself as larger fluctuation at the end of paradigm. In fact, a similar situation occurs during single-trial experiment with higher sampling rate for the same reason. Therefore, an important work in the future is to refine the prediction model for mitigating the error accumulation.

3.3 Method effectiveness under intense interferences

In the simulative investigation, the maximum amplitude of physiological interferences is set according to the literature, and in practice, this amplitude may increase with different subjects and stimulation tasks. Therefore, it is necessary to evaluate the effectiveness of the method under intense interferences. For example, the hemoglobin perturbation induced by the physiological interferences is increased by five times, i.e., $\delta C_{\textrm{HbO}}^r = 10uM$, $\delta C_{\textrm{HbO}}^h = 5.5uM$, $\delta C_{\textrm{HbR}}^r = 0.65uM$ and $\delta C_{\textrm{HbR}}^h = 0.37uM$. As illustrated in Fig. 7, the proposed method can still cope with this complicated situation. However, the impact of error accumulation has become more noticeable under the intense interferences condition.

 figure: Fig. 7.

Fig. 7. Simulative validations under intense interferences: (a) absorption perturbation maps at the selected time-points; (b) the corresponding time courses of the averaged absorption perturbation in the activation region; (c) quantitative evaluations of the reconstructed perturbation maps.

Download Full Size | PDF

3.4 In-vivo experiment

A measure of fNIRS research success is whether the task-related perturbation can be recognized during the stimuli trial. For this reason, an in-vivo experiment is further conducted to verify the performance of the proposed method.

3.4.1 Participants and experimental procedure

Twenty students (mean age = 24, 10 male and 10 female) that are recruited from Tianjin University participate in this study, and no one has motor or other neurological diseases. Before the experiment, written informed consent is obtained from the students and all of them are asked to relax mind and remain motionless. As illustrated in Fig. 8(a), the experimental paradigm consists of one trial, during which participants either perform a successive mental subtraction task (e.g., 488−5 = 483, 483−5 = 478, 478−5 = 473, etc.) or rest [30,31]. The measurements are acquired at a sampling rate of 4 Hz using our lab-made fNIRS system, and a lattice array with four sources (785 nm and 830 nm, respectively) and four photomultiplier detectors are secured to the participant’s forehead using a custom-made resin headband (see Fig. 8(b)) [32,33]. It should be pointed out that, the measurements with large motions are discarded, and the experiment will be repeated to obtain new high-quality data.

 figure: Fig. 8.

Fig. 8. In-vivo experiment configuration: (a) stimulation paradigm; (b) picture of the measurement.

Download Full Size | PDF

3.4.2 Results

As a benchmark, the measurements are simultaneously processed using the LP method with cut-off frequency of 0.1 Hz [3]. Due to the limited experimental condition, multi-modality imaging (i.e., fMRI-fNIRS) is infeasible in this work, so the ground-truth of the perturbation is not available. Therefore, we evaluate the performance by computing the classification accuracy. For reliable classification, a commonly used metric, Fisher Score, is used to select the optimal feature, where a higher Fisher Score guarantees larger separability between task- and rest-state [34]. In addition, to ensure the robustness of the classification result, 10-fold cross-validation is used to estimate the average accuracy. Specifically, the computed Fisher Score of each channel are arranged in descending order, and then the specific feature type, i.e., mean, skewness, and kurtosis, are extracted from the channel-wise time sequence with highest Fisher Score for achieving classification.

Linear discriminant analysis (LDA) and support vector machine (SVM) with linear kernel function are used as the classifiers [35], and the classification results using the specific feature from the channel with highest Fisher Score are shown in Fig. 9: SVM classification method has better accuracy than LDA in most cases, and the accuracy of the Kalman-LSTM method is equivalent to or superior than the LP method with cut-off frequency of 0.1 Hz. Further, we analyzed whether there is a significant difference between the classification accuracy using different methods, and the p-values for each feature are shown in Table 4. With an acceptable accuracy, kurtosis feature from HbR perturbation using the Kalman-LSTM method is significantly different with the LP method with cut-off frequency of 0.1 Hz (P < 0.01).

 figure: Fig. 9.

Fig. 9. A comparison of classification accuracy using different feature types from the channel with highest Fisher Score: M-LK, S-LK and K-LK denote the feature of mean, skewness and kurtosis using the Kalman-LSTM method, respectively; M-LP, S-LP and K-LP denote the feature of mean, skewness and kurtosis feature using the LP method with cut-off frequency of 0.1 Hz, respectively.

Download Full Size | PDF

Tables Icon

Table 4. Significance analysis of the classification accuracy

In practice, feature number, kernel function in SVM and cut-off frequency in the LP method are important factors in classification performance. Specifically, unreasonable cut-off frequency may incompletely denoise the interferences or destroy the activation, while unreasonable feature number and kernel function may induce complicated model or decreased accuracy. For this reason, above mentioned configurations are further performed to compare the classification accuracy. In view of relatively excellent accuracy using mean and kurtosis feature, the accuracy under the above mentioned configurations using SVM is shown in Fig. 10: the number of mean feature gives great effects on the classification accuracy, and the reason for this may be that the baseline information masks fluctuations evoked by the stimulation; kernel function has little effect on the result; the feature of kurtosis using the Kalman-LSTM method has a better classification accuracy without subjective configuration. Further, the statistical analysis is conducted, and the results show a significant difference between Kalman-LSTM and LP method with cut-off frequency of 0.2 Hz. In general, it is difficult to configure the cut-off frequency in the LP method, while the Kalman-LSTM method is priori-free, and provides a potential way to achieve online BCI.

 figure: Fig. 10.

Fig. 10. Classification accuracy using feature of mean (upper) and kurtosis (down) with linear (left), gaussian (middle) and polynomial (right) kernel function: LK-, LP1- and LP2-HbO/HbR denote the accuracy computed by HbO/HbR concentration perturbation using the Kalman-LSTM, LP (0.1 Hz) and LP (0.2 Hz), respectively.

Download Full Size | PDF

 figure: Fig. 11.

Fig. 11. Simulative validations using different Kalman-based method: (a) absorption perturbation maps at the selected time-points; (b) the corresponding time courses of the averaged absorption perturbation in the activation region; (c) quantitative evaluations of the reconstructed perturbation maps.

Download Full Size | PDF

4. Discussion and conclusion

In addition, there is another issue to note: whether it is possible to separate the hemodynamic response and physiological interferences by the Kalman filtering method without LSTM network, i.e., ${\textbf H}(k) = {\left[ \begin{array}{l} 1\\ 1 \end{array} \right]^\textrm{T}}$ or ${\textbf H}(k) = {\left[ \begin{array}{c} 1\\ \sin ( 2\pi {f_h}k) + \sin ( 2\pi {f_r}k) \end{array} \right]^\textrm{T}}$ in the Eq. (3), referred to as Kalman and Kalman-Sine method, respectively. To assess this, the mentioned configurations are performed in the measurements from the simulative investigation, and the results are illustrated in Fig. 11. Due to inadequate information of the interference distribution, the Kalman and Kalman-Sine method cannot accurately estimate the task-related perturbation.

Actually, the distribution of physiological interferences is complicated, and therefore, more samples should be acquired to further refine the model and improve its predictive power. It also should be pointed out that although this method is applicable to fDOT, it may not guarantee real-time performance due to the complexity of fDOT calculation. One possible solution is to develop fast fDOT strategy, which is also a direction of future research.

In practice, the sensitivity of fNIRS signals to the interferences usually hamper the performance of the activation reconstruction. To tackle this, a priori-free Kalman-LSTM method has been proposed in this work to suppress the physiological interferences and estimate task-related perturbation. The simulative investigations and in-vivo experiments have been implemented to highlight the effectiveness of the proposed method. The proposed method provides a cost-effective and real-time solution to the interference suppression that can be applied to improve the reconstruction performance of the task-related perturbation in fNIRS-based neuroimaging. More importantly, its real-time capacity provides a practically efficient route to online BCI.

Funding

National Natural Science Foundation of China (61575140, 62075156, 81871393, 81971656); Tianjin Science and Technology Committee (18JCYBJC29400).

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. D. A. Boas, C. E. Elwell, M. Ferrari, and G. Taga, “Twenty years of functional near-infrared spectroscopy: introduction for the special issue,” Neuroimage 85(Pt1), 1–5 (2014). [CrossRef]  

2. A. V. Lühmann, Y.-L. Zheng, A. O. Martinez, S. Kiran, D. C. Somers, A. C. Golomb, L. N. Awad, T. D. Ellis, D. A. Boas, and M. A. Yucel, “Towards neuroscience of the everyday world (NEW) using functional near infrared spectroscopy,” Curr. Opin. Biomed. Eng. 18, 100272 (2021). [CrossRef]  

3. F. Scholkmann, S. Kleiser, A. J. Metz, R. Zimmermann, J. M. Pavia, U. Wolf, and M. Wolf, “A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology,” Neuroimage 85(Pt1), 6–27 (2014). [CrossRef]  

4. N. Naseer and K.-S. Hong, “fNIRS-based brain-computer interfaces: a review,” Front. Hum. Neurosci. 9, 3 (2015). [CrossRef]  

5. M. Hu and T. Shealy, “Application of functional near-infrared spectroscopy to measure engineering decision-making and design cognition: literature review and synthesis of methods,” J. Comput. Civ. Eng. 33(6), 04019034 (2019). [CrossRef]  

6. J. Virtanen, T. Noponen, and P. Merilainen, “Comparison of principal and independent component analysis in removing extracerebral interference from near-infrared spectroscopy signals,” J. Biomed. Opt. 14(5), 054032 (2009). [CrossRef]  

7. A. Aarabi and T. J. Huppert, “Characterization of the relative contributions from systemic physiological noise to whole-brain resting-state functional near-infrared spectroscopy data using single-channel independent component analysis,” Neurophotonics 3(2), 025004 (2016). [CrossRef]  

8. D. Wyser, M. Mattille, M. Wolf, O. Lambercy, F. Scholkmann, and R. Gassert, “Short-channel regression in functional near-infrared spectroscopy is more effective when considering heterogeneous scalp hemodynamics,” Neurophotonics 7(03), 9035011 (2020). [CrossRef]  

9. X. Zhang, J. A. Noah, S. Dravida, and J. Hirsch, “Optimization of wavelet coherence analysis as a measure of neural synchrony during hyperscanning using functional near-infrared spectroscopy,” Neurophotonics 7(01), 1 (2020). [CrossRef]  

10. L. Duan, Z. Zhao, Y. Lin, X. Wu, Y. Luo, and P. Xu, “Wavelet-based method for removing global physiological noise in functional near-infrared spectroscopy,” Biomed. Opt. Express 9(8), 3805–3820 (2018). [CrossRef]  

11. D.-Y. Liu, P.-R. Zhang, Y. Zhang, L. Bai, and F. Gao, “Suppressing physiological interferences and physical noises in functional diffuse optical tomography via tandem inversion filtering and LSTM Classification,” Opt. Express 29(18), 29275–29291 (2021). [CrossRef]  

12. K. S. Hong and H. D. Nguyen, “State-space models of impulse hemodynamic responses over motor, somatosensory, and visual cortices,” Biomed. Opt. Express 5(6), 1778–1798 (2014). [CrossRef]  

13. X.-S. Hu, K.-S. Hong, S. S. Ge, and M. Y. Jeong, “Kalman estimator- and general linear model-based on-line brain activation mapping by near-infrared spectroscopy,” Biomed. Eng. Online 9(1), 82 (2010). [CrossRef]  

14. A. Ortega-Martinez, A. V. Luhmann, P. Farzam, D.-J. Rogers, E. M. Mugler, D. A. Boas, and M. A. Yucel, “Multivariate Kalman filter regression of confounding physiological signals for real-time classification of fNIRS data,” Neurophotonics 9(02), 025003 (2022). [CrossRef]  

15. G. Durantin, S. Scannella, T. Gateau, A. Delorme, and F. Dehais, “Processing functional near infrared spectroscopy signal with a Kalman filter to assess working memory during simulated flight,” Front. Hum. Neurosci. 9, 707 (2016). [CrossRef]  

16. A. V. Luhmann, X.-G. Li, K.-R. Muller, D. A. Boas, and M. A. Yucel, “Improved physiological noise regression in fNIRS: A multimodal extension of the general linear model using temporally embedded Canonical Correlation Analysis,” NeuroImage 208, 116472 (2020). [CrossRef]  

17. L. Minati, I. U. Kress, E. Visani, N. Medford, and H. D. Critchley, “Intra- and extra-cranial effects of transient blood pressure changes on brain near-infrared spectroscopy (NIRS) measurements,” J. Neurosci. Methods 197(2), 283–288 (2011). [CrossRef]  

18. S. Sutoko, Y. L. Chan, A. Obata, H. Sato, and Y. Tong, “Denoising of neuronal signal from mixed systemic low-frequency oscillation using peripheral measurement as noise regressor in near-infrared imaging,” Neurophotonics 6(01), 1 (2019). [CrossRef]  

19. Y.-Y. Gao, H.-Q. Chao, L. Cavuoto, P.-K. Yan, U. Kruger, J. E. Norfleet, B. A. Makled, S. Schwaitzberg, S. De, and X. Intes, “Deep learning-based motion artifact removal in functional near-infrared spectroscopy,” Neurophotonics 9(04), 041406 (2022). [CrossRef]  

20. L.-Y. Xu, Y.-Y. Liu, J. Yu, X.-J. Li, X. Yu, H.-Y. Cheng, and J. Li, “Characterizing autism spectrum disorder by deep learning spontaneous brain activity from functional near-infrared spectroscopy,” J. Neurosci. Meth. 331, 108538 (2020). [CrossRef]  

21. A. S. Heinsfeld, A. R. Franco, R. C. Craddock, A. Buchweitz, and F. Meneguzzi, “Identification of autism spectrum disorder using deep learning and the ABIDE dataset,” Neuroimage: Clinical 17, 16–23 (2018). [CrossRef]  

22. L. Kocsis, P. Herman, and A. Eke, “The modified Beer-Lambert law revisited,” Phys. Med. Biol. 51(5), N91–N98 (2006). [CrossRef]  

23. A. M. Chiarelli, D. perpetuini, C. Filippini, D. Cardone, and A. Merla, “Differential pathlength factor in continuous wave functional near-infrared spectroscopy: reducing hemoglobin's cross talk in high-density recordings,” Neurophotonics 6(03), 1 (2019). [CrossRef]  

24. B.-Y. Wang, T.-T. Pan, Y. Zhang, D.-Y. Liu, J.-Y. Jiang, H.-J. Zhao, and F. Gao, “A Kalman-based tomographic scheme for directly reconstructing activation levels of brain function,” Opt. Express 27(3), 3229–3246 (2019). [CrossRef]  

25. A. Yennu, F. Tian, A. Smith-Osborne, R. J. Gatchel, F. L. Woon, and H. Liu, “Prefrontal responses to Stroop tasks in subjects with post-traumatic stress disorder assessed by functional near infrared spectroscopy,” Sci. Rep. 6(1), 30157 (2016). [CrossRef]  

26. A. T. Eggebrecht, B. R. White, S. L. Ferradal, C. Chen, Y. Zhan, A. Z. Snyder, H. Dehghani, and J. P. Culver, “A quantitative spatial comparison of high-density diffuse optical tomography and fMRI cortical mapping,” NeuroImage 61(4), 1120–1128 (2012). [CrossRef]  

27. Q.-Q. Fang and S. J. Yan, “MCX Cloud-a modern, scalable, high-performance and in-browser Monte Carlo simulation platform with cloud computing,” J. Biomed. Opt. 27(08), 083008 (2022). [CrossRef]  

28. M. A. Kamran, M. Y. Jeong, and M. M. N. Mannan, “Optimal hemodynamic response model for functional near-infrared spectroscopy,” Front. Behav. Neurosci. 9, 151 (2015). [CrossRef]  

29. Z. Shoaib, M. Ahmad Kamran, M. M. N. Mannan, and M. Y. Jeong, “Approach to optimize 3-dimensional brain functional activation image with high resolution: a study on functional near-infrared spectroscopy,” Biomed. Opt. Express 10(9), 4684–4710 (2019). [CrossRef]  

30. L. C. Schudlo and T. Chau, “Dynamic topographical pattern classification of multichannel prefrontal NIRS signals: II. Online differentiation of mental arithmetic and rest,” J. Neural. Eng. 11(1), 016003 (2014). [CrossRef]  

31. S. D. Power, A. Kushki, and T. Chau, “Intersession consistency of single-trial classification of the prefrontal response to mental arithmetic and the no-control sate by NIRS,” PLoS ONE 7(7), e37791 (2012). [CrossRef]  

32. W.-T. Chen, X. Wang, B.-Y. Wang, Y. Wang, Y. Zhang, H.-J. Zhao, and F. Gao, “Lock-in-photon-counting-based highly-sensitive and large-dynamic imaging system for continuous-wave diffuse optical tomography,” Biomed. Opt. Express 7(2), 499–511 (2016). [CrossRef]  

33. D.-Y. Liu, B.-Y. Wang, T.-T. Pan, J. Li, Z.-P. Qin, L.-M. Zhang, Z.-X. Zhou, and F. Gao, “Towards quantitative near infrared brain functional imaging: lock-in photon counting instrumentation combined with tomographic reconstruction,” IEEE Access 7(1), 86829–86842 (2019). [CrossRef]  

34. H. J. Hwang, C. Han, J. Y. Kim, W. D. Chang, D. W. Kim, K. Kim, S. Jo, and C. H. Im, “Toward more intuitive brain–computer interfacing: classification of binary covert intentions using functional near-infrared spectroscopy,” J. Biomed. Opt. 21(9), 091303 (2016). [CrossRef]  

35. F. M. Noori, N. Naseer, N. K. Qureshi, H. Nazeer, and R. A. Khan, “Optimal feature selection from fNIRS signals using genetic algorithms for BCI,” Neurosci. Lett. 647, 61–66 (2017). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1. Workflow of the Kalman-LSTM method.
Fig. 2.
Fig. 2. Time overlapping strategy: ${\textbf I}(k)$ denotes the input vectors at the specific time-point during the network creation, where n is the total number of the time-points during baseline; $O(k)$ denotes the output of the LSTM network.
Fig. 3.
Fig. 3. 3-Dimensional and longitudinal sectional view of the brain-emulating model.
Fig. 4.
Fig. 4. Simulated hemodynamic response: (a) stimulation paradigm; (b) time courses of the HbO- and HbR- concentration perturbation induced by task and the corresponding absorption perturbation; (c) time courses of the HbO- and HbR- concentration perturbation induced by the physiological interferences (when $\textrm{A}_r^G({\textbf r}) = 1$ and $\textrm{A}_h^G({\textbf r}) = 1$) and the corresponding absorption perturbation.
Fig. 5.
Fig. 5. Simulative validations: (a) absorption perturbation maps at the selected time-points; (b) time courses of the averaged absorption perturbation in the activation region; (c) quantitative evaluations of the reconstructed perturbation maps.
Fig. 6.
Fig. 6. Simulative validations under cHRF variety in sequential trials: (a) stimulation paradigm; (b) time courses of the HbO- and HbR- concentration perturbation and the corresponding reconstructed absorption perturbation in the activation region.
Fig. 7.
Fig. 7. Simulative validations under intense interferences: (a) absorption perturbation maps at the selected time-points; (b) the corresponding time courses of the averaged absorption perturbation in the activation region; (c) quantitative evaluations of the reconstructed perturbation maps.
Fig. 8.
Fig. 8. In-vivo experiment configuration: (a) stimulation paradigm; (b) picture of the measurement.
Fig. 9.
Fig. 9. A comparison of classification accuracy using different feature types from the channel with highest Fisher Score: M-LK, S-LK and K-LK denote the feature of mean, skewness and kurtosis using the Kalman-LSTM method, respectively; M-LP, S-LP and K-LP denote the feature of mean, skewness and kurtosis feature using the LP method with cut-off frequency of 0.1 Hz, respectively.
Fig. 10.
Fig. 10. Classification accuracy using feature of mean (upper) and kurtosis (down) with linear (left), gaussian (middle) and polynomial (right) kernel function: LK-, LP1- and LP2-HbO/HbR denote the accuracy computed by HbO/HbR concentration perturbation using the Kalman-LSTM, LP (0.1 Hz) and LP (0.2 Hz), respectively.
Fig. 11.
Fig. 11. Simulative validations using different Kalman-based method: (a) absorption perturbation maps at the selected time-points; (b) the corresponding time courses of the averaged absorption perturbation in the activation region; (c) quantitative evaluations of the reconstructed perturbation maps.

Tables (4)

Tables Icon

Table 1. Mathematical and technical methods to suppress physiological interferences

Tables Icon

Table 2. Optical coefficient of the brain-emulating model

Tables Icon

Table 3. Free parameters in the cHRF variety

Tables Icon

Table 4. Significance analysis of the classification accuracy

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

{ δ O D λ ( k ) = L D P F λ δ μ a λ ( k ) δ μ λ ( k ) = ε λ HbO δ C HbO ( k ) + ε λ HbR δ C HbR ( k )
y ( k ) = H ( k ) β ( k ) + e ( k )
δ μ a R ( k ) = H ( k ) X ( k ) + e ( k )
X ( k ) = AX ( k 1 ) + w ( k )
{ X ^ ( k ) = A X ^ ( k 1 ) P ( k ) = AP ( k 1 ) A T + Q G ( k ) = P ( k ) H ( k ) [ H ( k ) P ( k ) H T ( k ) + R ] 1 X ^ ( k ) = X ^ ( k ) + G ( k ) [ δ μ a R ( k ) H ( k ) X ^ ( k ) ] P ( k ) = [ E G ( k ) H ( k ) ] P ( k )
C N R ( k ) = m e a n [ δ μ a T ( k ) ] m e a n [ δ μ a B ( k ) ] s t d [ δ μ a B ( k ) ] r R M S E ( k ) = || δ μ a ( k ) δ μ a G ( k ) || 2 2 || δ μ a G ( k ) || 2 2 D F ( k ) = F W H M ( k ) D ,  
{ δ C Hbo ( k ) = α 0 [ c H R F Hbo ( k ) n ={-}  u ( k ) ] c H R F HbO ( k ) = k α 1 1 β 1 α 1 e β 1 k Γ ( α 1 ) k α 2 1 β 2 α 2 e β 2 k 6 Γ ( α 2 )
{ δ C HbO I ( r , k ) = A r G ( r ) δ C HbO r sin ( 2 π f r k ) + A h G ( r ) δ C HbO h sin ( 2 π f h k ) δ C HbR I ( r , k ) = A r G ( r ) δ C HbR r sin ( 2 π f r k ) + A h G ( r ) δ C HbR h sin ( 2 π f h k )
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.