Open Access
Add to BibSonomy
Abstract
It is of great interest to explore the possibility of applying machine learning methods for tunable diode laser absorption spectroscopy (TDLAS). Conventional supervised machine learning could be used for accurate determination of gas properties such as temperature or concentration in TDLAS. However, it becomes quite challenging when there is only a small amount of measured data. In this work, we propose a transfer machine learning (TML) model for accurate temperature prediction from a small amount of measured data. In the experiment, a two-line thermometer is developed by exploiting H2O absorption lines centered at 1392 and 1371 nm. Based on the limited data of measured laser transmission and an easily obtained large amount of calculated absorption spectra, the proposed model attempts to reduce the feature difference between these two distinct types of datasets and then leverages the large labeled calculated data to build an accurate predictor for the unlabeled measured data. The TML method can achieve a temperature prediction with a mean absolute error of 0.02-0.77 K over the temperature range of 288-338 K and a Pearson correlation coefficient of -0.084 to 0.981.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Tunable diode laser absorption spectroscopy (TDLAS) is a widely applied technique for fast, sensitive, species-specific, and non-invasive detection of gas concentration, temperature, pressure, and flow velocity [1,2]. TDLAS-based sensors rely on the measurement of spectral absorbance by tuning the laser frequency across the target absorption lines. When the laser beam is transmitted through the uniform gas medium, the fractional transmission is related to the gas properties by Beer’s law:
In TDLAS, the wavelength of a diode laser is scanned across the target absorption line(s) by tuning the injection current or the laser temperature. The laser transmission encoded with the information of gas absorption is detected by a photodetector [2]. It should be noted that for diode lasers, the laser intensity also changes with the injection current. Hence, the baseline ${I_0}(\mathrm{\nu } )$ is normally obtained by fitting the non-absorption region of the measured transmission signal. If two absorption lines of a species are measured simultaneously at the same pressure, gas concentration, and optical path length, the ratio of these two integrated spectral absorbances is only a function of temperature [1]. Hence, the gas temperature can be derived from the spectral measurement of these two absorption lines, which is also known as two-line thermometry. TDLAS-based two-line thermometry has been widely used for non-invasive diagnostics for combustion applications [3]. Inferring gas properties such as temperature and concentration from raw measured data needs to fit or measure a baseline by human experience, which may cause human-introduced residual. Instead of taking efforts to obtain the non-absorption baseline, it is possible to find the mapping relation between the measured transmission signal and temperature.
Machine learning (ML) is one of the most rapidly developing research fields and has powered dramatically many aspects of society, from laboratory research to practical applications in science, engineering, technology and commerce [4,5]. Based on a dataset, this method learns the rules underlying the dataset and then builds a model to make predictions. Among different ML methods, supervised learning is the most common and mature one used for classification and regression [6]. The supervised learning method includes linear models, ridge regression, support vector machines, decision trees, random forest, and neural network models [6]. It has been used in the majority of ML studies in optical spectroscopy [7–14]. The relatively early application of ML in spectroscopy was to use neural-network-based retrieval algorithms to retrieve the surface temperature from the data of high-resolution infrared atmospheric sounding interferometer [7,8]. In combustion diagnostics, the temperature of CO2 and H2O was inferred from infrared emission spectrum using neural network-based retrieval techniques, which proved that ML had advantages of high precision, strong robustness and better adaptability [9–11]. The principal component analysis was used to reduce the dimensionality of high-resolution interferometric data, which demonstrated that ML was an efficient method to compress the high dimensional data and extract the features [12]. Additionally, the temperature was retrieved from calculated high-resolution CO2 spectra and infrared emission measurements using the neural network-based inversion model [13]. The ML model could solve highly nonlinear and high-dimensional optical spectral data [13,14]. However, these applications of ML (especially deep learning models) are computationally intensive and are based on a large amount of available data. When using emission or absorption spectroscopic techniques for practical applications, it is very difficult to collect a large measured dataset for training ML models.
The conventional supervised ML works well only under the assumption that the training and testing data are from the same feature space and distribution. When the feature distributions of the training and testing data are different, most supervised ML models are not applicable or need to be retrained from scratch. In contrast, transfer machine learning (TML) allows the domain distributions of training and testing data to be different. This method tries to transfer knowledge learned from the previous data and tasks to solve new problems when there are fewer training data in the new situation [15]. In general, TML methods have been widely applied in many different fields such as website classification, image recognition, natural language understanding, and drug molecule research [4,5,15,16].
In TDLAS, although it is hard or costly to collect many measured data (i.e. laser transmission data) of interest to train the ML model, the absorbance can be accurately calculated at a particular temperature, pressure, gas concentration, and path length, with the knowledge of spectroscopic parameters (i.e., line-strength and broadening coefficients) available in the HITRAN database [17]. Hence, transfer learning can fill the gap of using limited measured data and numerous calculated data in TDLAS measurements. In this paper, we propose the use of a TML model for TDLAS-based temperature measurements. In the demonstration, the dataset used for the proposed model includes a large amount of labeled calculated absorbance of H2O based on the HITRAN database and a small amount of measured laser transmission data generated by TDLAS measurements. The marginal and conditional distributions adaptation (MCDA) model is adopted to construct a new representation space for reducing the difference between these two types of data. These two types of data are utilized by four standard regression models for temperature prediction. We show that the temperature prediction made by the proposed TML model achieves a significant improvement in terms of mean absolute error and Pearson correlation coefficient when compared to the standard machine learning with dimensionality reduction techniques.
2. Data collection for transfer machine learning
In order to learn the temperature information that underlies the spectral data, the calculated absorbance of the target H2O lines and the measured laser transmission through the H2O mixtures under the same condition of concentration and temperature are used as the training and testing datasets. Hence, the data preparation for TML includes a large amount of calculated absorbance data and a small amount of laser transmission measurement.
2.1 Spectral data calculation
We select the line pair of H2O centered at 1371 and 1392 nm, respectively, for two-line thermometry. The spectroscopic parameters are taken from the HITRAN database and provided in Table 1. The absorbance of H2O near 1371 and 1392 nm as a function of wavenumber can be calculated using Eq. (1). Figure 1 depicts the representative absorption spectra of H2O at the temperature of 299.56 K, pressure of 1 atm and mole fraction of 2.9%. By changing the gas conditions, different absorption spectra can be calculated accordingly.
Fig. 1. Calculated absorbance of H2O lines centered at 1371 and 1392 nm: T = 299.56 K, P = 1 atm, xH2O = 2.9% and L = 13 cm.
2.2 Laser transmission measurement
Two distributed feedback (DFB) diode lasers at 1371 and 1392 nm are used to exploit the H2O absorption lines. As shown in Fig. 2, these two near-infrared (NIR) diode lasers with fiber output are combined as a single laser beam to transmit through a gas cell with a total length of 13 cm. The gas cell with a PID-temperature-controller is specially designed to provide a stable temperature environment in the range of 288-338 K. The transmitted laser beam is collected by an Au-coated concave mirror onto a photodetector. Here the wavelengths of the two diode lasers are scanned across the target absorption lines by tuning the injection current with time. The typical features of the calculated absorbance and the measured laser transmission are depicted in Fig. 2. By selecting the data of the same length, these two types of data can be directly used in the TML without any pre-processing.
3. Transfer machine learning strategy
The TML strategy aims to reduce the feature difference between different datasets to leverage the large labeled calculated data to build an accurate predictor for the measured data [15]. The TML model learns a certain transfer feature representation across domains in a reproducing kernel Hilbert space. This method maps these two distinct types of data into a shared low-dimensional potential space (new representation space), as shown in Fig. 3. As a result, new features are generated and then utilized by the standard ML models for gas property predictions.
Fig. 3. Schematic of transfer machine learning used for TDLAS. In the measured space, the laser transmission signal (in voltage) of the two DFB lasers (1371 and 1392 nm) is measured as a function of time (in microsecond). In the calculated space, the absorbance of the two H2O lines is calculated as a function of laser frequency (in wavenumber). In the new representation space, MCDA-TML generates the new calculated absorbance and measured laser transmission data, which are utilized as the training and testing data by the standard machine learning model (EN, electric net; SVR, support vector regression; RF, random forest; and KRR, kernel ridge regression) for temperature prediction.
We begin with transforming the problem of TDLAS-based temperature prediction into a cross-domain TML problem. The rich calculated absorption spectra are treated as a labeled data domain ${{{\cal D}}_s} = \{{ {{{{\cal X}}_s},{{{\cal Y}}_s}} |P({{\boldsymbol x}_s})} \}$, where ${{{\cal X}}_s}$ and ${{{\cal Y}}_s}$ denote the feature space and temperature space of the calculated absorption spectra, respectively; ${{\boldsymbol x}_s}$ denotes the vector format of the calculated absorption spectra (${{\boldsymbol x}_s} \in {{{\cal X}}_s}$), and ${y_s}$ denotes the temperature value corresponding to ${{\boldsymbol x}_s}$ (${y_s} \in {{{\cal Y}}_s}$); $P({{\boldsymbol x}_s})$ is the marginal probability distribution of the calculated absorption spectrum. Similarly, the measured transmission data by TDLAS are considered to be the unlabeled learning target domain ${{{\cal D}}_m} = \{{ {{{{\cal X}}_m},{{\cal Y}}{\textrm{t}_m}} |P({{\boldsymbol x}_m})} \}$, where ${{{\cal X}}_m}$ denotes the feature space of the measured laser transmission and ${{\boldsymbol x}_m}$ denotes the vector format (${{\boldsymbol x}_m} \in {{{\cal X}}_m}$); $P({{\boldsymbol x}_m})$ is the marginal probability distribution of the transmission measurement. Additionally, ${{\cal Y}}{\textrm{t }_m}$ denotes the unknown value to be predicted. Our task is to seek a target model ${f_m}({\cdot} )$ to predict the ${{\cal Y}}{\textrm{t }_m}$ in domain ${{{\cal D}}_m}$ using the knowledge obtained from ${{{\cal D}}_s}$ and ${{{\cal D}}_m}$. The subscripts $s$ and $m$ denote the calculated and measured data, respectively.
The goal of TML is to minimize the distance between the calculated absorption spectra and the measured laser transmission data. Here the distance is defined by the maximum mean discrepancy (MMD) in a reproducing kernel Hilbert space (${{\cal H}}$) [18]. To transfer knowledge cross domains, in this work the MCDA-TML method is utilized to seek a transformation matrix ${\mathbf A}$ to minimize the distance between $P({{\mathbf A}^\textrm{T}}{{\boldsymbol x}_s})$ and $P({{\mathbf A}^\textrm{T}}{{\boldsymbol x}_m})$, as well as to minimize the distance between the conditional probabilities $P({y_s}|{{{\mathbf A}^\textrm{T}}{{\boldsymbol x}_s}} )$ and $P({y_m}|{{{\mathbf A}^\textrm{T}}{{\boldsymbol x}_m}} )$. The complete distance between the two types of data distributions (calculated absorption spectra and measured laser transmission) is determined by:
4. Results and discussion
4.1 Assessment of feature similarity
To thoroughly evaluate the feature transfer performance of MCDA-TML for two distinct types of data, two conventional feature transformation techniques, the linear dimensionality reduction (principal component analysis, PCA [19]) and nonlinear dimensionality reduction (isometric feature mapping, Isomap [20]), are also implemented for comparison. Specifically, the same amount of calculated and experimental data is generated under the same condition (temperature 295 K-319 K, relative humidity 50-65%, pressure 1 atm, and absorption path length 13 cm). Each calculated absorption spectrum contains absorbance values at 1452 frequency points and the corresponding measurement contains voltage values at 1452 sampling time points. The 1452 features of calculated and measured data are transformed to 100 features by MCDA-TML, PCA and Isomap, respectively. The raw and transformed data are visualized by the t-distributed stochastic neighbor embedding (TSNE) method shown in Fig. 4 and more information of TSNE is given in the Supplement 1. The raw data and transformed data by PCA or Isomap can still be intuitively divided into two categories, whereas the transformed data by MCDA-TML can hardly be classified. Hence, MCDA-TML reduces the distance between the calculated and measured data domains.
Fig. 4. Qualitative visualization of the calculated absorbance data and the measured transmission data via MCDA-TML, PCA and Isomap: (a) raw data visualization, (b) transformed data visualization by PCA, (c) transformed data visualization by Isomap, and (d) transformed data visualization by MCDA-TML.
The hierarchical clustering and dynamic time warping (DTW) methods are further utilized to measure the similarity of raw and transformed data in a quantitative way. The detailed description of hierarchical clustering and DTW is given in the Supplement 1. From the point view of quantitative clustering, here we use the Euclidean distance to evaluate two distinct types of data. Then we build the hierarchy from individual spectral data by progressively merging clusters. Figure 5 presents the dendrogram of the hierarchical clustering of calculated and measured data, which are transformed by PCA, Isomap and MCDA-TML, respectively. The shorter distance between data points corresponds to a better data similarity. It is observed that the distance between the transformed calculated and measured data obtained by MCDA-TML (Fig. 5 (d)) is smaller than those obtained by PCA (Fig. 5 (b)) or Isomap (Fig. 5 (c)), which indicates that two distinct types of data are drawn more closely by MCDA-TML.
Fig. 5. Hierarchical clustering of measured data and calculated spectra: (a) raw data, (b) transformed data by PCA, (c) transformed data by Isomap, and (d) transformed data by MCDA-TML.
Regarding the quantitative curve similarity, DTW is an algorithm for measuring the similarity between two curves, which is used here for evaluating the similarity between calculated spectral curves and transmission signal curves [21]. The smaller value of DTW for the two curves means the higher similarity. Figure 6 depicts the heatmap of DTW values obtained between the calculated and measured data. Each pixel in the figure represents a DTW value. The DTW values of the raw and transformed data by PCA or Isomap are much larger than that of the transformed data by MCDA-TML. Hence, the transformation by MCDA-TML leads to the higher similarity between the calculated and measured data. Such a high similarity across different types of data guarantees the establishment of knowledge transfer.
Fig. 6. Similarity between the measured transmission data and the calculated spectra analyzed by the DTW method: (a) raw data, (b) transformed data by PCA, (c) transformed data by Isomap, and (d) transformed data by MCDA-TML.
4.2 Analysis of prediction results
We then evaluate the predictive capability of temperature using TML for TDLAS measurements under the condition of only a small amount of measured data available. Four regression models (EN, SVR, RF and KRR) are trained and tested in three scenarios listed in Table 2.
Table 2. Training and testing protocols in different scenarios
The prediction accuracies for scenarios A, B and C are illustrated in Fig. 7 and Fig. 8, respectively. All scenarios are evaluated statistically using mean absolute error (MAE) and Pearson correlation coefficient (PCC) as metrics. PCC, set between -1 and 1, is a measurement of the linear correlation between the predicted and true temperature values. For the boxplots shown in Fig. 7 and Fig. 8, the bottom and top of the box denote the lower quartile (25%) and upper quartile (75%), respectively. The whiskers correspond to the 1th percentile and 99th percentile of all the errors, and the dots correspond to the minimum and maximum of all the errors. There exist a significantly lower error and a higher PCC for scenario C (MAE = 0.02 − 0.77 K, PCC = -0.084 − 0.981) compared to scenario A (MAE = 15.24 K, PCC = -0.946) and scenario B (MAE = 0.82 − 23.76 K, PCC = -0.084 − 0.981). Hence, MCDA-TML significantly outperforms conventional ML models with dimensionality reduction. The larger PCC intuitively reflects a higher linear correlation between the prediction and true temperature values.
Fig. 7. Error and Pearson correlation coefficient between the true and prediction values for scenarios A and B. EN, electric net; SVR, support vector regression; RF, random forest; KRR, kernel ridge regression; Mea, measured data; PCA, principal component analysis; KPCA, kernel principal component analysis; Isomap, isometric feature mapping.
Fig. 8. Error and Pearson correlation coefficient between the true and prediction values for scenario C. The non, linear and rbf denote non-kernel, linear kernel and radial basis function kernel for constructing the kernel matrix of MCDA-TML, respectively.
Under the same test conditions of regression models, scenarios A and B evaluate the transfer learning ability and few sample learning ability of conventional supervised learning, respectively. Scenario C evaluates the transfer learning performance of TML across data domains. It is observed from Scenario A that conventional supervised learning could not be applied for the cross-domain prediction. The results of scenario B indicate that the conventional supervised learning cannot work well only using a small amount of measured data. Notably, the results of scenario C demonstrate that MCDA-TML is an effective method by leveraging the labeled calculated spectra to construct an accurate regressor for the measured transmission signal. In addition, non-kernel, linear kernel and radial basis function (RBF) kernel are used for constructing the kernel matrix of MCDA-TML. The linear kernel achieves a better performance than RBF kernel and non-kernel function, which agrees well with the conclusion that the linear kernel is often adequate for high-dimensional data [22]. The comparison of computational time and the influence of the dimension of transformed features are provided in Supplement 1.
4.3 Influence of hyper-parameter
In MCDA-TML, the regularization term (λ, in Eq. (3)) has a direct influence on the prediction performance of gas temperature. This parameter guarantees the optimization formulation (Eq. (3) to be a well-defined optimization problem. As the training and testing data belong to different data types, the cross-validation method commonly used in ML could not be used for tuning the optimal parameters. Thus we empirically tune the parameter for the optimal parameter. The searching range for λ is set from 10−4 to 50 as shown in Fig. 9. Theoretically, the smaller λ makes the optimization problem ill-defined and thus the MAE of temperature prediction increases. The larger λ value could reduce the regularization term and may not transform the features of spectral data to the new robust representations.
Fig. 9. Influence of regularization term (λ) on the MAE of temperature prediction based on MCDA-TML.
5. Conclusions
In summary, we propose a TML model (MCDA-TML) to make temperature predictions by using the small amount of measured data (two-line thermometry at 1392 and 1371 nm) and the easily obtained calculated absorption spectra (HITRAN database). The hierarchical clustering and dynamic time warping are used to investigate the similarity between measured and calculated data. The results highlight that the MCDA-TML method enables a high similarity between the transformed calculated spectra and measured data. Besides, our results present that the proposed model achieves a higher prediction accuracy (MAE = 0.02-0.77 K, PCC = -0.084-0.981) than the conventional supervised regression model under the common scenario of a small amount of measured data. This method is promising for high-accuracy temperature prediction when only a small amount of measured absorption spectra are available. Future work also involves the application of the proposed MCDA-TML method for TDLAS measurements in harsh environments.
Funding
State Key Laboratory of Control and Simulation of Power System and Generation Equipment (SKLD21KM02); Natural Science Foundation of Shenzhen City (JCYJ20200109143008165); Basic and Applied Basic Research Foundation of Guangdong Province (2020A0505090010, 2021A1515012276).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are available in Ref. [23].
Supplemental document
See Supplement 1 for supporting content.
References
1. C. S. Goldenstein, R. M. Spearrin, J. B. Jeffries, R. Hanson, and C. Science, “Infrared laser-absorption sensing for combustion gases,” Prog. Energy Combust. Sci. 60, 132–176 (2017). [CrossRef]
2. G. Rieker, H. Li, X. Liu, J. Jeffries, R. Hanson, M. Allen, S. Wehe, P. Mulhall, and H. Kindle, “A diode laser sensor for rapid, sensitive measurements of gas temperature and water vapour concentration at high temperatures and pressures,” Meas. Sci. Technol. 18(5), 1195–1204 (2007). [CrossRef]
3. E. Furlong, D. Baer, and R. Hanson, “Combustion control using a multiplexed diode-laser sensor system,” in Symposium (International) on Combustion, (Elsevier, 1996), 2851–2858.
4. M. I. Jordan and T. Mitchell, “Machine learning: Trends, perspectives, and prospects,” Science 349(6245), 255–260 (2015). [CrossRef]
5. Y. LeCun, Y. Bengio, and G. . Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]
6. T. M. Mitchell, “Machine learning,” (Mcgraw Hill1997).
7. J. Hadji-Lazaro, C. Clerbaux, and S. Thiria, “An inversion algorithm using neural networks to retrieve atmospheric CO total columns from high-resolution nadir radiances,” J. Geophys. Res.: Atmos. 104(D19), 23841–23854 (1999). [CrossRef]
8. F. Aires, A. Chédin, N. A. Scott, and W. Rossow, “A regularized neural net approach for retrieval of atmospheric and surface temperatures with the IASI instrument,” J. Appl. Meteor. 41(2), 144–159 (2002). [CrossRef]
9. E. Garcia-Cuesta, F. de la Torre, and A. J. de Castro, “A comparative study of supervised learning techniques for the radiative transfer equation inversion,” in Proceedings of the world congress on engineering and computer science, (Citeseer, 2007), 24–26.
10. E. García-Cuesta, I. M. Galván, and A. de Castro, “Multilayer perceptron as inverse model in a ground-based remote sensing temperature retrieval problem,” Eng. Appl. Artif. Intel. 21(1), 26–34 (2008). [CrossRef]
11. E. García-Cuesta, A. J. de Castro, I. M. Galván, and F. López, “Temperature profile retrieval in axisymmetric combustion plumes using multilayer perceptron modeling and spectral feature selection in the infrared CO2 emission band,” Appl. Spectrosc. 68(8), 900–908 (2014). [CrossRef]
12. H.-L. Huang and P. Antonelli, “Application of principal component analysis to high-resolution infrared measurement compression and retrieval,” J. Appl. Meteorol. 40(3), 365–388 (2001). [CrossRef]
13. S. Cięszczyk, “Determination of plume temperature distribution based on the ratios of the radiation intensities of multiple CO2 lines,” in Optical Sensors 2015, (International Society for Optics and Photonics, 2015), 950623.
14. T. Ren, M. F. Modest, A. Fateev, G. Sutton, W. Zhao, and F. Rusu, “Machine learning applied to retrieval of temperature and concentration distributions from infrared emission measurements,” Appl. Energy 252, 113448 (2019). [CrossRef]
15. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010). [CrossRef]
16. F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, “A comprehensive survey on transfer learning,” Proc. IEEE 109(1), 43–76 (2021). [CrossRef]
17. I. E. Gordon, L. S. Rothman, C. Hill, R. V. Kochanov, Y. Tan, P. F. Bernath, M. Birk, V. Boudon, A. Campargue, K. Chance, and R. Transfer, “The HITRAN2016 molecular spectroscopic database,” J. Quant. Spectrosc. Radiat. Transfer 203, 3–69 (2017). [CrossRef]
18. M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer feature learning with joint distribution adaptation,” in Proceedings of the IEEE international conference on computer vision, 2013), 2200–2207.
19. S. Wold, K. Esbensen, and P. Geladi, “Principal component analysis,” Chemom. Intell. Lab. Syst. 2(1-3), 37–52 (1987). [CrossRef]
20. J. B. Tenenbaum, V. De Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science 290(5500), 2319–2323 (2000). [CrossRef]
21. T. Rakthanmanon, B. Campana, A. Mueen, G. Batista, M. B. Westover, Q. Zhu, J. Zakaria, and E. Keogh, “Addressing big data time series: Mining trillions of time series subsequences under dynamic time warping,” ACM Trans. Knowl. Discov. Data 7(3), 1–31 (2013). [CrossRef]
22. S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, “Domain adaptation via transfer component analysis,” IEEE Trans. Neural Netw. 22(2), 199–210 (2011). [CrossRef]
23. Y. Yi, D. Kun, R. Li, R. Ni, and W. Ren, “Absorption spectra,” figshare, 2021, https://figshare.com/articles/dataset/absorption_spectra/16577303.
