Open Access
Add to BibSonomy
Abstract
The multispectral radiometric temperature measurement technique is affected by the unknown emissivity, and there is no multispectral radiometric temperature inversion algorithm applicable to any scene or target. To address the above problems, this paper converts the multispectral radiometric temperature inversion problem into an image recognition problem containing the temperature information to be measured, and proposes a graphical multispectral radiometric temperature adaptive inversion algorithm. In this paper, we use the difference between spectral channels to convert the one-dimensional radiation data into a two-dimensional radiation map; use the generalized inverse to obtain the spectral emissivity distribution features, fuse them with the two-dimensional radiation map, and use an improved deep learning network to achieve adaptive temperature inversion. It is experimentally verified that the algorithm proposed in this paper can achieve simultaneous inversion of temperature and emissivity for any scene or target with sufficient data set.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Multispectral pyrometry has the advantages of fast response, non-contact measurement and no upper temperature limit [1–4]. It has been widely used in many fields, such as temperature monitoring in processes in petroleum refining industry, transportation field, defense science and aerospace industry [5–8]. This technology of multispectral thermometry has been developed for decades, but its data processing still has a problem of how to accurately solve the radiation equation without information on the emissivity of the object.
Internationally, for the data processing of multispectral radiation thermometers, many scholars use the least squares method, which must assume a functional relationship between emissivity and wavelength, otherwise it cannot be solved [9–10], developed a quadratic measurement method based on the assumption that emissivity is linearly related to temperature at different wavelengths [11–13]. This method can calculate the temperature and emissivity at two consecutive temperature measurement points simultaneously. However, during the actual temperature measurement, the variation of the spectral emissivity of an object is often inconsistent at different measurement locations. Therefore, fixed-assumption models are difficult to obtain.
Neural network technology brings a new development direction for multispectral radiometric data processing. Cong et al. applied RBF networks to multispectral radiometric temperature measurement to achieve automatic identification of emissivity models and eliminate the errors caused by emissivity hypothesis models [14]. For the defect that BP neural network is easy to fall into local minima, Sun et al. proposed a multispectral radiometric temperature measurement data processing method based on genetic algorithm and neural network [15]. Song et al. designed a secondary identification method based on BP neural network, which can convert the measured brightness temperature to the real temperature [16]. Chen et al. established an adaptive emissivity model-based multispectral thermometry, which combined neural networks with genetic algorithms to obtain highly accurate temperature inversion results [17], but required the collection of a large number of data samples and long training time.
Multispectral radiation temperature measurement technology is the most important technical means in the field of high temperature measurement. It measures multiple spectral radiation brightness information of the target and obtains the target temperature by inversion algorithm. However, due to the influence of unknown emissivity, no inversion algorithm of multispectral radiative temperature measurement can be applied to any scene or target. The high precision and strong generalization ability shown by deep learning algorithm in the field of image feature recognition provides a direction to solve the above problems. Xing et al. for the first time will convert one-dimensional data into two-dimensional images for data processing, avoid using spectral emissivity hypothesis model, and can establish the nonlinear statistical relationship between temperature and multi-spectrum [18]. However, the accuracy of this algorithm is low, and half of the data is repeated. In this paper, the problem of multispectral radiation temperature inversion is transformed into an image recognition problem containing the information of temperature to be measured, and a graphic-based adaptive inversion algorithm for multispectral radiation temperature measurement is proposed. Using the data difference between spectral channels, one-dimensional radiation data is transformed into two-dimensional radiation graph. The spectral emissivity distribution features are obtained by the generalized inverse, and the two-dimensional radiation graph is fused with the two-dimensional radiation graph to construct the two-dimensional radiation graph data containing the temperature information and spectral characteristics to be measured, and the ResNet 50 model is improved.
This method solves the problems of data duplication, low accuracy and slow recognition speed of the multi-spectral radiation temperature inversion algorithm based on deep learning. The visual-near-infrared target simulation platform is built to provide data sets and experimental means for algorithm research. Simulation experiments show that the proposed high-precision and highly adaptive multi-spectral radiation temperature measurement algorithm can achieve simultaneous inversion of temperature and emissivity of any scene or target under the condition of sufficient data set, which will push the application of deep learning in multi-spectral radiation temperature measurement to a new height.
2. Improved algorithm based on generalized inverse matrix
If the multi-wavelength thermometer has n channels, the output signal of the ith channel can be expressed as
Based on the reference model of temperature, ${V_i}$ is output signal of the ith channel of the multi-wavelength thermometer, as shown in Eq. (2), at the fixed-point blackbody reference temperature ${T_c}$, the output signal of the ith channel is ${V_i}^{\prime}$, as shown in Eq. (3):
From Eq. (2) and Eq. (3), we have
For a multi-wavelength thermometer with n channels, there are n equations, but contain (n + 1) unknowns, i.e., the target true temperature T and n spectral emissivities $\varepsilon (\lambda i,T)$, and to solve this problem, a deep learning algorithm is used. At present, the spectral emissivity is mostly solved by the spectral emissivity hypothetical model as the main method, when the hypothetical model is close to the actual situation, the accuracy of the inverse temperature and spectral emissivity is very high, when the two do not match, the inverse result is very different from the actual situation, for the temperature measurement in the case of complex materials and dynamic changes of material properties during combustion, the method of the spectral emissivity hypothetical model has blindness. With the development of computer technology, deep learning algorithms with good results have emerged to invert the performance $\frac{{{V_i}}}{{V_i^{\prime}}}$, through different emissivity models, into two-dimensional images by Eq (5):
In Eq. (5), ai is $\frac{{{V_i}}}{{V_i^{\prime}}}$, the calculated value of $\frac{{{V_i}}}{{V_i^{\prime}}}$ is transformed into a value between the interval [0,1] to facilitate the definition of color classes. The result of the conversion into two-dimensional data is shown in Eq. (6):
This algorithm has been proposed in the Ref. [18]. and fully justified, which is called method 1 in this paper. half of the data of this algorithm is found to be repeated in the experiment, which not only increases the amount of invalid calculation, but also the accuracy of the algorithm is not satisfactory, based on which the generalized inverse algorithm is added, a set of emissivity that is consistent with the trend of emissivity change is performed by the generalized inverse, which is converted into a two-dimensional image, and the image is stitched diagonally with the previously The image obtained contains both the relationship between voltage and temperature, and at the same time adds an important influence factor affecting the composition of its relationship emissivity variation model, which is called method 2 in this paper. the theoretical basis is as follows:
Converting Eq. (4) into a linear equation for the temperature to voltage ratio is shown in Eq. (7):
Denoting the known quantity in Eq. by ${Y_i}$, the emissivity unknown quantity by ${X_i}$ and the containing temperature unknown quantity by Z, we have Eq. (8):
Translating this mathematical model into the form of a matrix can be expressed as Eq. (9):
That is the form of $Y = AX$, where A is $n \times ({n + 1} )$ matrix, Eq. (9) is underdetermined system of equations, can use the generalized inverse matrix theory to solve the system of equations by the formula (9), the rank of matrix A is $rankA = n$, that is, A row full rank, so the plus sign inverse A+ can be expressed in Eq. (10):
The unique solution of Eq. (9) can be expressed as Eq. (11):
X is the minimal parametric solution that matches $\begin{array}{l} \min ||x ||\\ Y = Ax \end{array}$, $Y = AX$ converges to this point. When the emissivity is $\begin{array}{l} \min ||\varepsilon ||\\ Y = Ax \end{array}$, the solution will be close to the actual emissivity. Translated into matrix form as Eq. (12):
Calculate ${x_i}$ and transform it into a two-dimensional array between [0,1] by using Eq. (5) and Eq. (6).
Four emissivity models are selected for as shown in Fig. 1, and the emissivity through the generalized inverse performance is compared with the target emissivity, which is basically the same in terms of change trend, Fig. 2 is a two-dimensional image of the voltage ratio, the reference temperature, the actual temperature point, the emissivity model for the four shown in Fig. 1, the inverse voltage ratio is performed and transformed into a two-dimensional image as shown in Fig. 3 through the formula calculation, through the voltage ratio according to the formula inverse performance emissivity change trend, through the formula to generate a two-dimensional image as shown in three, through the stitching to generate a two-dimensional image as shown in Fig. 4.
Fig. 2. According to the four emissivity models of A,B,C and D in Fig. 1, the two dimensional images of voltage ratio are inverted successively (The reference temperature is 1400 K, the actual temperature is 2000K)
Fig. 3. The two-dimensional images of emissivity models A,B,C and D are inverted by generalized inverse method (The reference temperature was 1400 K, and the actual temperature was 2000K)
Fig. 4. The spectral emissivity distribution features were obtained by generalized inverse, and fused with two-dimensional radiation graphs to construct two-dimensional radiation graph data containing temperature information and spectral characteristics to be measured
3. Experimental design and comparison
Since the amount of data collected continuously is large in measured flame, it is also very necessary to quickly invert it. The information carried by two-dimensional images in this paper is not complicated, and the image texture is clear with obvious color and bright contrast. In order to make the model more suitable for temperature inversion, an improved ResNet50 network configuration was proposed based on the characteristics of the two-dimensional radiation graph, as shown in Fig. 5.
In the network layer, in order to maximize the accuracy of the model and realize efficient deep learning, the layer is lightweight. The four layers in the traditional ResNet50 model, including 3, 4, 6 and 3 basic blocks, are simplified into 3, 2, 3 and 3 residual units. The shallow and deep feature extraction layers are retained, and part of the middle layer is deleted to greatly compress the space and reduce the amount of calculation, so as to achieve the balance between speed and accuracy.
In this paper, the BN layer is used before the activation function [19], which can accelerate the model convergence, make the model training more stable, avoid gradient explosion or gradient disappearance, and have a certain regularization effect, avoid the use of relatively unstable Dropout. In this paper, the classical ReLU function is selected as the activation function of the network. Compared with Sigmoid and Tanh, ReLU function has no saturation region and does not have the problem of disappearing gradient. Moreover, the calculation is simpler and the actual convergence is faster.
The last layer of the traditional CNN network is the fully connected layer (such as AlexNet), which leads to a large number of network parameters and is easy to cause overfitting. Lin et al. proved that replacing the original fully connected layer with global average pooling (GPA) can greatly reduce network parameters. Global average pooling can simplify the convolution structure by strengthening the consistency between feature graphs and categories, and can also sum spatial information, which is more robust to the input spatial transformation. Therefore, the GPA layer is selected in the last layer of the network in this paper.
The training temperature range was 2000K∼2400 K and the reference temperature T′ was 1400 K. The simulated dataset was generated according to Ref. [20]. A total of 401 temperature points were used. Four emissivity model classes A, B, C, and D were used at each temperature point. Each emissivity model class consists of 10 stochastic emissivity models with the same trend, for a total of 16040 data. Of these, 75% were used as the training set, 15% as the validation set, and 10% as the test data.
In the Table 1, seven training methods are used for training and comparison, because the other six methods have better training effect compared with other deep learning algorithms, they are also more commonly used algorithms, and the network model is relatively stable. In this experiment, ResNet network has better performance, better accuracy and better training speed. In this paper, ResNet 50 network with better performance is optimized and improved. From Fig. 5, it can be seen that the improved ResNet 50 model converges faster in the training process. At about the 30th Epoch, the model training accuracy reaches more than 85.0%, and the recognition time is faster. The improved network not only gives consideration to the accuracy rate but also improves the identification speed, which has more advantages in the measured temperature
Table 1. Comparison of seven types of neural networksa
By using the improved ResNet50 network, the paper compares (Method 1, Ref. [18].) and the deep learning-based generalized inverse-graph multispectral radiation temperature inversion algorithm (Method 2, our work). As can be seen from Fig. 6, in four different emissivity models (class A, class B, class C, and class D), error of the noise-free graphical deep learning multispectral radiation algorithm using method 1 is less than 0.75%, and the error using method 2 is less than 0.62%, which is an accuracy improvement of 0.13%. After adding 5% random noise, the error is less than 1.80% for method 1 and less than 1.56% for method 2. The accuracies were improved by 0.27%, respectively. The simulation results show that the incorporation of the emissivity model with generalized inverse performance significantly improves the accuracy and noise immunity under the same algorithm.
4. Experimental verification
To verify the algorithm described in this paper, an experimental platform was built using the blackbody (DY-HT3, ε≥0.995, 573 K ∼ 1473 K), filters (blue, green and red), and the micro spectrometer (Ocean Optics FLAME-S-VIS-NIR-ES) for the verification experiments, as shown in Fig. 7. The blackbody is used as the target to be measured, and the filter simulates the actual emissivity. By collecting multi-spectral information from the spectrometer and combining with the inversion algorithm proposed in this paper, the blackbody temperature is retrieved. The transmittance of filters is shown in Fig. 8. The reference temperature $T^{\prime}$ is 1023 K. The validation results are shown in Fig. 9. Simulation experiments (specific data) again verify the superiority and stability of the improved ResNet 50 algorithm model and the performance of the generalized inverse graphical multi-spectral radiometric adaptive inversion algorithm, whose inversion emissivity values are basically close to the measured emissivity values with small relative errors, verifying the high accuracy and high adaptability of the multi-spectral radiometric algorithm, which can realize the simultaneous inversion of temperature and emissivity of any scene or target under the condition of sufficient data set.
Fig. 8. Comparison of the transmittance obtained from the calibration experiments of Methods 1 (algorithms that do not fuse generalized inverses, Ref. [18].) and 2(ours)with the inversion emissivity of the three filters (calibration experiment temperature is 1147 K).
Fig. 9. The relative error verification experiment of temperature inversion in Method 1(algorithms that do not fuse generalized inverses, Ref. [18].) and Method 2(ours)
5. Conclusion
In this paper, we proposed a graphical multi-spectral radiometric adaptive inversion algorithm. The data differences between spectral channels are used to transform the 1D radiometric data into 2D radiograms; the generalized inverse is used to obtain the spectral emissivity distribution features and fuse them with 2D radiograms to construct 2D radiogram data containing the temperature information and spectral features to be measured. As for the training method, this paper improves the ResNet 50 network, and makes a comparative analysis with other six algorithms, verifying that the training time and accuracy of the improved algorithm model are greatly improved. In order to verify the feasibility of the algorithm, this paper builds a visual-near-infrared target simulation platform to provide data sets and experimental means for the algorithm research. Simulation experiments show that the temperature error of the algorithm inversion in this paper is less than 0.6%, which verifies that the multi-spectral radiation temperature measurement algorithm proposed in this paper has high accuracy and strong adaptability. It is expected to solve the simultaneous inversion of arbitrary target temperature and emissivity.
Funding
National Natural Science Foundation of China (32371864, 61975028, 62305053); Fundamental Research Funds for the Central Universities (2572022BH02); Natural Science Foundation of Heilongjiang Province (LH2023F005); Hei Long Jiang Postdoctoral Foundation (LBH-Z22052).
Disclosures
The authors declare no conflicts of interest.
Data Availability
Data underlying the results presented in this paper are notpublicly available at this time but may be obtained from the authors upon reasonable request.
References
1. Y. Zhang, Z. Zou, and F. A. Yan, “Ridge estimation-sequence quadratic programming constraint optimization algorithm for data processing in multi-spectral thermometry,” Meas. Sci. Technol. 33(8), 085501 (2022). [CrossRef]
2. Z.-M. Li, S.-B. Wu, J. Gao, et al., “Fast correlated-photon imaging enhanced by deep learning,” Optica 8(3), 323–328 (2021). [CrossRef]
3. C. Yang, Y. Yu, D. Zhao, et al., “Study on modeling of multispectral emissivity and optimization algorithm,” IEEE Trans. Neural Netw. 17(1), 238–242 (2006). [CrossRef]
4. C. Rodiet, B. Remy, and A. Degiovanni, “Optimal wavelengths obtained from laws analogous to the wien's law for monospectral and bispectral methods, and general methodology for multispectral temperature measurements taking into account global transfer function including non-uniform emissivity of,” Infrared Phys. Technol. 76, 444–454 (2016). [CrossRef]
5. H. Huang, Z. Zhang, Y. Niu, et al., “Review of multispectral radiation thermometry data processing algorithms,” Infrared Phys. Technol. 129, 104573 (2023). [CrossRef]
6. T. Liu and A. Fiore, “Designing open channels in random scattering media for on-chip spectrometers,” Optica 7(8), 934–939 (2020). [CrossRef]
7. J. Ryu, J. Kim, H. Kim, et al., “Multispectral interaction convolutional neural network for pedestrian detection,” Comput Vis Image Und 223, 103554 (2022). [CrossRef]
8. Z. Hayran, A. Chen, and F. Monticone, “Spectral causality and the scattering of waves,” Optica 8(8), 1040–1049 (2021). [CrossRef]
9. P. Liu and K. B. Eom, “Compressive sensing of noisy multispectral images,” IEEE Geosci. Remote Sensing Lett. 11(11), 1931–1935 (2014). [CrossRef]
10. Y. Zhang, W. Yang, W. Zhang, et al., “Two-step ResUp&Down generative adversarial network to reconstruct multispectral image from aerial RGB image.,” Comput Electron Agr 192, 106617 (2022). [CrossRef]
11. C.-D. Wen and T.-Y. Chai, “Examination of multispectral radiation thermometry using linear and log-linear emissivity models for aluminum alloys,” Heat Mass Transfer 47(7), 847–856 (2011). [CrossRef]
12. P. Y. C. R. Taunay and E. Y. Choueiri, “Multi-wavelength pyrometry based on robust statistics and cross-validation of emissivity model,” Rev. Sci. Instrum. 91(11), 114902 (2020). [CrossRef]
13. J. Xing, R. S. Rana, and W. Gu, “Emissivity range constraints algorithm for multi-wavelength pyrometer (MWP),,” Opt. Express 24(17), 19185 (2016). [CrossRef]
14. D. C. Cong, J. M. Dai, X. G. Sun, et al., “Study of the application of RBF network to multi-spectral thermometry,” J. Infrared Millim W 20(2), 97–101 (2001).
15. S. Xiao-Gang and Y. Gui-Bin, “Multi-spectral thermometry based on GA-BP algorithm,” Spectroscopy and Spectral Analysis 27(2), 213–216 (2007).
16. S. Yang, X. Sun, and T. Hong, “Method for changing brightness temperature into true temperature based on twice recognition method,,” Opt Lett 5(8), 457–459 (2007).
17. A. Lc, S. A. Shang, G. A. Shan, et al., “Multi-spectral temperature measurement based on adaptive emissivity model under high temperature background-sciencedirect,” Infrared Phys. Technol. 111, 103523 (2020). [CrossRef]
18. J. Xing, J. Guo, S. Cui, et al., “Graphical multispectral radiation temperature inversion algorithm based on deep learning,” Opt. Lett. 48(8), 2166–2169 (2023). [CrossRef]
19. Y. Xu and Y. Ma, “Evolutionary neural architecture search combining multi-branch ConvNet and improved transformer,” Sci. Rep. 13(1), 15791 (2023). [CrossRef]
20. X. G. Sun, J. M. Dai, D. C. Cong, et al., “True temperature measurement by radiation method based on neural network model,” J. Infrared Millim W 20(2), 151–153 (2001).
