Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Rapid in-situ calibration of computational micro-spectrometer with few-shot meta-learning

Open Access Open Access

Abstract

Computational micro-spectrometers comprised of detector arrays and encoding structure arrays, such as on-chip Fabry-Perot (FP) cavity filters, have great potential in many in-situ applications owing to their compact size and snapshot imaging ability. Given manufacturing deviation and environmental influence are inevitable, easy and effective calibration for spectrometer is necessary, especially for in-situ applications. Currently calibration strategies based on iterative algorithms or neural networks require accurate measurements of pixel-level (spectral) encoding functions through monochromator or large amounts of standard samples. These procedures are time-consuming and expensive, thereby impeding in-situ applications. Meta-learning algorithms with few-shot learning ability can address this challenge by incorporating the prior knowledge in the simulated dataset. In this work, we propose a meta-learning algorithm free of measuring encoding function or large amounts of standard samples to calibrate a micro-spectrometer with manufacturing deviation effectively. Our micro-spectrometer comprises 16 types of FP filters covering a wavelength range of 550-720nm. The center wavelength of each filter type deviates from the design up to 6nm. After calibration with 15 different color data, the average reconstruction error on the test dataset decreased from 7.2 × 103 to 1.2 × 103, and further decreased to 9.4 × 104 when the calibration data increased to 24. The performance is comparable to algorithms trained with measured encoding function both in reconstruction error and generalization ability. We estimated that the cost of in-situ calibration through reflectance measurements of color chart decreased to one percent of the cost through monochromator measurements. By exploiting prior deviation information in simulation data with meta-learning, the efficiency and cost of calibration are significantly improved, thereby facilitating the large-scale production and in-situ application of micro-spectrometers.

© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Spectrometers play an indispensable role in chemical analysis, biomedicine, geological research, physical astronomy, and other fields [14]. Traditional spectrometers are bulky and expensive because of sophisticated interference components, large dispersive optical elements and complex design of optical paths [57], which hinder their application in portable in-situ measurements and cost-sensitive scenarios.

In recent years, miniaturized spectrometers have developed rapidly. The technical approaches can be divided into three categories, including miniaturization of dispersion elements [8], miniaturization of interference components [9], and computational spectrometer based on encoding detectors [1012]. Recently, computational spectrometers are appealing to both academia and industry due to its snapshot operation mode and deep integration with artificial intelligence [1318]. Machine learning algorithms, such as compressed sensing [19,20] or neural network algorithms [14], have been proved to be advantageous to learn complex and non-linear relationship from data. Thus, it's promising to utilize machine learning algorithms to reconstruct the original spectrum encoded by non-linear encoding functions.

Encoding detectors employ different techniques, including integrated photonic crystals array [19], integrated Fabry-Perot (FP) cavities array [21], component gradients material [22], or responsive tunable material [23]. Since FP cavity and photonic crystal are compatible with mature semiconductor manufacturing processes, they have become the most practical strategies. Unfortunately, the encoding functions of the FP cavity and photonic crystal are extremely sensitive to the thickness or geometric pattern of the film. Since current semiconductor manufacturing accuracy is limited, the deviation of the actual encoding functions from the design result is inevitable. In addition, environmental influence on encoding functions is also unavoidable. Current strategy to calibrate spectrometers is to measure the encoding function of each pixel with monochromator or large amounts of standard samples [24]. This process is time-consuming and expensive. Especially for in-situ applications which require frequent calibration, the cost and timeliness of these calibration procedures are unacceptable. Meta-learning is a training strategy used in neural networks for multi-task scenarios [25]. By incorporating prior knowledge about different tasks in the training datasets, it meta-trains the neural network to obtain hyperparameters for a set of tasks. When performing specific tasks, few-shot experimental data are used to optimize suitable hyperparameters for the task quickly. By considering various deviation knowledge as different tasks, this method could be leveraged for rapid calibration of the micro-spectrometers.

In this work, we proposed an in-situ deviation calibration strategy based on meta-learning to develop a (computational) micro-spectrometer free from traditional expensive calibration procedure. The spectrometer is composed of a CMOS chip and an integrated FP cavity array. The FP cavity array has 16 types of filters, covering the spectral range from 550 to 720 nm. The full width at half maxi-mum (FWHM) of each channel varies from 8 nm to 10 nm. The average deviation between the experiment center wavelength and the simulation is 5 nm. Consequently, the average reconstruction error (mean square error, simplified as MSE) of the test dataset rises from 4.5 × 10−4 to 7.2 × 10−3. Since the prior knowledge of fabrication deviation is included in the simulation dataset, the meta-learning-assisted micro-spectrometer can be calibrated by few-shot measurement. After calibration through 15 color reflectance in color chart, the MSE dropped to 1.2 × 10−3, and further decreased to 9.4 × 10−4 when the calibration data is increased to 24. This strategy greatly improves the calibration timeliness and cost of micro-spectrometers, facilitating the large-scale production and application of micro-spectrometers.

2. Operational principle

The workflow of the computational spectrometer is described in Fig. 1(a). The spectra of the test sample X(λ) are encoded by the detector pixels of the spectrometer into a photocurrent vector [I1, I2, …, In]. Then, the photocurrent vector input into a trained algorithm can be expressed by Eq. (1) [26]:

$${I_i} = \mathop \smallint \nolimits_{{\lambda _1}}^{{\lambda _2}} \frac{{\lambda q}}{{hc}}QE(\lambda )P(\lambda ){T_i}(\lambda )X(\lambda )d\lambda ,$$
in which λ1 and λ2 are the lower and upper limits of the incident spectral distribution, λ, q, h, c, QE(λ), P(λ) and Ti(λ) represent wavelength, unit charge, plank constant, light velocity, quantum efficiency, the radiation spectrum of the light source and the transmittance (function) of the i-th FP filter, respectively. Supposing $A(\lambda )= \frac{{\lambda q}}{{hc}}P(\lambda )$ [27], Eq. (1) is finally simplified to Eq. (2):
$${I_i} = \mathop \smallint \nolimits_{{\lambda _1}}^{{\lambda _2}} A(\lambda )QE(\lambda ){T_i}(\lambda )X(\lambda )d\lambda = \sum A(\lambda )QE(\lambda ){T_i}(\lambda )X(\lambda ).$$

 figure: Fig. 1.

Fig. 1. Deviation calibration strategies for micro-spectrometer. (a) The workflow of the meta-learning assisted micro-spectrometer system. (b) The manufacturing deviations for the design of filters, λ1 and TiS represent the simulated transmittance (function) of an FP-filter, λ2 and TiE represent the experimental transmittance of an FP-filter. (c) The process of calibration data acquisition. (d) The reconstruction process with different algorithms. (e) The evaluation of timeliness and cost of different calibration strategies.

Download Full Size | PDF

The reconstruction results are evaluated using the MSE as the metric, and Eq. (3) defines the MSE for a single spectrum:

$$MSE = \frac{1}{S}\|{X_R} - {X_{GT}}\|_{2^2},$$
where XR and XGT represent the reconstructed spectrum and ground truth spectrum, respectively, and S is the length of XR and XGT. The performance of the algorithm is also evaluated by MSE, $average{\; }MSE = \frac{1}{N}\mathop \sum \nolimits_{k = 1}^N MSE$, in which N is the amount of data in the dataset.

The experimental transmittance of the FP filter TiE usually deviates from the simulated result TiS due to limited manufacturing accuracy, as shown in Fig. 1(b). The typical solution is measuring the transmittance of each pixel or large amounts of standard samples, which is expensive and time-consuming. As shown in Fig. 1(c), if an iterative algorithm is employed, the iteration process of each spectrum lasts at least for minutes, which is not suitable for real-time applications. When a typical data-driven algorithm is employed, the measurement of experimental transmittance or large amounts of standard samples is also necessary. An alternative solution leverages meta-learning strategy developed in artificial intelligence as illustrated in Fig. 1(b) and Fig. 10(a), several simulated transmittance datasets related to different manufacturing deviations are achieved through simulation software. Thereafter, different simulated photocurrent datasets are generated with the transmittance datasets. These photocurrent datasets are used to meta-train the neural network to obtain hyperparameters considering all kinds of manufacturing deviations. In the following procedure, the photocurrent of a color chart is collected with the spectrometer . With these photocurrent data, the neural network would be calibrated rapidly to adapt to spectrometer with deviation. The MSE is comparable to the other calibration strategies utilizing the experimental encoding function dataset while the efficiency is the best.

3. Simulation and device fabrication

First, we theoretically analyzed the influence of uniformity during manufacturing. Wafer uniformity is a key indicator of the process. All the nanofabrication equipment utilized for manufacturing the FP filter array would encounter processing non-uniformity, including the photolithography equipment (SUSS MJB4), the deposition equipment (OTFC-1300), and the etching equipment (Trion Minilock-Phantom III). Figure 2(a) shows the schematic of the in-wafer process uniformity of an etching equipment. Max is the maximum value measured on the wafer, while min indicates the minimum value measured on the wafer. The FP filter contains alternating low-refractive SiO2 and high-refractive TiO2. Narrow-band filtering with different center wavelengths can be achieved by adjusting the thickness of the intermediate SiO2. The simulation transmittance of the 16 FP filter types is described in Fig. 6(a).

 figure: Fig. 2.

Fig. 2. The evaluation of the manufacturing deviation in the filter array. (a) The schematic diagram of in-wafer uniformity of a 4-inch etching equipment. (b) Relationship between intermediate SiO2 thickness versus center wavelength of FP filter, gray area is the range of center wavelength shift after considering 5% uniformity deviation of processing equipment. (c) The simulated transmittance considering 5% thickness deviation for 569 nm peak wavelength filter. (d) The SEM image of two fabricated FP filter. (e) The transmittance of the filter array sample A measured by a UV-visible near-infrared micro-spectrophotometer. (f) The peak position discreteness for each type of filters at different positions within the 20 × 20 mm-sample A.

Download Full Size | PDF

Then we conduct simulations for intermediate SiO2 layer with deviation up to 5%, while keeping a constant thickness for other layers. As depicted in Fig. 2(b), the center wavelengths exhibited various degrees of shifting up to ±10 nm. The transmittance curve of one type is drawn in Fig. 2(c). The design center wavelength is 569 nm. It shifts to 562 nm when the thickness deviation is minus 5% and to 577 nm at a positive 5% thickness deviation. The envelope shape and the transmittance intensity are also influenced by the thickness. Figure 2(d) shows the scanning electron microscope (SEM) image of two filter type (Manufactured by OTFC-1300), confirming the alternating low-refractive dielectric and high-refractive dielectric structure. The specific preparation process of the 16 FP filter types is described in Fig. 6(b) and (c). Figure 2(e) shows the measured transmittance results of filter sample A using a UV-visible near-infrared micro-spectrophotometer [CRAIC 20/30PV]. The transmittance is Gaussian-like. The interval of each peak is about 10 nm. The FWMH of the transmittance varies from 8 nm to 10 nm. The statistic of the peak deviation from the simulation result in Fig. 6(a) in the 20 mm square sample A is concluded in Fig. 2(f). The transmittance peak of one type at different positions varies with each other up to 4 nm, which has a serious impact on the reconstruction result.

The schematic structure of the micro-spectrometer is illustrated in Fig. 3(a). The micro-spectrometer consists of a 2K × 2 K CMOS sensor (ams CMV4000) with 5.5 µm-pixel-pitch and a 16-types FP filter array. Each type is an FP cavity with a different center wavelength design, as shown in Fig. 6(a). The unit filter is a square with a side length of 11 µm, covering 2 × 2 CMOS pixels as shown in Fig. 3(a). The period of the filter array is 4 × 4. Figure 3(b) shows the photo of the micro-spectrometer after bonding. The bonding process for the detector and filter arrays is described in Fig. 7. The micro-spectrometer is encapsuled in vacuum with a glass window before connected the signal readout circuit. The completed micro-spectrometer as a module is shown in Fig. 3(c). Figure 3(d) shows the normalized quantum efficiency of the original CMOS detector and the encoding function ${T_i}(\lambda )\times QE(\lambda )$ of different pixels after being integrated with the filter array sample B. The interval of the peak is also ∼10 nm. But the measured results deviate from the simulation results both in peak position and intensity as shown in the picture. The position of the peaks deviates from the simulation result up to 6 nm. The encoding function curves have secondary diffraction peaks, which exacerbates the reconstruction errors. We propose a deep residual network model consisting of 16 residual blocks for spectrum reconstruction. The detailed information is given in Fig. 8. The original spectra dataset contains 7200 spectra augmented by different data preprocessing methods as depicted in Fig. 9. Different photocurrent datasets are generated using these spectra and simulated transmittance functions related to different thickness deviations as shown in Fig. 10(a). Then those photocurrent datasets are used to meta-train the network with a meta-learning method to achieve a meta-trained network. Afterward, the meta-trained network is calibrated by a few shots of calibration data to optimize for a spectrometer with deviation. The simulation of the calibration effect is displayed in Fig. 3(e) and (f). The simulated calibration dataset contains 50 data. Figure 3(e) shows the relationship between the calibration (training) epochs and the MSE on the test samples. It can be seen that the meta-learning method exhibits lower MSE on the test samples than transfer learning and meta-learning requires less calibration epochs to obtain stable MSE than transfer learning (The detail of transfer learning method is shown in Fig. 10(b)). The average MSE decreases to 9.9 × 10−4 with only twenty epochs for the meta-learning method, while the MSE of the transfer learning maintains at 2.4 × 10−3 after 50 epochs. Notably, the time for calibration training only takes a few minutes. Additionally, we investigate the influence of the calibration data count and the light source. The results are presented in Fig. 3(f). The reconstructed average MSE can be decreased from 7.2 × 10−3 to 2.2 × 10−3 with only 10 calibration data, while it further decreases to 7.8 × 10−4 when 50 calibration data, are used. These simulation results prove the superiority of our calibration strategy and provide a lot of guidance for subsequent experiments.

 figure: Fig. 3.

Fig. 3. The schematic of the micro-spectrometer and the simulation of the calibration effect. (a) The schematic of the micro-spectrometer, each filter is a square with 11 µm side length and integrated by a gluing process with a CMOS detector with 5.5 µm-pixel-pitch. (b) The photo of the micro-spectrometer encapsuled beneath a glass window. (c) The photo of the finished micro-spectrometer module. (d) Solid line: measured encoding function ${T_i}(\lambda )\times QE(\lambda )$ of different pixels after gluing with filter array, dotted line: simulated encoding function ${T_i}(\lambda )\times QE(\lambda )$ of different pixels after gluing with filter array, dashed line: the normalized quantum efficiency of the CMOS detector. (e) The simulated calibration effect on the meta-learning method and the transfer learning method, metrics: average MSE of the test dataset after different calibration epoch iterations. (f) Relationship between the calibration data count and the final average MSE on the test dataset in meta-learning method.

Download Full Size | PDF

 figure: Fig. 4.

Fig. 4. Experimental calibration effect of meta-learning method. (a) The optical path for reflectance testing of the standard color chart by the proposed micro-spectrometer. (b) The photo of the standard color chart, the triangular symbols represent fine-tuned training samples and the penta-gram symbols represent test samples. (c) Reflectance spectra measured and reconstructed by spectrometers with meta-trained network. (d) Reflectance spectra measured and reconstructed by spectrometers with calibrated network.

Download Full Size | PDF

The generalization of the calibration effect is evaluated with 42 unknown reflectance spectral data from USGS [28] in Fig. 5. The neural network is first meta-trained with simulated encoding functions. Subsequently, we measured the real encoding functions to calculate the photocurrents for calibration step. Totally 24 reflectance spectral datasets from USGS were used for calibration and 18 for testing. The reconstructed results of diopside, plastic pipe and almandine are shown in Fig. 5. Figure 5(a) shows the reconstructed result of the meta-trained network. The reconstructed results do not match well with the ground truth with average MSE of 7.2 × 10−3 on test dataset. After the spectrometer calibrated with 24 calibration data, the average MSE decreases to 9.4 × 10−4. The reconstructed result is significantly improved compared with the result before calibration. The calibration effect is also compared with the traditional calibration strategy by measuring the spectral function of the filters. The results of the traditional strategy are shown in Fig. 5(c). The average MSE on the test dataset is 4.5 × 10−4. Though the MSE of our strategy is slightly bigger, all the features in the spectra are all revert. The difference is not obvious in practical usage.

 figure: Fig. 5.

Fig. 5. Generalization of the proposed calibration strategy. (a) Solid line: The ground truth of three different chemistries, dashed line: reconstructed results the meta-trained neural network. (b) Solid line: The ground truth of three different chemistries, dashed line: reconstructed results with the calibrated neural network. (c) Solid line: The ground truth of three different chemistries, dashed line: the recon-structed results of a neural network trained with the measured spectral function of the filters.

Download Full Size | PDF

4. Experimental performance evaluation

The reflectance spectrum of the standard color chart is used as the calibration data in the calibration experiment. The measurement setup is shown in Fig. 4(a). When measuring the reflectance of the color chart, a halogen lamp light source (OSRAM 64440S 12V50W) was utilized to emit a uniform light spot covering the range of 550 to 720 nm, which closely matches sunlight. The light is reflected by the color chart before collected by the camera. As shown in Fig. 4(b), eighteen areas with triangular and pentagram symbols on the color chart were measured. A focus lens with 25 mm focal length was mounted before the micro-spectrometer to collect the reflectance light. Once the photocurrent I was collected by the spectrometer, it served as an input into the meta-trained network to reconstruct the original spectrum. Since the network was not optimized for this spectrometer, the reconstruction results did not match the ground truth very well, as shown in Fig. 4(c). The meta-trained network was then calibrated for 50 epochs using 15 of the 18 samples. After a calibration process, the rest 3 of 18 samples were fed into the neural network to reconstruct the reflectance of the color chart. The results shown in Fig. 4(d) are in good agreement with the standard reflectance spectrum. The average MSE decrease from 7.4 × 10−3 to 1.2 × 10−3.

From the above results, we can conclude that the meta-learning method is effective in micro-spectrometer calibration. More importantly, this strategy is free of measuring the spectrum of the encoding structure or large amounts of standard samples. It becomes feasible for users to in-situ calibrate the spectrometer. This is very useful for many application scenarios. Because the performance of micro-spectroscopy is affected by many factors except the manufacturing process, such as temperature and humidity, regular calibration is necessary. Compared to the monochromator in the laboratory, the color chart is a more convenient tool. The more calibration data, the better the calibration effect. To achieve better calibration result, we can utilize color chart with more colors and a light source equipped with a homogenizer. The price of these accessories is hundreds of dollars at most, which is negligible compared to the current price of a micro-spectrometer (more than thousands of dollars).

5. Conclusion

In summary, we developed an in-situ calibration strategy for micro-spectrometers with deviation. This strategy is based on the meta-learning method free of measured spectral function of encoding structure. A priori information about deviation is incorporated into the simulation dataset in the meta-training stage. In the calibration stage, a few shots of calibration data are used to optimize the algorithm to adapt to the micro-spectrometer with deviation quickly. We verified our strategy on a micro-spectrometer composed of a CMOS array and an FP filter array. The FP filter array contains 16 types of FP filters with a center wavelength interval of 10 nm, covering the spectral range of 550-720 nm. The FWHM of each filter ranges from 8-10 nm. Due to limited semiconductor manufacturing accuracy, the deviation of the filters’ center wavelength is inevitable. The maximum deviation from the design result within a 2-centimeter sample is 4 nm, while the deviation in a 4-inch wafer would increase to 10 nm. Consequently, the average MSE on the test dataset rises from 4.5 × 10−4 to 7.2 × 10−3. By calibration through reflectance measurements of color chart instead of a monochromator, the calibration cost decreased to one percent of the traditional procedure. With 15 color measurements, the average reconstruction error on the test dataset decreased from 7.2 × 10−3 to 1.2 × 10−3, and further decreased to 9.4 × 10−4 when the calibration data is increased to 24. The calibration process only takes a few minutes. The reconstruction results of 42 unknown spectra verify the good generalization effect after calibration. In general, the meta-learning method significantly improves the timeliness and cost of the calibration process. Therefore, our work facilitates the large-scale production and application of computational micro-spectrometers.

Appendix I Fabrication details

Multi-channel FP filter arrays are fabricated on a silicon substrate using a Bragg reflector structure (DBR) of alternating silicon dioxide and titanium dioxide layers. The specific manufacturing method is as follows:

  • 1) Determine the material and thickness of each layer with simulation software, as shown in Fig. 6(a).
  • 2) Employ ion beam-assisted electron beam evaporation coating to deposit a layer of silicon dioxide followed by a layer of titanium dioxide on the substrate, alternating these layers to form the bottom DBR. During the deposition process, monitor the thickness of SiO2 and TiO2.
  • 3) Use ion beam-assisted electron beam evaporation coating to deposit an intermediate SiO2 layer on top of the bottom DBR to form the FP cavity layer. During the deposition process, monitor the thickness of SiO2.
  • 4) As shown in Fig. 6(b), to form a 16-channel FP filter, employ inductively coupled plasma (ICP) etching on the FP cavity layer after lithography. After etching, remove the photoresist. By repeating these procedures 16 times, sixteen FP cavities of varying thickness are created.
  • 5) Place the etched sample into a vacuum coating apparatus and continue to deposit the top DBR using the same process parameters as in step 2. The order of depositing SiO2 and TiO2 should be symmetric to that in step 1. During the deposition process, monitor the thickness of SiO2 and TiO2. The FP filter array is completed after the top DBR deposition, as shown in Fig. 6(c).

 figure: Fig. 6.

Fig. 6. Multi-channel FP filter array fabrication method. (a) Simulation transmission spectra of 16 FP cavities. (b) Diagram illustrating the fabrication of FP cavities with varying thicknesses. (c) Schematic of FP cavity filter array final product.

Download Full Size | PDF

The precision bonding of the filter array with the CMOS image sensor is a crucial step. First, the surfaces of the filter array and the CMOS image sensor must be thoroughly cleaned. Once cleaning is complete, the CMOS image sensor is precisely positioned on the chip bonding machine. Next, a specialized bonding glue is evenly applied to the corners of the CMOS image sensor. Subsequently, the filter array is aligned with the CMOS image sensor, which has been applied with glue, to continue bonding process. Finally, the bonding glue is cured by UV light, as shown in Fig. 7.

 figure: Fig. 7.

Fig. 7. The schematic drawing of the bonding process.

Download Full Size | PDF

Appendix II Reconstruction network details

The spectral reconstructions were performed on a server computer with Intel Xeon Platinum 8280 CPU and two NVIDIA RTX 3090 graphics processing card. Figure 8 shows the proposed deep residual convolutional network structure, the network consists of 16 residual convolution layers for feature extraction (Each residual convolutional layer consists of a residual block multiplied by a scaling factor of 0.1. The number of channels is 64. The kernel size is 5), linear interpolation up-sampling layer for interpolating the signal and a linear fully connected layer for adjusting the dimensionality of the output.

 figure: Fig. 8.

Fig. 8. The proposed deep residual convolutional network structure.

Download Full Size | PDF

We trained the network with the Adam optimizer. Batch size, epochs and initial learning rate are set as 4, 300, and 0.0001, respectively. The learning rate was adjusted to 0.9 times of the original rate every 10 rounds of training.

Appendix III Data preprocessing

The original 600 spectra data consist of transmission spectra in the LWIR, MWIR from SDBS (Spectral Database for Organic Compounds) [29], THz spectra from THz database [30] and reflectance spectra provided in ENVI5.3. The operating range of the designed spectrometer is 550-720 nm. The spectrum step is 1 nm. Therefore, the spectral data are preprocessed to 171 data points as training labels. We also performed data augmentation, as shown in Fig. 9. A spectrum is split from the middle and sampled into two spectra by double interpolation. Then, the three spectra are augmented by mirroring up and down or right and left. Finally, one spectrum can be augmented as 12 spectra. The photocurrent is calculated by Eq. (2) in the main text.

 figure: Fig. 9.

Fig. 9. The data augmentation process. First, the spectrum is split in the middle and sampled into two spectra by double interpolation. The three spectra obtained are augmented by the flipping strategy.

Download Full Size | PDF

Appendix IV Calibration algorithms

Figure 10(a) shows the training process of meta-learning, several photocurrent datasets served as multi-tasks were generated with different encoding functions indicating manufacturing errors. The network model is first trained using those tasks to find an initial model hyperparameter θM that is more adaptive to new tasks from random θ0. The next step is a fine-tuning step, where few-shot experimental data are collected to adapt the neural network to a practical spectrometer, about 50 iterations are required for the fine-tuning step. The resulting network model is used for the final reconstruction test.

 figure: Fig. 10.

Fig. 10. The details of calibration algorithms. (a) The training process of meta-learning. (b) The training process of transfer learning.

Download Full Size | PDF

Figure 10(b) shows the training process of transfer learning method for comparison, firstly, the designed encoding function without considering manufacturing errors is directly utilized to obtain NN θM. Subsequently, the few-shot experimental data obtained from the sample with error is used for the transfer learning training. At this stage, only the parameters of the up-sampling and the linear fully-connected layer can be updated in the structure of the network.

Funding

National Key Research and Development Program of China (2023YFB2806703); National Natural Science Foundation of China (62175045, 11933006, U2141240); Hangzhou Science and Technology Bureau (TD2020002); National Key Laboratory Foundation of China (SKLIP2021006); Research Funds of Hangzhou Institute for Advanced Study (2022ZZ01007, B02006C019019).

Acknowledgment

The author thanks Changchun Changguang Chen Spectrum Technology Co., Ltd. for the assistance in terms of hardware and equipment. Thanks to Dr. Yuan Cheng and Dr. Pei Sheng from Westlake University for measurement support.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. E. L. Izake, “Forensic and homeland security applications of modern portable Raman spectroscopy,” Forensic Sci. Int. 202(1-3), 1–8 (2010). [CrossRef]  

2. C. P. Bacon, Y. Mattley, and R. Defrece, “Miniature spectroscopic instrumentation: Applications to biology and chemistry,” Rev. Sci. Instrum. 75(1), 1–16 (2004). [CrossRef]  

3. S. Kim, D. Cho, J. Kim, et al., “Smartphone-based multispectral imaging: system development and potential for mobile skin diagnosis,” Biomed. Opt. Express 7(12), 5294 (2016). [CrossRef]  

4. C. Li, Z. Wang, R. Xu, et al., “The Scientific Information Model of Chang'e-4 Visible and Near-IR Imaging Spectrometer (VNIS) and In-Flight Verification,” Sensors 19(12), 2806 (2019). [CrossRef]  

5. V. Protopopov, Practical Opto-Electronics: An Illustrated Guide for the Laboratory (2014), pp 253–308.

6. A. Li, C. Yao, J. Xia, et al., “Advances in cost-effective integrated spectrometers,” Light: Sci. Appl. 11(1), 174 (2022). [CrossRef]  

7. J. Malinen, A. Rissanen, H. Saari, et al., “Advances in miniature spectrometer and sensor development,” Proc. SPIE 9101, 91010C (2014). [CrossRef]  

8. J. Huang, Q. Wen, Q. Nie, et al., “Miniaturized NIR spectrometer based on novel MOEMS scanning tilted grating,” Micromachines 9(10), 478 (2018). [CrossRef]  

9. M. Erfan, Y. M. Sabry, M. Sakr, et al., “On-chip micro-electro-mechanical system Fourier transform infrared (MEMS FT-IR) spectrometer-based gas sensing,” Appl Spectrosc 70(5), 897–904 (2016). [CrossRef]  

10. S. Yuan, C. Ma, E. Fetaya, et al., “Geometric deep optical sensing,” Science 379(6637), eade1220 (2023). [CrossRef]  

11. J. Lee, Y. Park, H. Kim, et al., “Compact meta-spectral image sensor for mobile applications,” Nanophotonics 11(11), 2563–2569 (2022). [CrossRef]  

12. M. Liu, M. Yang, J. Zhu, et al., “High-sensitivity computational miniaturized terahertz spectrometer using a plasmonic filter array and a modified multilayer residual CNN,” Nanophotonics 12(23), 4375–4385 (2023). [CrossRef]  

13. L. Huang, R. Luo, X. Liu, et al., “Spectral imaging with deep learning,” Light: Sci. Appl. 11(1), 61 (2022). [CrossRef]  

14. C. Brown, A. Goncharov, Z. S. Ballard, et al., “Neural network-based on-chip spectroscopy using a scalable plasmonic encoder,” ACS Nano 15(4), 6305–6315 (2021). [CrossRef]  

15. J. Zhang, X. Zhu, and J. Bao, “Solver-informed neural networks for spectrum reconstruction of colloidal quantum dot spectrometers,” Opt. Express 28(22), 33656–33672 (2020). [CrossRef]  

16. L. Gao, Y. Qu, L. Wang, et al., “Computational spectrometers enabled by nanophotonics and deep learning,” Nanophotonics 11(11), 2507–2529 (2022). [CrossRef]  

17. J. Zhu, J. Liu, H. Zhu, et al., “General-purpose mid-infrared micro-spectrometer based on hierarchical residual CNN and data augmentation,” Opt. Express 31(10), 16974–16984 (2023). [CrossRef]  

18. W. Zhang, H. Song, X. He, et al., “Deeply learned broadband encoding stochastic hyperspectral imaging,” Light: Sci. Appl. 10(1), 108 (2021). [CrossRef]  

19. Z. Wang, S. Yi, A. Chen, et al., “Single-shot on-chip spectral sensors based on photonic crystal slabs,” Nat. Commun. 10(1), 1020 (2019). [CrossRef]  

20. Z. Wang and Z. Yu, “Spectral analysis based on compressive sensing in nanophotonic structures,” Opt. Express 22(21), 25608–25614 (2014). [CrossRef]  

21. H. Hillmer, C. Woidt, A. Istock, et al., “Role of nanoimprint lithography for strongly miniaturized optical spectrometers,” Nanomaterials 11(1), 164 (2021). [CrossRef]  

22. Z. Ma, W. Liu, W. Li, et al., “Additive manufacturing of functional gradient materials: A review of research progress and challenges,” J. Alloys Compd. 971, 172642 (2023). [CrossRef]  

23. R. Zheng, Y. Wei, Z. C. Zhang, et al., “Stimuli-responsive active materials for dynamic control of light field,” Responsive Materials 1(2), e20230017 (2023). [CrossRef]  

24. M. H. Bitarafan and R. G. DeCorby, “On-chip high-finesse Fabry-Perot microcavities for optical sensing and quantum information,” Sensors 17(8), 1748 (2017). [CrossRef]  

25. T. Hospedales, A. Antoniou, P. Micaelli, et al., “Meta-learning in neural networks: A survey,” IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 5149–5169 (2022). [CrossRef]  

26. J. Xiong, X. Cai, K. Cui, et al., “Dynamic brain spectrum acquired by a real-time ultraspectral imaging chip with reconfigurable metasurfaces,” Optica 9(5), 461–468 (2022). [CrossRef]  

27. S. Rinku and R. J. Curry, “Lead sulphide nanocrystal photodetector technologies,” Nat. Photonics 10(2), 81–92 (2016). [CrossRef]  

28. R. Kokaly, R. Clark, G. Swayze, et al., “USGS spectral library version 7 data: US Geological Survey data release,” United States Geological Survey (USGS) (2017), https://www.usgs.gov/data/usgs-spectral-library-version-7-data.

29. T. Yamazaki, M. Nabeshima, T. Saito, et al., “Spectral Database for Organic Compounds SDBS,” AIST(2015), https://sdbs.db.aist.go.jp.

30. “Terahertz Database,” thzdb.org, http://www.thzdb.org.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1.
Fig. 1. Deviation calibration strategies for micro-spectrometer. (a) The workflow of the meta-learning assisted micro-spectrometer system. (b) The manufacturing deviations for the design of filters, λ1 and TiS represent the simulated transmittance (function) of an FP-filter, λ2 and TiE represent the experimental transmittance of an FP-filter. (c) The process of calibration data acquisition. (d) The reconstruction process with different algorithms. (e) The evaluation of timeliness and cost of different calibration strategies.
Fig. 2.
Fig. 2. The evaluation of the manufacturing deviation in the filter array. (a) The schematic diagram of in-wafer uniformity of a 4-inch etching equipment. (b) Relationship between intermediate SiO2 thickness versus center wavelength of FP filter, gray area is the range of center wavelength shift after considering 5% uniformity deviation of processing equipment. (c) The simulated transmittance considering 5% thickness deviation for 569 nm peak wavelength filter. (d) The SEM image of two fabricated FP filter. (e) The transmittance of the filter array sample A measured by a UV-visible near-infrared micro-spectrophotometer. (f) The peak position discreteness for each type of filters at different positions within the 20 × 20 mm-sample A.
Fig. 3.
Fig. 3. The schematic of the micro-spectrometer and the simulation of the calibration effect. (a) The schematic of the micro-spectrometer, each filter is a square with 11 µm side length and integrated by a gluing process with a CMOS detector with 5.5 µm-pixel-pitch. (b) The photo of the micro-spectrometer encapsuled beneath a glass window. (c) The photo of the finished micro-spectrometer module. (d) Solid line: measured encoding function ${T_i}(\lambda )\times QE(\lambda )$ of different pixels after gluing with filter array, dotted line: simulated encoding function ${T_i}(\lambda )\times QE(\lambda )$ of different pixels after gluing with filter array, dashed line: the normalized quantum efficiency of the CMOS detector. (e) The simulated calibration effect on the meta-learning method and the transfer learning method, metrics: average MSE of the test dataset after different calibration epoch iterations. (f) Relationship between the calibration data count and the final average MSE on the test dataset in meta-learning method.
Fig. 4.
Fig. 4. Experimental calibration effect of meta-learning method. (a) The optical path for reflectance testing of the standard color chart by the proposed micro-spectrometer. (b) The photo of the standard color chart, the triangular symbols represent fine-tuned training samples and the penta-gram symbols represent test samples. (c) Reflectance spectra measured and reconstructed by spectrometers with meta-trained network. (d) Reflectance spectra measured and reconstructed by spectrometers with calibrated network.
Fig. 5.
Fig. 5. Generalization of the proposed calibration strategy. (a) Solid line: The ground truth of three different chemistries, dashed line: reconstructed results the meta-trained neural network. (b) Solid line: The ground truth of three different chemistries, dashed line: reconstructed results with the calibrated neural network. (c) Solid line: The ground truth of three different chemistries, dashed line: the recon-structed results of a neural network trained with the measured spectral function of the filters.
Fig. 6.
Fig. 6. Multi-channel FP filter array fabrication method. (a) Simulation transmission spectra of 16 FP cavities. (b) Diagram illustrating the fabrication of FP cavities with varying thicknesses. (c) Schematic of FP cavity filter array final product.
Fig. 7.
Fig. 7. The schematic drawing of the bonding process.
Fig. 8.
Fig. 8. The proposed deep residual convolutional network structure.
Fig. 9.
Fig. 9. The data augmentation process. First, the spectrum is split in the middle and sampled into two spectra by double interpolation. The three spectra obtained are augmented by the flipping strategy.
Fig. 10.
Fig. 10. The details of calibration algorithms. (a) The training process of meta-learning. (b) The training process of transfer learning.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

I i = λ 1 λ 2 λ q h c Q E ( λ ) P ( λ ) T i ( λ ) X ( λ ) d λ ,
I i = λ 1 λ 2 A ( λ ) Q E ( λ ) T i ( λ ) X ( λ ) d λ = A ( λ ) Q E ( λ ) T i ( λ ) X ( λ ) .
M S E = 1 S X R X G T 2 2 ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.