We introduce a compressive sensing based approach for single pixel hyperspectral chemical imaging in a broad spectral range in the near-infrared. Fully integrated MEMS based Fabry-Pérot tunable filter spectrometers and a digital micro-mirror device were employed to achieve spectral and spatial resolution, respectively. The available spectral range from 1500 to 2200 nm covers molecular overtone vibrations enabling chemical identification. Hyperspectral images of different adhesives deposited on a textile were recorded revealing their chemical composition. Furthermore, spectrally resolved near-infrared images with compression rates up to 90% are presented. The approach of single pixel imaging illustrates a promising technology for the infrared spectral range superior to conventionally used costly focal plane arrays.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Compressive Sensing (CS) [1,2], also known as sparse sampling, paved the way for new data acquisition schemes in various fields such as photonics, communications and networks or atomic force microscopy [3–6]. CS basically enables the reconstruction of sparse signals from measurements with less samples than postulated by the Shannon-Nyquist theorem. A particularly elegant utilization of CS is its combination with the matured technology of digital micro-mirror devices (DMD). This combination led to a method for imaging with a single element detector, known as single pixel imaging (SPI) [7,8]. In recent years several applications and improvements of this technology in the visible spectral range were shown, e.g. for SNR enhancement , real-time imaging (frame rates up to 10 Hz) [10,11] or higher compression rates [12,13].
However, the main benefit of SPI, namely replacing a costly focal plane array (FPA) becomes only effective in the infrared spectral region. While FPAs are low-cost in the visible spectral range, those devices become expensive in the upper near-infrared (NIR) and even more expensive in the mid-infrared (MIR) spectral region. Furthermore, DMDs offer much higher pixel numbers than infrared FPAs and single pixel detectors offer superior detectivity. Recently, these advantages of SPI were exploited for low-cost near-infrared (800-1700 nm) thermographic microscopy . Nevertheless, up to now, hyperspectral SPI was only executed from the visible range  up to approximately 1100 nm . Shibuya et al. showed hyperspectral SPI at 1550 nm based on frequency combs with the small spectral bandwidth of ca. 16 nm . However, chemical identification in a hyperspectral imaging approach would require broadband spectral coverage, ideally covering higher wavelength regions in the upper NIR and even MIR spectral range [18,19].
In this study, we present a significant extension of the spectral coverage for hyperspectral SPI up to a wavelength of 2200 nm thereby gaining access to molecular-specific vibrational absorption bands. At the same time the size of used components was decreased further. This was achieved by employing a DMD for spatial encoding in combination with a fully-integrated spectrally selective MEMS based Fabry-Pérot tunable filter (FPTF) with built-in single element InGaAs-detector. Such FPTFs are the enabling technology for low-cost and robust miniature spectrometers (also known as “microspectrometers”), which are continuously replacing the standard FTIR spectrometers in field applications. In this contribution we are introducing MEMS based microspectrometer technology for SPI. We demonstrate for the first time that FPTFs are ideally suited for hyperspectral SPI as they provide high throughput, a large aperture size and small overall size. The performance of this approach was characterized. In particular, the employed CS reconstruction method was validated, and hyperspectral images of different samples, including adhesives on a textile surface were recorded. Chemical classification of the obtained hyperspectral images was executed by means of principle component analysis (PCA).
2. Materials and methods
CS allows to sample data in a compressed way under the premise that the respective data is sparse or at least sparse in some domain, which is typically valid for natural images . Spatial encoding of the scene is achieved by applying patterns of a sampling basis via a DMD according to20]. Each column is reshaped into an array of the lengthto be displayed on the DMD. The amount of displayed patterns can be reduced to a fraction M of the total pixel number. The application of semi-Hadamard matrices delivers computational speedup  and thermal drifts can be neglected due to differential detection of inverse pattern pairs. By exploiting the sparsity of discrete cosine transformed images LASSO optimization  is used to reconstruct the 2D image of the scene.
The developed imaging system (Fig. 1) operates in diffuse reflection mode. A broadband halogen lamp with parabolic reflector illuminates the sample and the objective lens creates the image of the object in the DMD plane, resulting in a field of view (FOV) of 25 mm by 25 mm. An optical system, consisting of a telescope (10 × demagnification) and an additional lens, was used to focus light on the detector in order to maximize the throughput of the signal at the aperture of the detector (Ø 0.3 mm). The spatial resolution of this system can be calculated from the FOV (25 mm) and the amount of effective pixels, being the pixels displayed on the DMD, resulting in approximately 0.4 mm for 64 pixels.
Spectral detection was executed by commercially available MEMS-based FPTF spectrometers operating in the NIR spectral range (1450 nm – 1850 nm, Hamamatsu C13272-02 and 1750 nm – 2200 nm, C14273; both with InGaAs detector). The spectral resolution was approximately 22 nm and the time constant 1 ms. Spectral cross-talk between the pixels of the final image can be excluded, as a fixed set of Hadamard patterns was applied at each filter position of the tunable filter spectrometer. The DMD (Texas Instruments DLP4500, 912 × 1140 pixels) was controlled by using it as second monitor (via HDMI connection). For each Hadamard pattern displayed 3 frames were acquired. This resulted in an acquisition time of less than 4 minutes at a compression rate of 50% (for a single image at a specific wavelength with 2048 patterns plus inverse patterns applied, indicating a resolution of 64 by 64 pixels).
Hyperspectral images of three samples were recorded. First, images of a graphic printed with black toner on white paper (standard office paper) were collected. The second sample was created by drawing two different markers (edding 360, edding 3000; labelled as marker 1 and marker 2) on white paper. The third sample consisted of four different adhesives deposited on a green colored textile (65% polyester, 35% cotton). The adhesives were labelled from 1 to 4 corresponding to following brand names: ‘Loctite Power Flex’ (ethyl-2-cyanoacrylate), ‘UHU Alleskleber’ (ethanol, acetone), ‘Pattex Kraft Mix’ (epoxy) and ‘RS Superglue Gel’ (ethyl-2-cyanoacrylate).
The pixel values of the hyperspectral images shown in this work are calculated as absorbance normalized to a background recording. Preprocessing and statistical treatment of the absorption spectra, that were obtained pixel-wise from the data cube, was carried out in Orange 3.16  by executing smoothing (Savitzky-Golay filter, window size: 15, polynomial order: 2), baseline correction (rubber band method) and standard normal variate (SNV) normalization. PCA was applied to mean centered data from representative pixels of each adhesive type and background. Logistic regression using a L2-norm (i.e. ridge) penalty was used to fit a classification model on the previously selected pixels. Optimization of the ridge parameter was achieved through 10-fold cross-validation. The final model was subsequently employed to classify the remaining pixels in the hyperspectral image. The dynamic range (DR) of the setup was calculated by23]). For the evaluation of the reconstruction quality a zero compression measurement at a single wavelength was executed. Subsequently, the columns of the full Hadamard matrix were reassembled according to their corresponding signal magnitude indicated by the absolute signal value normalized by the maximum, denoted as Norm(S). The PSNR of the compressed images compared to the fully reconstructed image (zero compression) was calculated according toEq. (3), P0,i denotes the ith pixel of the zero compression image and Pr,i stands for the ith pixel of the reconstructed image at a certain compression rate.
The calculation of the dynamic range yields 43 dB at zero compression and 40 dB at 90% compression at a wavelength of 2190 nm. In Fig. 2 the reconstructed images with different compression rates are shown. Using only ten percent of the most contributing patterns results in an already well recognizable image with a PSNR of 23 dB. For specific applications, the exclusive selection of the most contributing patterns of the Hadamard-set will strongly reduce acquisition times. Examples of such applications would be repeated measurements of similar samples (e.g. process monitoring in production lines) or measurements of samples changing their chemical composition over time, while preserving spatial distribution.
NIR spectroscopy has the potential to resolve structures not perceivable in the visible range (VIS). This characteristic is illustrated (Fig. 3) by imaging a sample consisting of the letter ‘R’ written with marker 1 and subsequently overpainted by marker 2. While the photograph in the visible range, Fig. 3(b), cannot resolve the two types of black markers, the single pixel image in Fig. 3(c) (λ = 1820 nm, 64 by 64 pixels, 50% compression) reveals the different absorption characteristics of the two respective markers. In contrast to thermal imaging cameras, which are integrating over a broad spectral range, hyperspectral SPI allows selective analysis of certain spectral positions at which the desired chemical information is expected.
In order to show the benefit of exploiting the full spectral information by hyperspectral SPI in the NIR spectral range, adhesive samples on a textile (Fig. 4(c)) were measured at 60 different wavelengths between 1560 and 1835 nm. The single-pixel images with 64 by 64 pixels were recorded with 50% compression. Representative absorption spectra of each adhesive were extracted pixel-wise and plotted in Fig. 4(a), showing high quality of spectra recorded with the MEMS-FPTF (spectral resolution = 22 nm). The observed absorption bands correspond to the overtone stretching vibrations of C-H. These bands slightly differ for the used adhesives because of their particular chemical compositions. We performed PCA analysis, shown in Fig. 4(b), and fitted a logistic regression model on the previously selected pixels in order to generate a false color image. This classification results in a clear differentiation between the four adhesives, presented in Fig. 4(d). At the margins of the adhesive stains, some pixels were misclassified indicating the influence of the surface texture at the edges. In Figs. 4(e) and 4(f) the absorption images at 1765 nm and 1605 nm respectively, are shown. The pixel values of the different adhesives correspond well to the shown absorption spectra, as at 1605 nm adhesive 3 exhibits the highest absorption and at 1765 nm adhesive 2 exhibits the lowest absorption.
4. Conclusions and outlook
In summary, a broadband NIR hyperspectral single pixel imaging setup able to perform measurements up to 2200 nm using fully-integrated MEMS-based FPTF spectrometers has been developed and characterized. By means of compressive sensing, images with compression rates of up to 90 percent have been obtained. The developed setup enabled spectrally and thereby chemically resolved imaging. Experimental verification was performed by resolving an image consisting of two chemically different markers. Furthermore, PCA-based classification of recorded hyperspectral images lead to a differentiation of four different adhesives deposited on a textile. The achieved acquisition time per image was less than 4 minutes. However, as shown in [24,25] there is a significant potential for a further reduction of the acquisition time by applying an elaborated synchronization approach, which is taking advantage of the full 60 Hz frame rate and the 24 bit-planes per frame. With this approach pattern rates of up to 650 Hz could be achieved with the employed DMD . Furthermore, newly available DMDs offer frame rates higher than 40 kHz enabling the execution of our approach at video rate.
In spectral regions above 1100 nm highly sensitive focal plane arrays, such as InGaAs, InSb and MCT become rather expensive. The obtained results suggest a potential low-cost replacement for such arrays using the developed SPI approach. Thereby, larger array sizes and higher pixel numbers at significantly lower costs become available. Furthermore, single element detectors typically offer a higher detectivity than array detectors of the same detector material increasing the sensitivity. MEMS-based FPTF spectrometers have been proven to be an ideal implementation for obtaining spectral resolution in a single pixel imaging system. These filter devices provide relatively large aperture sizes not reducing the FOV. Thus, they also bear great potential for miniaturized low-cost hyperspectral imaging systems operating in the MIR range . This spectral range is even preferable over the NIR range as it offers more sensitive chemical response. The presented results are a significant step towards sensitive low-cost hyperspectral imaging technologies using fully integrated single-pixel spectrometers in the respective domain. Furthermore, the miniaturized and highly robust MEMS filter technology used to gain spectroscopic information can facilitate hyperspectral imaging for high volume applications.
Province of Upper Austria (Innovative Upper Austria 2020); H2020 Marie Skłodowska-Curie Actions (722380, SUPUVIR); EFRE Urban Innovative Actions (IWB2020).
We thank Florian Hinterleitner for building the electronic driver for the FPTF. The authors acknowledge financial support by the strategic economic- and research program “Innovative Upper Austria 2020” of the province of Upper Austria, the Marie Skłodowska-Curie Action SUPUVIR of the European Union's H2020-MSCA-ITN-2016 and by the federal government of Upper Austria and the European Regional Development Fund in the framework of the program IWB2020 within the project “CO2 Remote Sensing zur Reduktion von CO2 Emissionen im Straßenverkehr”.
1. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). [CrossRef]
2. E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52(2), 489–509 (2006). [CrossRef]
3. S. Qaisar, R. M. Bilal, W. Iqbal, M. Naureen, and S. Lee, “Compressive sensing: From theory to applications, a survey,” J. Commun. Netw. (Seoul) 15(5), 443–456 (2013). [CrossRef]
5. S. B. Andersson and L. Y. Pao, “Non-raster sampling in atomic force microscopy: A compressed sensing approach,” in 2012 American Control Conference (ACC) (IEEE, 2012), pp. 2485–2490. [CrossRef]
6. C. R. Berger, Z. Wang, J. Huang, and S. Zhou, “Application of compressive sensing to sparse channel estimation,” IEEE Commun. Mag. 48(11), 164–174 (2010). [CrossRef]
7. D. Takhar, J. N. Laska, M. B. Wakin, M. F. Duarte, D. Baron, S. Sarvotham, K. F. Kelly, and R. G. Baraniuk, “A new compressive imaging camera architecture using optical-domain compression,” in C. A. Bouman, E. L. Miller, and I. Pollak, eds. (International Society for Optics and Photonics, 2006), Vol. 6065, pp. 606509–606509–10.
8. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 83–91 (2008). [CrossRef]
9. M. Sun, M. P. Edgar, D. B. Phillips, G. M. Gibson, and M. J. Padgett, “Infrared single-pixel imaging utilising microscanning,” Arxiv (2015).
10. N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, “Single-pixel infrared and visible microscope,” Optica 1(5), 285 (2014). [CrossRef]
11. M. P. Edgar, G. M. Gibson, R. W. Bowman, B. Sun, N. Radwell, K. J. Mitchell, S. S. Welsh, and M. J. Padgett, “Simultaneous real-time visible and infrared video with single-pixel detectors,” Sci. Rep. 5(1), 10669 (2015). [CrossRef] [PubMed]
12. G. Zhang, S. Jiao, and X. Xu, “Compressed sensing and reconstruction with Semi-Hadamard matrices,” in 2010 2nd International Conference on Signal Processing Systems (IEEE, 2010), pp. V1–194–V1-197. [CrossRef]
13. M.-J. Sun, L.-T. Meng, M. P. Edgar, M. J. Padgett, and N. Radwell, “A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging,” Sci. Rep. 7(1), 3464 (2017). [CrossRef] [PubMed]
15. F. Magalhães, F. M. Araújo, M. Correia, M. Abolbashari, and F. Farahi, “High-resolution hyperspectral single-pixel imaging system based on compressive sensing,” Opt. Eng. 51(7), 071406 (2012). [CrossRef]
16. S. Jin, W. Hui, Y. Wang, K. Huang, Q. Shi, C. Ying, D. Liu, Q. Ye, W. Zhou, and J. Tian, “Hyperspectral imaging using the single-pixel Fourier transform technique,” Sci. Rep. 7(1), 45209 (2017). [CrossRef] [PubMed]
17. K. Shibuya, T. Minamikawa, Y. Mizutani, H. Yamamoto, K. Minoshima, T. Yasui, and T. Iwata, “Scan-less hyperspectral dual-comb single-pixel-imaging in both amplitude and phase,” Opt. Express 25(18), 21947–21957 (2017). [CrossRef] [PubMed]
19. J. Kilgus, G. Langer, K. Duswald, R. Zimmerleiter, I. Zorin, T. Berer, and M. Brandstetter, “Diffraction limited mid-infrared reflectance microspectroscopy with a supercontinuum laser,” Opt. Express 26(23), 30644–30654 (2018). [CrossRef] [PubMed]
20. W. K. Pratt, J. Kane, and H. C. Andrews, “Hadamard transform image coding,” Proc. IEEE 57(1), 58–68 (1969). [CrossRef]
21. R. Tibshirani, “Regression shrinkage and selection via the lasso: a retrospective,” J. R. Stat. Soc. Ser. B. Stat. Methodol. 73(3), 273–282 (2011). [CrossRef]
22. J. Demšar, T. Curk, A. Erjavec, Č. Gorup, T. Hočevar, M. Milutinovič, M. Možina, M. Polajnar, M. Toplak, A. Starič, M. Štajdohar, L. Umek, L. Žagar, J. Žbontar, M. Žitnik, and B. Zupan, “Orange: Data Mining Toolbox in Python,” J. Mach. Learn. Res. 14, 2349–2353 (2013).
23. P. R. Griffiths and J. A. De Haseth, Fourier Transform Infrared Spectrometry (Wiley-Interscience, 2007).
24. S. S. Welsh, M. P. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. J. Padgett, “Fast full-color computational imaging with single-pixel detectors,” Opt. Express 21(20), 23068–23074 (2013). [CrossRef] [PubMed]
25. J. Suo, L. Bian, Y. Xiao, Y. Wang, L. Zhang, and Q. Dai, “A self-synchronized high speed computational ghost imaging system: A leap towards dynamic capturing,” Opt. Laser Technol. 74, 65–71 (2015). [CrossRef]
26. J. Kilgus, R. Zimmerleiter, K. Duswald, F. Hinterleitner, G. Langer, and M. Brandstetter, “Application of a Novel Low-Cost Hyperspectral Imaging Setup Operating in the Mid-Infrared Region,” Proceedings 2(13), 800 (2018). [CrossRef]