Open Access
Add to BibSonomy
Abstract
In this paper, a single-pixel hyperspectral imager is developed based on the Hadamard transformation. The imager’s design, fabrication, signal processing method, and experimental results are discussed. The single-pixel hyperspectral imager works in pushbroom mode and employs both spatial encoding and spectral encoding to acquire the hyperspectral data cube. Hadamard encoding patterns, which are known for their multiplexing advantage to achieve high signal-to-noise ratio (SNR), are used in both encoding schemes. A digital micromirror device (DMD) from Texas Instruments (TI) is used for slow spatial encoding and a resonant scanning mirror in combination with a fixed Hadamard mask is used for fast spectral encoding. In addition, the pushbroom operation can be achieved internally by spatially shifting the location of the Hadamard encoded slit on the DMD, thus the imager is able to acquire 3D data cubes without the need to scan it across the object. Although our experimental results demonstrate the hyperspectral data cubes of various objects over a 450 nm ∼ 750 nm visible spectral range, the proposed imager can be easily configured to be used at other wavelengths due to the single-pixel detection mechanism used.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
A hyperspectral imager records a detailed broadband electromagnetic spectrum at each pixel within its field-of-view (FOV) [1]. Different from conventional cameras, a hyperspectral imager can record the spectral characteristic of an object in its operational spectral band. Thus, in addition to capturing an image of an object, it can also offer critical information by processing its spectrum to identify the material. Therefore, hyperspectral imagers are widely applied in the field of agriculture, biomedical imaging, food processing, surveillance, astronomy, environmental monitoring, and many others [2–5].
Hyperspectral imagers typically adopt one-dimensional (1D) or two-dimensional (2D) detector arrays as the sensing devices. Several different imaging methodologies, such as whiskbroom scanning, pushbroom scanning, and filtered camera, have been developed [6–9]. Hyperspectral imagers based on large-scale photodetector arrays are relatively developed in the visible to near infrared (VNIR) wavelengths (0.4 µm to 1 µm). The main material of the VNIR array detectors, i.e. silicon, is very efficient for photon-to-electron conversion in this spectral region and at the same time it is also ideal for large-scale electronics integration. As a result, hyperspectral imagers in the VNIR range are now relatively low cost and compact. However, when operated at wavelengths where silicon is blind, for example in the short-wave infrared (SWIR) or mid-wave infrared (MWIR) wavelengths, hyperspectral imagers become complicated, bulky, and expensive as their detector arrays require non-silicon technologies such as InGaAs and InSb.
In recent years, imaging with just a single-pixel photodetector has attracted much attention. Using a single-pixel detector can potentially lower the overall cost, package size, and weight of an imaging system compared to that using an arrayed detector [10–16]. More importantly, the use of a single-pixel detector allows the imaging system to operate at wavelengths currently unavailable or prohibitively expensive for a conventional arrayed detector. Besides, without the array uniformity errors, the calibration process for single-pixel-based hyperspectral imagers may be simplified.
Several methods have been reported to employ a single-pixel detector scheme in a hyperspectral imager. For example, one way is to utilize a microelectromechanical systems (MEMS) based vibratory diffraction grating for spatial scanning and a rotation mirror for spectral scanning [17,18]. As a result, the single-pixel detector records a hyperspectral image as it scans sequentially point-by-point through both the spatial and spectral dimensions. However, due to the framerate requirement, the integration time of the detector at each spatial and spectral point is limited, thus leading to low signal-to-noise ratio (SNR). To increase the signal at the single-pixel detector and overcome the issue of low SNR, multiplexing sensing scheme is used, for example by replacing the array detector directly with a single-pixel Fourier-transform infrared spectroscopy (FTIR) spectrometer and a spatial encoder [19]. FTIR is the most popular solution for sensing infrared spectrum since it can collect high spectral resolution data in a wide spectral range simultaneously. Another example is by using a tunable narrow-band wavelength filter after a spatial light modulator and sensing same encoded scene in different spectrums sequentially [20]. A significant advantage in both methods is its multiplexing sensing. With multiplexing sensing, light intensities from different positions (pixels) of the object are measured simultaneously, which can dramatically increase signal levels and hence improve the SNR. However, both methods have some limitations in their applications. A FTIR is usually a benchtop equipment with precision movable optomechanics. And based on a Michelson interferometer design, a FTIR is sensitive to external disturbances and thus is limited for use in controlled laboratory environment. On the other hand, tuning a variable wavelength filter in a hyperspectral imager might be time consuming and thus reduces the frame rate of the imager.
The Hadamard transform [21] is another multiplexing scheme that has Fellgett’s advantage, and hence gained popularity in single-pixel imaging in recent decades. One way to implement the Hadamard transform in the single-pixel imaging system is through the use of cyclic S-matrices where a weighted pattern (encoding pattern) is generated on the incoming image plane [22–24]. The pattern has “ON” and “OFF” pixels that block or allow the imaging light to reach the single-pixel detector. A digital micromirror device (DMD) [25] is a device that is frequently used for implementing light modulation by Hadamard transform patterns. Some literatures also introduced the method to use the DMD as the spatial light modulator to modulate the light source directly and illuminate the sample with a spatially varying light pattern [26].
In this paper, we propose a new design of single-pixel hyperspectral imager that combines Hadamard transform-based multiplexing sensing and single-pixel imaging technologies. The imager achieves single-pixel hyperspectral sensing with a two-stage spatial and spectral encoding scheme. In combination with a DMD for spatial encoding, a resonant scanner with a fixed encoding mask are used for spectral encoding. With a single-pixel detector and a compound parabolic concentrator (CPC), the imager becomes more compact and affordable in its potential applications in the VNIR and even IR spectral bands. Furthermore, the hyperspectral imager is designed to operate in pushbroom mode and the pushbroom operation can be achieved internally by spatially shifting the location of the Hadamard encoded slit on the DMD. The internal pushbroom operation allows the proposed hyperspectral sensor to acquire the 3D data cube without the need to move the imager physically across the object.
2. Optical design
The proposed hyperspectral imaging system operates in pushbroom mode and uses Hadamard patterns to encode both the spatial and spectral dimensions. As shown in Fig. 1, when light reflected from the object is collected by the fore-optics [Mamiya RZ67 110mm f/2.8 lens, with dimensions of 104mm (W) × 133.5mm (H) × 211.5mm (L) and a flange focal distance of 105 mm], a bandpass filter is used to limit the wavelength band entering the imaging system to a set of M bands from λ1, λ2, … to λM. The fore-optics then images the object to a DMD, where a column of micromirrors are used as the entrance slit to an imaging spectrometer for pushbroom mode operation. A DMD (DLP7000) from TI (Texas Instruments) with a 1024 × 768 micromirror array is used. Each micromirror in the array can be programmed to rotate in two directions to represent the encoding pattern ‘1’ or ‘0’, respectively. As shown in Fig. 1, the image is separated into two parts by the micromirrors on the DMD. When the micromirror rotates to the right-hand side (which stands for ‘1’ or ‘open’ state), it reflects the light to a Czerny-Turner imaging spectrometer system (red box). On the other hand, when the micromirror rotates to the left-hand side (denoted as ‘0’ or ‘closed’ state), the light will be reflected to an auxiliary imaging system (blue box). Both the imaging spectrometer and auxiliary imager share the same fore-optics. We use a column of micromirrors as a programmable slit on the DMD, which is also the entrance slit to the imaging spectrometer. Those mirrors in the slit are opened or closed depending on the spatial encoding patterns, while other micromirrors on DMD are always closed when in operation thus reflecting light to the auxiliary imager and forming a conventional image of the object on the CCD. This auxiliary image of the object is helpful in identifying the actual FOV of the pushbroom operation of the imaging spectrometer, which appears as a thin dark line in the CCD image.
Fig. 1. Schematic of the hyperspectral imager with two parts: a pushbroom single-pixel hyperspectral imager (red box) and a conventional imaging system (blue box).
From the entrance slit on the DMD, the light is collimated by a collimating mirror and reflected onto a diffraction grating. The diffracted beams from the grating then reflect off a resonant scanning mirror and a focusing mirror to form a spectral image of the entrance slit on the exit plane of the imaging spectrometer, where a fixed spectral encoding mask is located. The spectral encoding process is implemented by moving the spectral image of the slit across the fixed spectral encoding mask, which is done by rotating the scanning mirror. The light that passes through the mask is then spectrally encoded, which is subsequently collected by a compound parabolic concentrator (CPC) [27] onto a single pixel detector. The single-pixel detector outputs a voltage signal proportional to the intensity of the light gathered.
Figure 2 further highlights the details of the two-stage spatial and spectral Hadamard encoding mechanisms for single-pixel hyperspectral imaging. As shown, a selected column of micromirrors (i.e. the entrance slit) is set at the nth pattern (n = 1, 2, …, N), where the red micromirrors are set at ‘open’ state and the blue micromirrors are set at ‘closed’ state, with the weights of the corresponding encoding pattern indicated on the left side of the DMD front view. Light that falls on the slit is then spatially encoded as an,I I(i, j), where an,i is the weight of the ith pixel (i.e. the ith micromirror) in the nth pattern and I(i, j) is the light intensity at the ith pixel and at wavelength λj. The spatially encoded slit is then spectrally dispersed and imaged on the fixed spectral encoding mask, as shown in the figure. It should be noted that the zeroth order diffracted light is outside of the spectral encoding mask and is therefore blocked throughout the scanning/encoding process. The second order diffracted light does appear in the encoding region on the mask. However, its intensity is negligibly smaller than that of the first order due to the blazed grating used. Hence, with the bandpass filter used, the spectrally dispersed slit image on the second encoding mask is then limited to the first diffraction order and spans from λ1 to λM as shown in Fig. 2. As the slit image scans across the fixed encoding mask, the light passing through is encoded a second time. As shown, the white regions on mask are transparent and the black ones are opaque, and the weights of the corresponding encoding pattern are also indicated at the bottom of the mask. In summary, at the mth scan position (m = 1, 2, …, M), the measured total intensity M(n, m) at the single-pixel photodetector can be expressed with:
Fig. 2. Schematics of the proposed spatial and spectral Hadamard encoding mechanisms for single-pixel hyperspectral imaging.
Thus, the above equation can be simplified as:
where A is the spatial encoding matrix, B is the spectral encoding matrix, M is the measurement matrix, and I is the hyperspectral image of the slit. Clearly, the hyperspectral image I can be recovered by: where A−1 and B−1 are the inverse matrices of A and B, respectively.2.1 Imaging spectrometer design
Our imaging spectrometer is constructed based on a modified Czerny-Turner design. As shown in Fig. 3(a), the optical path is designed and simulated in Zemax. To reduce the aberrations presented in the Czerny-Tuner imaging spectrometer, we referenced a design reported by Xue et. al [28]. Firstly, the incident angles of the central rays on optical components (i.e. mirrors and grating) and the tangential radii of the collimating and focusing mirrors are determined with the aim of reducing coma based on the Shafer equation [29]. These are fixed as constants to facilitate optimization in Zemax in the subsequent steps. With these constants established, the distances between the optical components are then calculated. Next, when the positioning of the optical components in the system is determined to be mechanically viable, the movement of the scanning mirror and the reduction of astigmatism using toroidal mirrors is then accounted for through one final optimization.
Fig. 3. (a) The imaging spectrometer in our hyperspectral sensor is designed based on a modified Czerny-Turner design; (b) Full-field spot diagram for different fields and scan angles. Red – 750 nm, Blue – 600 nm, Green – 450 nm; (c) Spot diagram of wavelength 450 nm at different fields and scan angles. Scale = 200 µm; (d) Spot diagram of wavelength 600 nm at different fields and scan angles. Scale = 200 µm (e) Spot diagram of wavelength 750 nm at different fields and scan angles. Scale = 200 µm.
We also conduct the simulation of the spot size on the mask. As shown in Fig. 3(b), the full field spot diagram shows the overall image size on the mask based on the different field of views (i.e. 0° and ± 2.25°) and the different mirror scan angle configurations of 0° and ± 1.38°. Green spots represent the results of wavelength 450 nm and red spots represent those of 750 nm. The spots of 600 nm is barely visible in this representation as they overlap with the spots of the other wavelengths. The detailed spot diagrams are individually shown in Figs. 3(c), 3(d), and 3(e) for the three wavelengths, where we can observe that the all spot sizes are contained within a maximum of 150 µm in diameter. Since the second mask is a slit mask, the keystone error is not of importance. Instead, the smile error would affect the resolution of the system. For our system, the smile error for all wavelengths and scan angles are controlled between 35 µm and 60 µm, which is smaller than the encoding slit width of 90 µm on the second encoding mask.
2.2 Auxiliary imaging system design
The auxiliary imaging system is designed on the opposite side of the imaging spectrometer, as shown in Fig. 4(a). It shares the same fore optics with the imaging spectrometer. This allows the DMD pixels that are not used in the spectrometer entrance slit to be functional to the auxiliary imaging system, thus utilising them to update the user of the target area that is being pushbroom scanned. This is an off-axis imaging system and generally suffers from relatively large optical aberration, thus we spend minimal design and fabrication efforts on its aberration correction as it is not critical to the performance of the hyperspectral imaging system. Here, we refer to an off-axis imaging theory proposed by Chang et al. [30], which states that the linear astigmatism of a confocal off-axis two-mirror system can be eliminated by fulfilling the following equation:
Fig. 4. (a) Layout of the auxiliary imaging system used for locating the FOV of the hyperspectral imaging system; (b) Spot diagram of the system. Scale = 100 µm.
For ellipse 1, a1 and b1 are 30 mm and 25.84 mm respectively. The focus, c1 is 15.24 mm. For ellipse 2, a2 and b2 are 52.50 mm and 48.55 mm respectively. The focus, c2 is 19.99 mm. Due to the tilted image plane on the DMD, the image formed on the CCD is slightly distorted, with the spot diagram shown in Fig. 4(b). Nevertheless, the image formed is relatively clear and allows the user to look at the view that is being captured by the imaging spectrometer.
3. Experimental setup and signal processing
3.1 Mechanical design, fabrication, and calibration
The mechanical components of the hyperspectral imager are designed in SolidWorks and fabricated using precision machining. A photo of the assembled system is shown in the Fig. 5(a); all components are mounted on the baseboard and secured by screws. To facilitate the alignment of the components, dowel pins are used to mark the precision-machined reference positions on the baseboard. The component mounts and the mirrors are fabricated by precision machining of aluminum. The mirrors are additionally diamond turned and polished to form the optical surfaces required. To reduce the weight of the system, the enclosure is fabricated by plastic 3D printing. The enclosure of the system does not appear in Fig. 5(a) for the purpose of displaying the internals of the system.
Fig. 5. (a) Photo of the fabricated hyperspectral imager prototype; (b) Phto of the spectral encoding system, which includes an encoding mask, a CPC, and a single-pixel detector. The encoding mask comprises alignment marks, calibration marks and encoding patterns; (c) schematic showing the calibration method to determine the sampling positions when the dispersed spectrum is scanned on the spectral encoding mask.
As shown in Fig. 5(b), we design alignment marks on the fixed spectral encoding mask to ensure the image of the DMD slit can be aligned parallel to the mask encoding pattern, which contains a plural of open and closed slits. Since the slit on the DMD is formed by a vertical column of micromirrors, we use some of the remaining micromirrors away from the slit to generate patterns that can be imaged onto the spectral encoding mask and compare them with the alignment marks on the mask. We then slightly adjust the position of the spectral encoding mask and align the alignment patterns under an optical microscope. After the alignment, the position of the spectral encoding mask is fixed on the baseboard and the alignment marks on it is subsequently covered by black tapes to prevent unwanted light from leaking through the alignment marks and reaching the detector.
To calibrate the sampling positions of the dispersed spectrum of the slit when it scans across the fixed spectral encoding mask, we design and place calibration marks on the glass encoding mask both above and below the actual encoding region [see Fig. 5(b) and 5(c)]. As shown schematically in Fig. 5(c), the encoding region on the glass mask consists of a complete set of encoded slits designed according to a S-matrix of order M (M is the number of resolvable spectral bands of the hyperspectral imager) and then followed immediately by a duplicated set of encoded slits with no gaps. In the figure, the white areas denote transparent regions and black areas denote opaque regions. The calibration marks for the 450 nm wavelength laser is located above the encoding region and is aligned to the first slit of the encoding pattern. We then turn on one column of micromirrors above the encoding slit on DMD. More specifically, this column is in fact along the vertical extension line of the slit. The rest of the micromirrors including those in the encoding slit are closed. We use a laser beam at wavelength 450 nm to illuminate the micromirrors above the slit in the calibration region as shown. An image of the illuminated micromirror column is then produced on the glass mask. When the scanning mirror rotates, this image moves along the plane of the mask. The angular position of the scanning mirror and the output voltage from the single-pixel detector are recorded and monitored in real-time by a data acquisition card from National Instruments (NI). As schematically shown in Fig. 5(c), the signal from the photodetector V thus has two peaks. The first peak at time t1 (corresponding to the scanner angular position θ1) denotes the sampling start position because the illumination laser at 450 nm is exactly the start wavelength of the designed spectral coverage of the hyperspectral imager. The second peak at time tM+1 (corresponding to angular position θM+1) indicates that a complete spectral encoding cycle has just ended. The angular range of the scanner between θ1 to θM+1 is then equally divided into M intervals, i.e. θ1, θ2, …, θM, and these are sampling positions where the spectrum is correctly encoded by the designated encoding pattern as it moves across the mask. Since the scanning mirror is closed-loop controlled with an integrated precision angular position sensor, this calibration process is only required once at the initial setup stage. After calibration, the calibration marks are again covered by black tapes to prevent unwanted light leakage. It should be noted that we also placed calibration marks for a 750 nm laser (i.e. the end wavelength of the system spectral coverage) below the encoding region on the mask and a similar procedure can be performed to verify these sampling positions.
An outstanding advantage of using a DMD in the proposed hyperspectral imaging system is that it supports an internal pushbroom scanning scheme, which allows the hyperspectral imager to capture the 3D hyperspectral data cube without moving the sensor with respect to the object. As shown schematically in Fig. 5(c), the DMD consists of multiple columns of micromirrors, and each column can be utilized as a spatially encoded entrance slit to the imaging spectrometer. Therefore, the slit (i.e. a column of micromirrors) on DMD can be dynamically programmed to shift horizontally one-by-one across the FOV of the system, this is equivalent to the effect of moving the slit across the object using pushbroom scanning (flying over the object for example). In our work, we use a total of 45 slits, whose positions are highlighted in the figure, and each slit is individually calibrated using the above-mentioned calibration method. This unique design using DMD allows the imager to capture the 3D hyperspectral data cube without any movement.
3.2 Signal processing
The signal flow chart of the hyperspectral imaging system is shown in Fig. 6(a). In the proposed control system in Fig. 6(a), a desktop computer having a NI’s LabVIEW platform sends out signals to control the DMD (DLP7000), which can change the directions of micromirrors in the slit for spatial encoding. The resonant scanner (EOPC SC-30) is operated oscillatorily at its natural frequency (3.7 kHz) with the scan amplitude stabilized by its own AGC driver for spectral encoding. The desktop also receives the angular position signal from the AGC driver and the intensity signal from the single-pixel photodetector through a NI data acquisition card. Each time when a spatial encoding pattern is prepared on the DMD, the desktop receives a ready signal from the DMD and waits for the resonant scanner to get to the start position for spectral encoding. Once the resonant scanner gets to the start position, the desktop begins recording the detector intensity signal for a complete set of spectral encoding patterns at the predetermined scanner angular positions until the scanner reaches the stop position. Then, the micromirrors on the DMD is switched to the next spatial encoding pattern and the process is repeated until an entire spatial encoding process is completed. Finally, the hyperspectral image is reconstructed using Eq. (3). In our work here, the spatially encoded slit on the DMD has 359 micromirrors or pixels. The spectral encoding mask is designed according to a S-matrix of order 63 (i.e. supporting 63 resolvable spectral bands). The wavelength coverage is from 450 nm to 750 nm, which means that the spectral resolution of the system is about 4.8 nm.
Fig. 6. (a) Flow chart of the signal processing for the proposed hyperspectral imager; (b) synchronized spatial and spectral encoding processes.
It is noted that this proof-of-principle system is not optimized efficiently for high-frame rate operation due to the signal delays among individual components and the desktop computer. And currently, the acquisition time for a hyperspectral image is about 20 seconds. It is expected that the frame rate can be significantly improved with a customized real-time signal control and processing system. Our estimation for the ideal frame rate of the system is as follows. Here, we designed the slit with 359 micromirrors or pixels, which means that the spatial encoding pattern in the slit needs to be changed 359 times to acquire a hyperspectral image. The resonant frequency of the scanning mirror is 3.7 kHz, which means the time required for a complete set of spectral encoding is 1/(3700×2) ≈ 0.14 ms assuming dual-directional encoding. Figure 6(b) further shows schematically the synchronization scheme. As shown, the sinusoidal oscillation of the resonant scanner as a function of time for spectral encoding is highlighted. During the time period from t1 to t2, the scanner scans in one-direction. Its angular velocity is relatively linear and the spectral encoding is carried out in this period. From time t2 to t3, the scanner is altering its direction and the angular velocity is highly nonlinear, such period cannot be used for spectral encoding. However, resetting the micromirrors to the next slit spatial encoding pattern can be nicely carried out in this period. Due to the high operation speed of the DMD (as high as 30 kHz), setting up the next slit spatial encoding pattern can be done at the microsecond level. In this way, the spatial and spectral encoding schemes are implemented in a staggered way in the time domain. Hence, the time needed to complete the push-broom spatial and spectral encoding process is 359×0.14 ≈ 50 ms. In other words, the frame rate to acquire a 2D spatial-spectral image is expected to be about 20 fps with a customized control system. It should also be noted that the frame rate to acquire a full 3D hyperspectral date cube will be lowered by a factor equal to the total number of push-broom scan locations in the 3D data cube.
4. Experimental results
Firstly, we test the resolution of the prototype hyperspectral imager with two laser beams having different wavelengths respectively at 450 nm and 635 nm. As shown in the Fig. 7(a), we direct these two laser beams to a white paper and align them vertically. We then further align the pushbroom slit in-line with the two laser spots. Figure 7(b) shows the obtained hyperspectral image of the two laser spots by the developed system. We take two rows of data from the hyperspectral image to analyze the spectral accuracy and resolution quantitatively, as shown in Fig. 7(c). It is observed that the peaks of the recovered spectra are located at 450 nm and 638.7 nm respectively indicating a wavelength accuracy less than an encoding pixel on the fixed mask, which is about 5 nm. It is also observed that the full width at half maximum (FWHM) of these two spectra peaks is about 10 nm ∼ 14 nm indicating that the experimental spectral resolution is worse than the expected resolution of 5 nm. This could be due to the mechanical fabrication errors that introduce aberrations and broaden the spot sizes on the fixed Hadamard mask.
Fig. 7. (a) Image of two laser spots on a white paper recorded by the auxiliary imaging system. (b) Hyperspectral image of two laser spots. (c) Recorded spectra of two laser spots.
Secondly, we test the prototype hyperspectral imager with various objects placed against a dark background about 4 meters away from the imager. The objects are illuminated by a white LED light source. Figure 8 shows the experimental results, where these targets are the letters “NUS” made of three different colored papers in Fig. 8(a), a toy cube with colored surfaces and numbers in Fig. 8(b), and a toy flower made of colored papers in Fig. 8(c). For each result, the image captured by the auxiliary imaging system is shown on the left and the hyperspectral image of the pushbroom slit is shown on the right. The auxiliary images captured are slightly distorted, which is expected as the aberrations of the auxiliary system is not fully optimized. It is acceptable because the function of auxiliary imaging system is only for checking the FOV and has no influence on the performance of the hyperspectral imager. For the hyperspectral images shown in Fig. 8, the vertical axis indicates the spatial dimension and the horizontal axis indicates the spectral dimension. Due to the integrated auxiliary optical imaging system, we can visualize the actual FOV of the hyperspectral imager and at the same time identify the location of the pushbroom slit on the target that is being scanned in real-time. The pushbroom slit is indicated as a thin dark line in each target image captured on the left side, which is expected because the DMD micromirrors in the pushbroom slit are tilted towards the imaging spectrometer instead of the auxiliary imager. From the results shown in Fig. 8, it is quite clear that the captured hyperspectral images are indeed correct, and they properly capture both the heights and spectra of the targets at the location of the pushbroom slit.
Fig. 8. Conventional CCD images and hyperspectral images at the pushbroom slit location of (a) letters “NUS”, (b) a toy cube, and (c) a toy flower.
To demonstrate the internal pushbroom scanning to acquire the 3D data cube without moving the hyperspectral imager relative to the target, we use the letters “NUS” as the object in this experiment. The slit position on the DMD shifts horizontally 45 times leading to a hyperspectral data cube of 359 × 45 × 63 pixels in the X, Y, and λ directions as shown in Fig. 9(a). We separated the data cube into two parts at the 600 nm position in the λ direction to see the image more clearly. Figure 9(b) shows the recovered spectra of three chosen patches from the object. The three chosen patches are in green, blue and red colors respectively, and the peaks of the recovered spectra lines clearly match well with the expected wavelengths of those three colors. Figure 9(c) further shows some narrowband images of the object at 488 nm, 537 nm, 570 nm, 600 nm, 638 nm, and 672 nm wavelengths. It can be seen that when we take the 488 nm wavelength image, the blue letter ‘U’ is visible. And when we take the 537 nm wavelength image, the green letter ‘N’ appears on the image with the intensity becoming stronger in the 570 nm wavelength image. Next, when we take the 600 nm wavelength image, the red letter ‘S’ appears while the intensities of ‘N’ and ‘U’ become weak. When the wavelength is at 638 nm, the red letter ‘S’ shows the highest intensity. Finally, when we take the image at 672 nm, ‘N’ and ‘U’ almost disappear with only ‘S’ left on the image. These images clearly demonstrate the capability of the proposed single-pixel hyperspectral imager.
Fig. 9. (a) The 3D hyperspectral data cube of the object; (b) Spectra from the chosen patches of the object; (c) CCD image of the object and narrowband images of the object sliced at different wavelengths from the data cube.
Figure 10 further highlights some of the narrowband images of different objects extracted from their captured 3D hyperspectral data cubes using the developed hyperspectral imager. The results are quite satisfactory and the slices of the narrowband images at different wavelengths match the colors of the objects very well.
Fig. 10. Narrowband images sliced at different wavelengths of their respective 3D data cubes of three test objects.
5. Conclusion
In this paper, we report a novel design of a single-pixel hyperspectral imager with two cascaded encoding processes, i.e. a spatial encoding process implemented by a DMD and a spectral encoding process implemented using a resonant scanner with a fixed mask. Both encoding processes employ the Hadamard transform to achieve multiplexing sensing. The proposed hyperspectral imager uses a single-pixel detector as the image sensing device, which does not require large-scale integration of detector array with electronics on a sensor chip. The imager can thus potentially be operated at IR wavelengths where IR focal plane arrays are expensive and have relatively low performance. In our proof-of-principle demonstration here, the hyperspectral imager has a spectral coverage from 450 nm to 750 nm with 63 resolvable spectral bands. The image is operated in a pushbroom mode with a slit spatially encoded with 359 pixels. An internal pushbroom operation is also demonstrated to show the capability of recording the 3D data cube without moving the imager relative to the target. This is achieved through digitally programming the encoding slit to shift horizontally across the DMD.
It is also noted that the performance of the hyperspectral imager can be further improved. The frame rate of the imager can be enhanced through customized control electronics and data processing systems. Higher resolution can be achieved by cascading multiple single-pixel detectors in both the spatial and spectral dimensions [31]. The footprint of the imager can be reduced by combining the grating and scanning mirror using MEMS technology. The proposed single-pixel hyperspectral imager might be useful in a range of applications including industrial process control, onsite material identification and verification, onsite environment monitoring, biomedical point of care testing, food and beverage quality assessment, real-time analysis of coating and films, as well as pharmaceutical research and drug development.
Funding
Ministry of Education - Singapore (R-265-000-557-112).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. D. Landgrebe, “Hyperspectral image data analysis,” IEEE Signal Process. Mag. 19(1), 17–28 (2002). [CrossRef]
2. A. A. Gowen, C. P. O’Donnell, P. J. Cullen, G. Downey, and J. M. Frias, “Hyperspectral imaging – an emerging process analytical tool for food quality and safety control,” Trends Food Sci. Technol. 18(12), 590–598 (2007). [CrossRef]
3. S. McGregor, S. Lacroix, and A. Monmayrant, “Adaptive hyperspectral imager: design, modeling, and control,” J. Opt. 17(8), 085607 (2015). [CrossRef]
4. M. M. P. Arnob, H. Nguyen, Z. Han, and W.-C. Shih, “Compressed sensing hyperspectral imaging in the 0.9–2.5 µm shortwave infrared wavelength range using a digital micromirror device and InGaAs linear array detector,” Appl. Opt. 57(18), 5019–5024 (2018). [CrossRef]
5. A. Plaza, J. A. Benediktsson, J. W. Boardman, J. Brazile, L. Bruzzone, G. Camps-Valls, J. Chanussot, M. Fauvel, P. Gamba, A. Gualtieri, M. Marconcini, J. C. Tilton, and G. Trianni, “Recent advances in techniques for hyperspectral image processing,” Remote Sens. Environ. 113, S110–S122 (2009). [CrossRef]
6. F. Sigernes, M. Syrjäsuo, R. Storvold, J. Fortuna, M. E. Grøtte, and T. A. Johansen, “Do it yourself hyperspectral imager for handheld to airborne operations,” Opt. Express 26(5), 6021–6035 (2018). [CrossRef]
7. G. Lua and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19(1), 010901 (2014). [CrossRef]
8. N. Gupta, R. Dahmani, and S. Choy, “Acousto-optic tunable filter based visible- to near-infrared spectropolarimetric imager,” Opt. Eng. 41(5), 1033–1038 (2002). [CrossRef]
9. N. Gat, “Imaging Spectroscopy Using Tunable Filters: A Review,” Proc. SPIE 4056, 50–64 (2000). [CrossRef]
10. H. Jiang, S. Zhu, H. Zhao, B. Xu, and X. Li, “Adaptive regional single-pixel imaging based on the Fourier slice theorem,” Opt. Express 25(13), 15118–15130 (2017). [CrossRef]
11. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 83–91 (2008). [CrossRef]
12. S. S. Welsh, M. P. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. J. Padgett, “Fast full-color computational imaging with single-pixel detectors,” Opt. Express 21(20), 23068–23074 (2013). [CrossRef]
13. Z. Zhang, X. Wang, G. Zheng, and J. Zhong, “Hadamard single-pixel imaging versus Fourier single-pixel imaging,” Opt. Express 25(16), 19619–19639 (2017). [CrossRef]
14. Z. Zhang, X. Ma, and J. Zhong, “Single-pixel imaging by means of Fourier spectrum acquisition,” Nat Commun 6 6225, (2015).
15. K. M. Czajkowski, A. Pastuszczak, and R. Kotynski, “Real-time single-pixel video imaging with Fourier domain regularization,” Opt. Express 26(16), 20009–20022 (2018). [CrossRef]
16. L. Bian, J. S. X. Hu, F. Chen, and Q. Dai, “Efficient single pixel imaging in Fourier space,” J. Opt. 18(8), 085704 (2016). [CrossRef]
17. K. K. L. Cheo, Y. Du, G. Zhou, and F. S. Chau, “Post-corrections of image distortions in a scanning grating-based spectral line imager,” IEEE Photonics Technol. Lett. 25(12), 1103–1106 (2013). [CrossRef]
18. G. Zhou, K. K. L. Cheo, Y. Du, F. S. Chau, H. Feng, and Q. Zhang, “Hyperspectral imaging using a microelectrical-mechanical-systems-based in-plane vibratory grating scanner with a single photodetector,” Opt. Lett. 34(6), 764–766 (2009). [CrossRef]
19. S. Jin, W. Hui, Y. Wang, K. Huang, Q. Shi, C. Ying, D. Liu, Q. Ye, W. Zhou, and J. Tian, “Hyperspectral imaging using the single-pixel Fourier transform technique,” Sci. Rep. 7(1), 45209 (2017). [CrossRef]
20. P. Gattinger, J. Kilgus, I. Zorin, G. Langer, R. Nikzad-Langerodi, C. Rankil, M. Groeschl, and M. Brandstetter, “Broadband near-infrared hyperspectral single pixel imaging for chemical characterization,” Opt. Express 27(9), 12666–126673 (2019). [CrossRef]
21. M. S. Harwit and N. J. A. Sloane, Hadamard Transform Optics. (Academic1979).
22. M. J. Sun, L. T. Meng, M. P. Edgar, M. J. Padgett, and N. Radwell, “A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging,” Sci. Rep. 7(1), 3464 (2017). [CrossRef]
23. Y. Du and G. Zhou, “A MEMS-driven Hadamard transform spectrometer,” Proc. SPIE 10545, 31 (2018). [CrossRef]
24. J. Xu, H. Liu, C. Lin, and Q. Sun, “SNR analysis and Hadamard mask modification of DMD Hadamard Transform Near-Infrared spectrometer,” Opt. Commun. 383, 250–254 (2017). [CrossRef]
25. L. J. Hornbeck, “The DMD (TM) projection display chip: A MEMS-based technology,” MRS Bull. 26(4), 325–327 (2001). [CrossRef]
26. Z. Zhang, X. Wang, G. Zheng, and J. Zhong, “Fast Fourier single-pixel imaging via binary illumination,” Sci. Rep. 7(1), 12029 (2017). [CrossRef]
27. W. T. Welford and R. Winston, High Collection Nonimaging Optics (Academic1989).
28. Q. Xue, S. Wang, and F. Lu, “Aberration-corrected Czerny-Turner imaging spectrometer with a wide spectral region,” Appl. Opt. 48(1), 11–16 (2009). [CrossRef]
29. A. B. Shafer, L. R. Megill, and L. Droppleman, “Optimization of the Czerny-Turner spectrometer,” J. Opt. Soc. Am. 54(7), 879–887 (1964). [CrossRef]
30. S. Chang, J. H. Lee, P. Kim, H. Kim, W. J. Kim, and I. Song, “Linear astigmatism of confocal off-axis reflective imaging systems and its elimination,” Appl. Opt. 45, 484 (2006). [CrossRef]
31. L. Li, Y. Qi, Z. H. Lim, G. Zhou, F. S. Chau, and G. Zhou, “MEMS-based self-referencing cascaded line-scan camera using single-pixel detectors,” Opt. Express 27(18), 25457–25469 (2019). [CrossRef]
