1. Introduction

Hyperspectral imaging spectrometers have been utilized for a wide range of applications, including, but not limited to environmental monitoring [1,2], biomedical imaging [3, 4], surveillance [5], food safety inspection and control [6], agricultural monitoring [7, 8], and mineralogy [9]. Both spaceborne and airborne hyperspectral imagers can provide critical data for a wide range of scientific, economic and national security needs. Despite the high expected value of Earth-scale spectral information, high cost and complexity has limited the number of fielded spaceborne hyperspectral imagers, making global, persistent coverage with rapid revisits infeasible [10].

Here we present a computational imaging approach to hyperspectral imaging spectroscopy that is suitable for system miniaturization while maintaining high sensitivity. We call our approach the Computational Reconfigurable Imaging Spectrometer, or “CRISP” for short. The CRISP spectrometer design does not require active components (unlike some other hyperspectral imaging solutions [11]) and is compatible with a broad range of wavelengths. It uses a dual-disperser reimaging design, with a static coding mask in the image plane, and relies on platform motion or sensor scanning to achieve coding diversity. The CRISP spectrometer makes spatial-spectral coded measurements of a 3D data cube on a 2D detector array. Along-track platform or scan motion produces a time-modulated intensity pattern which is sampled at the frame rate of the imager. The resulting frames are recovered by inverting a function defined by the mask pattern, from which the individual spectral images are extracted. This measurement approach provides significant radiometry advantages versus traditional pushbroom spectrometer designs. These radiometric advantages grow as the format size of the focal plane array increases, and are especially pronounced for detector-noise limited measurement systems such as uncooled microbolometer arrays.

Computational spectrometers have been previously reported. Most notably, Duke University researchers have pioneered single- and dual-disperser computational imaging spectrometer designs [12], [13] similar to that of CRISP, but with a generally different research emphasis. The Duke research has primarily focused on compressive sensing, in which a 3D spatial / spectral data cube is reconstructed using one or a small number of imaging snapshots [13]. The data cube is assumed to have a sparse representation such that the number of unknowns is much smaller than the total size of the data cube, and also smaller than the number of measurements made. In contrast, we rely on platform motion, and capture a sequence of measurements sufficient in number to fully determine and reconstruct the data cube, without requiring sparsity. The Duke researchers have previously also reported a near-IR single-disperser pushbroom coded spectrometer microscope [12], which is the closest precursor to our work. The authors note that this design has a throughput advantage (Jacquinot’s advantage [14]) versus many slit-based spectrometers, as well as a multiplex advantage (Fellgett’s advantage [15]), in which signal-to-noise ratio (SNR) is enhanced in the recovered spectrum for detector-noise-limited measurements versus dispersive designs. Our implementation differs from this Duke contribution in a number of ways. For example, we use a dual-disperser configuration rather than a single-disperser configuration for our pushbroom measurement, as the dual disperser measurement recombines the image spectrally, making the frame-to-frame alignment required for this measurement much easier to observe. In addition, our intended application is a moving platform which images scenes in the far field rather than a microscope. As a result, our optical measurement system requires no moving parts, while the microscopy application requires the spectrograph hardware to move relative to a stationary microscope objective.

More fundamentally, CRISP builds on this earlier work by extending these radiometric advantages further, with a new advantage we call “coding gain”: As the along-track format size of the focal plane array grows beyond the minimum number of platform shifts needed to reconstruct the data cube, the measurement becomes more overdetermined, and the SNR of the reconstructed data cube continues to improve. The result is that the SNR can grow ever-larger in proportion the square root of the along-track focal plane array format size, for detector- and shot-noise-limited systems. CRISP also operates in the longwave infrared regime with noisy uncooled microbolometers, maximizing the SNR impact of the multiplex advantage. With the combined impact of the coding gain and the multiplex advantage, the result is that an uncooled CRISP system can obtain hyperspectral radiometric performance competitive with conventional pushbroom imaging spectrometers that require cooled detectors.

1.1 CRISP performance

A concept schematic of the CRISP optical setup is illustrated in Fig. 1(a). The first optical element is a prism (or grating) which disperses the scene [Fig. 1(b)], spectrally. A lens focuses the dispersed scene onto the coding mask [Fig. 1(c)], and a second lens / prism (or grating) pair recombines the dispersed light. One final lens focuses the now encoded scene onto the 2D focal plane array (FPA) of the imager. The resulting 2D radiance field sampled by the imager contains light summed from the whole spectrum, but with a position-dependent weighting as a function of wavelength. Platform motion produces a time-modulated intensity pattern which is sampled at the frame rate of the imager.

 

Fig. 1 CRISP measurement scheme. (a) Optical setup. The scene’s (b) colors are dispersed by a prism and focused onto an encoding mask, as depicted in (c). The colors are then recombined using a second prism and imaged by a microbolometer camera. After sufficient unique measurements (which are obtained as the dispersed scene travels across the mask due to platform motion), the collected data can be inverted to yield the hyperspectral cube as shown in (d). A potential space-based concept of operations for a CRISP system that relies on platform motion (e).

Download Full Size | PDF

The 3D spatial-spectral data cube $x$ is encoded in this measurement sequence $y$ with a known measurement matrix $\text{φ}$. With a well-designed mask (for example, a binary S-matrix code [16]), $\text{φ}$ is invertible or overdetermined when the number of measurements exceeds that number of spectral bands. Using the pseudoinverse, ${\text{φ}}^{+}$, one can estimate spectra, $\widehat{x}$, that minimize the mean squared errors (MSE) between the measurements and the measurements generated by the estimate based on the forward model $\text{φ}$ [see example decoded spatial-spectral data cube in Fig. 1(d)]. The dispersion can be performed in the along-track dimension or cross-track dimension, with suitable choice of coding mask (neither direction is preferred over the other from a performance standpoint). The reconstruction can be performed efficiently, in part by exploiting repetitive and cyclic patterns within $\text{φ}$, and / or S-matrix orthogonality to reduce computation. Cyclic S-matrices are orthogonal by design (plus a constant offset), simplifying computation of ${\text{φ}}^T\text{φ}$. The mask can also be designed with a repeating pattern of length $N_{\text{λ}}$, resulting in a measurement matrix with repeating blocks. Repetitive structure reduces the complexity of computing this product, as we are performing a multiplication only on a smaller matrix. The dominant computational cost is computing the pseudo inverse which is approximately $O\left(N_{\lambda }{}^3\right)$ Spectral recovery is decoupled across pixels resulting in moderate to small problem sizes, and can be parallelized as well. CRISP performance is expected to be optimal in platforms moving with constant velocity, making satellites an ideal application [see Fig. 1(e) for illustration of a potential space-based concept of operations]. Using the processing technique discussed in this paper, the spatial resolution would be determined by the frame rate of the imager. We are currently investigating advanced algorithms in which frame-rate limitations on spatial resolution are improved. We are also investigating algorithms to compensate for deviations from non-constant velocity such as platform jitter.

The CRISP approach for hyperspectral imaging enables two sensitivity gains, the traditional Fellgett or multiplex advantage found in other multiplexed measurement (e.g., Hadamard spectrometer) designs and an additional coding gain. These gains can be obtained simultaneously in practical designs and applications. We define the number of wavelength bins as $N_{\text{λ}}$, and the number of along-track measurements as $M$ ($M\ge N_{\text{λ}}$ is required for computing the pseudoinverse without introducing prior minimum norm assumptions about the signal). By combining the signal from multiple spectral bins in each measurement, CRISP gains a multiplex sensitivity advantage in SNR in the case where signal independent detector noise is dominant. For an S-matrix mask, where close to half of all spectral bins are combined in each measurement, the SNR gain is $\sqrt{N_{\text{λ}}}/2$ relative to a dispersive slit spectrometer that measures the signal in each spectral bin separately. In our analysis, SNR is defined as average signal intensity over the signal reconstruction error standard deviation. This result can be seen through the analysis presented in Harwitt and Sloan [16] which shows that the expected mean squared error for such a system is $MSE=\frac{{\sigma }^2}{M}Trace{\left({\text{φ}}^T\text{φ}\right)}^{-1}$ when the measurement noise is Gaussian with variance ${\sigma }^2$. The multiplex advantage result is obtained when $M=N_{\text{λ}}$ and the measurement matrix $\text{φ}$ is an S-matrix. For many practical applications such as atmosphere sounding, the number of wavelengths of interest can be exceeded by the number of measurements in part due to the format and frame rates achieved by modern focal plane arrays. For this case, when $M>N_{\text{λ}}$, an additional coding gain of $\sqrt{M/N_{\lambda }}$ is achievable. To see this, consider the case when $M/N_{\lambda }$ is an integer and the measurement matrix consists of multiple vertically concatenated S-matrices, i.e., $\text{φ}=\left[\begin{array}{c}{S}_1\\ ⋮\\ S_{M/N_{\lambda }}\end{array}\right]$. In the following section, we empirically demonstrate that this gain can also be obtained for $M$ sufficiently larger than $N_{\lambda }$ using random binary matrices. For detector-noise limited measurements, the multiplex and coding gains can be obtained simultaneously, resulting in an SNR enhancement for CRISP versus dispersive slit designs of $N_{\lambda }$. This combined sensitivity gain enables sensitive measurements using lower performance detectors such as uncooled microbolometers. Under shot-noise limited conditions, SNR enhancement relative to existing Fourier-transform infrared (FTIR) and dispersive slit spectrometers is possible as well via the coding gain.

Noting the above SNR gains, we see that CRISP has several advantages over existing technologies, including conventional dispersive spectrometers and FTIR spectrometers [17]. With a well-designed coding mask (one that minimizes $MSE=\frac{{\sigma }^2}{M}Trace{\left({\text{φ}}^T\text{φ}\right)}^{-1}$) and suitable estimation algorithm (a least squares estimator), a multiplex (Fellgett) and coding gain can be achieved, improving the system SNR beyond what is possible via alternative architectures assuming a sufficiently large focal plane array. The coding gain enables SNR improvements even under shot-noise limited imaging conditions, under which traditional multiplexed spectral imaging architectures can at best offer no loss relative to non-multiplexed designs. CRISP therefore provides a sensitivity gain in both shot-noise limited and detector-noise limited cases compared to existing technologies. Additionally, CRISP does not fundamentally require moving parts. These advantages translate directly into reduced size, weight, and power (SWaP) requirements, as expensive cooling components can be either be reduced or eliminated, and compact optical designs may be used. In addition to the sensitivity and SWaP advantages, CRISP can be reconfigured by swapping or changing the coding mask mid-mission. CRISP’s sensitivity gains can yield improvements in area coverage rates and allow for lower SWaP instruments with the same SNR as existing systems.

FTIR and dispersive spectrometer types cannot produce the coding gain, or can only with prohibitive difficulty (such as replicating large portions of the instrument). FTIR systems do possess a Fellgett advantage of $\frac{\sqrt{N_{\lambda }}}{4\sqrt{2}}\approx \frac{\sqrt{N_{\lambda }}}{2\times 2.8}$ over dispersive systems when detector-noise limited [17]. However, this advantage is less than CRISP by a factor of $2.8\sqrt{M/N_{\lambda }}$ [17] due to losses and inefficiencies resulting from the beam splitter, cosine modulation, and symmetric interferograms. In addition, imaging FTIR designs typically require complex moving parts, and must sweep through collection of the entire interferogram before the scene shifts out of view, which becomes impractical on a moving platform without gimbaling of the instrument. Spatial FTIR approaches that exploit platform motion exist and have shown great promise at reducing mechanical complexity versus conventional FTIR (for example [18],), albeit at the expense of additional radiometric losses.

2. Experiments and results

2.1 CRISP calibrated blackbody measurements

We demonstrate the CRISP spectrometer concept using commercial off-the-shelf components. The breadboard CRISP instrument is shown in Fig. 2 [schematic Fig. 2(a), and actual Fig. 2(b)]. We chose a design in the longwave infrared (LWIR) using an uncooled microbolometer camera (significantly noisier than state-of-the-art HgCdTe detectors) because a detector-noise limited system best showcases the CRISP advantages. The breadboard was built for concept demonstration and thus has not been optimized for optical throughput. It uses a 50 fps 640 x 480 resolution microbolometer camera with 17 µm pixels and 52 mm, f/1 lenses (aspheric triplets) by FLIR (model A655sc), ZnSe dispersive prisms (no antireflective coating), and an antireflective coated ZnSe coding mask with a random checkerboard pattern applied using contact lithography. The feature sizes of the coding mask are matched to the pixel size of the imager. While not optimal, the randomly patterned coding mask matches the performance of more well-designed masks (i.e., S-matrix) once a measurement has become ~5X overdetermined. This result was obtained through several simulations of the expected mean squared error ($MSE$) for a given mask weighing design $\text{φ}$, which can be calculated as [16] $MSE=\frac{{\sigma }^2}{M}Trace{\left({\text{φ}}^T\text{φ}\right)}^{-1}$.

 

Fig. 2 CRISP breadboard instrument, schematic (a) and actual (b).

Download Full Size | PDF

Results of these simulations are shown in Fig. 3(a) [with example matrices shown in Fig. 3(b)]. We conjecture that this occurs since a fixed number of randomly chosen vectors (columns of the measurement matrix) are more likely to be well separated as the dimension of the vector space increases.

 

Fig. 3 Simulations of the mean squared error (MSE) versus level of determination $\left(M/N_{\lambda }\right)$ of a CRISP measurement using a randomly patterned coding mask versus a more efficient S-matrix (Hadamard) coding mask (a). Both masks are binary (0’s and 1’s), with examples shown on the right (b).

Download Full Size | PDF

The mask “truth” (used to decode the hyperspectral data cube) is experimentally determined by peering into an integrating sphere illuminated by a single longitudinal mode QCL with $\lambda =8.85$ µm with the CRISP breadboard. To capture the mask “truth”, we average together one thousand frames from the integrating sphere with the QCL turned on, and subtract from this a background which is obtained by similarly averaging another one thousand frames with the QCL turned off. We note that the 17 µm pixels of our imager and the same sized features in our coding mask are smaller than the diffraction-limited point spread function (PSF) in the LWIR (~20 – 34 um), and as a result, our coding mask appears gray instead of perfectly binary. The optical design of the CRISP breadboard yields $N_{\text{λ}}=67$ spectral bins across 7.7 – 14 µm, a resolution of $\Delta \lambda =~94$ nm.

Spectra of an extended source blackbody (CI-Systems SR-800) are shown in Fig. 4. Figure 4(a) shows a single frame of the calibrated blackbody source at 100 °C imaged through the CRISP system. For these experiments, the object is positioned ~3 m from the CRISP system. We translate the blackbody source via a motorized translation stage to simulate platform motion (the white arrow indicates the direction of motion). The velocity of the translation stage is set so that it moves at a rate of $v=\left(1 sample distance\right)\cdot \left(frame rate\right)$. A total of $M=335$ frames of the blackbody source are captured as it translates across the image plane, making our data sets 5X overdetermined. Figure 4(b) shows the resultant raw CRISP spectra (after decoding the data set) of the calibrated blackbody source at 40 °C, 100 °C, and 138 °C. After adjusting for the wavelength specific sensitivity of the FLIR microbolometer, the resulting spectral shape matches perfectly to blackbody radiation theory, as shown in Fig. 4(c). To demonstrate the spectral resolution of our CRISP breadboard, we placed narrow bandpass filters (Spectrogon Inc.) between the CRISP system and the blackbody source. The resulting spectral shapes measured by CRISP agree well with the filter transmission data provided by the vendor, as shown in Fig. 4(d). We note that the spectra shown in Fig. 4 is after we subtract off a spectrum captured with the input of the CRISP system blocked. This technique is common in many types of spectrometers and helps to eliminate any trends in the baseline or any clutter in the spectrum due to narcissus or other effects.

 

Fig. 4 CRISP measurements of a calibrated blackbody. (a) 100 °C blackbody (CI-Systems SR-800) as imaged by CRISP. Arrow indicates direction of motion of the scene. (b) Raw blackbody spectra: 40 °C, 100 °C, and 138 °C (100 px by 100 px averages). (c) Blackbody spectra adjusted for focal plane array wavelength sensitivity. Solid black lines represent theoretically calculated blackbody curves (Planck’s law) for the given temperatures. (d) Blackbody target as seen through 7.90 µm and 10.27 µm Spectrogon narrow bandpass filters. Solid black lines represent filter transmission data provided by the vendor.

Download Full Size | PDF

2.2 Comparison to pushbroom spectrometer

As mentioned previously, a major advantage of CRISP is that the coding gains enable enhanced sensitivity for both shot-noise limited and detector-noise limited systems. These gains become apparent when we compare the CRISP spectrometer performance to a pushbroom style spectrometer (see Fig. 5). The pushbroom style spectrometer we use for the comparison is the CRISP breadboard reconfigured for pushbroom operation, i.e., with the first prism replaced with an uncoated ZnSe window of similar size (to keep the optical throughput consistent) and the coding mask replaced with a slit whose width is matched to the pixel size of the imager. After 335 frames, CRISP outperforms the pushbroom by ~5X for a 40 °C blackbody, and by ~3X for a 138 °C blackbody. All CRISP measurements are compared to a single-shot of the pushbroom spectrometer.

 

Fig. 5 SNR of the CRISP spectrometer compared to same spectrometer reconfigured to operate in a traditional pushbroom measurement scheme (i.e., remove the first dispersive prism, and replace the coding mask with a slit). The CRISP SNR gains grow as $\sqrt{M}$ (shown as solid lines). The shaded regions depict error bars ( ± σ), and the dashed lines represent theoretical upper and lower bounds of performance improvements (see text for details).

Download Full Size | PDF

The measured performance enhancements at 335 frames for both the 40 °C and 138 °C blackbody are within the theoretical upper-bound expected for a detector-noise limited measurement $\left(\sqrt{M}/2 \approx 9\right)$, and the theoretical lower-bound expected for a shot-noise limited measurement $\left(\sqrt{M/\left(2\cdot N_{\text{λ}}\right)}\approx 1.5; N_{\text{λ}}=67\right)$. Following [19], we predict 40 – 70 mK effective shot noise for a 40 °C and 138 °C blackbody viewed through our CRISP breadboard with a ~11% optical throughput. Our measured detector noise is 100 – 200 mK, which puts our measurements in a modestly detector-noise limited regime. Regardless of whether we operate in a shot-noise limited or detector-noise limited regime, the above equations predict that SNR will improve as $\sqrt{M}$, which is clearly observed in Fig. 5. We note that the relative root mean square deviation from the 138 °C blackbody was 22% for a single pixel with 335 frames of CRISP processing. Larger format imagers would thus be expected to increase the CRISP performance. We believe that achievement of the detector-noise limited performance can be approached via a combination of higher optical throughput and a more carefully measured mask function used to decode the hyperspectral cube.

2.3 CRISP spectroscopic analysis of trace gases

To explore CRISP’s ability for spectroscopic analysis of trace gases, we filled a thin polyethylene bag (~80% transmissive in the LWIR) with a trace amount (~2000 ppm) of 1,1,1,2-tetrafluoroethane (R-134a), buffered with nitrogen to 1 atm. First, we measure the transmission spectrum of the room temperature R-134a. The bag with trace gas is positioned between the CRISP breadboard and a blackbody source set to 100 °C during a data collect. The transmission spectrum measured by CRISP (referenced to a polyethylene bag filled with only nitrogen) is shown in Fig. 6(a), and is found to closely resemble the transmission spectrum computed from data found in the PNNL database [20]. Next, we measure the emission spectrum of the same bag of R-134a heated to a temperature of ~50 °C. The measured emission spectrum of the heated R-134a is shown in Fig. 6(b). To achieve this result, we subtracted the room temperature blackbody spectrum baseline. Again, the result matches closely to the PNNL database. We note that the spectra displayed in Fig. 6 are the result of 100 px by 100 px averaging.

 

Fig. 6 CRISP spectroscopic analysis of trace gases. (a) CRISP transmission spectrum of ~2000 ppm of 21 °C (room temperature) R-134a and (b) CRISP emission spectrum of the same gas at ~50 °C. Lines are PNNL transmission data [20]. (c) Two polyethylene bags, one filled with air, and one that contains ~2000 ppm of R-134a mixed with air are placed side-by-side in front of a 100 °C blackbody is shown on the left. The R-134a signal heat map (depth of the $\lambda =8.4$ µm transmission line) is shown on the right, with a white dotted line depicting the outline of the blackbody source.

Download Full Size | PDF

As an additional demonstration, we show the ability to detect and locate a trace gas within the scene. For this experiment, we place our bag of trace R-134a gas adjacent to a bag full of room air, both of which are positioned directly in front of a 100 °C blackbody radiator (see Fig. 6(c), left). The bag of air in this case simply exists as the null experiment. We collect a data set and produce a heat map from the data that shows the location of the R-134a in the scene. The heat map is produced by simply plotting the height of a known peak in the transmission spectrum of the R-134a gas. The result is shown in Fig. 6(c), right. A white dotted border depicts the perimeter of the extended blackbody source.

3. Summary

In summary, we have successfully demonstrated a computational imaging spectrometer with significant performance gains compared to existing architectures that unlocks the ability to build compact, sensitive systems with less-expensive components (e.g., uncooled microbolometers). Specifically, after 335 frames, CRISP outperforms the pushbroom by ~5X for a 40 °C blackbody, which is within 2X of the theoretical upper-bound expected for a detector-noise limited measurement. Blackbody radiation spectra were obtained from a benchtop, proof-of-concept CRISP spectrometer and are found to be in good agreement with theory. In addition, transmission / emission spectra from a trace gas (R-134a) were obtained and are found to match well with published data from PNNL. CRISP performance gains are measured with respect to the performance of a pushbroom spectrometer of identical hardware components, and are found to be close to our theoretical predictions. The methods described here provide a new architecture for developing compact, sensitive hyperspectral imaging systems, with particular appeal for use on moving platforms such as small satellites.