## Abstract

Coded apertures with random patterns are extensively used in compressive spectral imagers to sample the incident scene in the image plane. Random samplings, however, are inadequate to capture the structural characteristics of the underlying signal due to the sparsity and structure nature of sensing matrices in spectral imagers. This paper proposes a new approach for super-resolution compressive spectral imaging via adaptive coding. In this method, coded apertures are optimally designed based on a two-tone adaptive compressive sensing (CS) framework to improve the reconstruction resolution and accuracy of the hyperspectral imager. A liquid crystal tunable filter (LCTF) is used to scan the incident scene in the spectral domain to successively select different spectral channels. The output of the LCTF is modulated by the adaptive coded aperture patterns and then projected onto a low-resolution detector array. The coded aperture patterns are implemented by a digital micromirror device (DMD) with higher resolution than that of the detector. Due to the strong correlation across the spectra, the recovered images from previous spectral channels can be used as *a priori* information to design the adaptive coded apertures for sensing subsequent spectral channels. In particular, the coded apertures are constructed from the *a priori* spectral images via a two-tone hard thresholding operation that respectively extracts the structural characteristics of bright and dark regions in the underlying scenes. Super-resolution image reconstruction within a spectral channel can be recovered from a few snapshots of low-resolution measurements. Since no additional side information of the spectral scene is needed, the proposed method does not increase the system complexity. Based on the mutual-coherence criterion, the proposed adaptive CS framework is proved theoretically to promote the sensing efficiency of the spectral images. Simulations and experiments are provided to demonstrate and assess the proposed adaptive coding method. Finally, the underlying concepts are extended to a multi-channel method to compress the hyperspectral data cube in the spatial and spectral domains simultaneously.

© 2020 Chinese Laser Press

## Corrections

Chang Xu, Tingfa Xu, Ge Yan, Xu Ma, Yuhan Zhang, Xi Wang, Feng Zhao, and Gonzalo R. Arce, "Super-resolution compressive spectral imaging via two-tone adaptive coding: publisher’s note," Photon. Res.**8**, 892-892 (2020)

https://www.osapublishing.org/prj/abstract.cfm?uri=prj-8-6-892

7 April 2020: A typographical correction was made to the author affiliations.

## 1. INTRODUCTION

Hyperspectral imaging acquires the spatio-spectral data cube of a scene over dozens to hundreds of narrow spectral bands [1,2]. Benefiting from the abundant spectral information, hyperspectral imaging has been used in a diverse range of applications from precision agriculture [3], food safety [4], medical diagnosis [5], to mineral mapping [6], and so on. Based on the data cube acquisition modes, hyperspectral imaging approaches can be classified into four categories known as whiskbroom [7], pushbroom [8], snapshot [9], and staring [10] approaches. Specifically, whiskbroom and pushbroom approaches are based on scanning in pointwise and linewise fashions, respectively. Snapshot approaches are proposed to multiplex the three-dimensional (3D) data cube onto a two-dimensional (2D) sensor, which is able to preserve the 2D spatial information as images typically have underlying spatial properties. However, there is a trade-off between the scanning process and spatial resolution [11]. To address this issue, staring hyperspectral imagers are well defined to capture the 2D spatial information of the scene at once, and sequentially scan the spectra using a rotating filter wheel or a tunable filter [12]. As they flexibly select the interested spectral range, staring approaches have been widely used in the field of hyperspectral imaging [13].

Recently, liquid crystal tunable filters (LCTFs) have been used as spectral bandpass filters in staring hyperspectral imagers attributed to their advantages of simple design, versatility, low wavefront distortion, flexible throughput control, faster speed, and large aperture with wide field of view [14–17]. However, the conventional LCTF-based hyperspectral imager is limited by the constant spatial resolution of its detector. In addition, high-resolution detectors may not be available in some cases [18]. Moreover, the substantial volume of hyperspectral images causes a dilemma for subsequent transmission and storage of the overall data [19]. The fast-emerging compressive sensing (CS) theory is well defined as a useful tool to reconstruct high-dimensional signals from much fewer multiplexed measurements than those required by the Nyquist sampling theorem, based on the sparse representation assumption [20–22]. Therefore, high-resolution images can be recovered from measurements on the low-resolution detector using the CS method [23]. In addition, compressive data are obtained during the acquisition stage, requiring less volume of total measurements than the original data cube. This benefits data transmission and storage. With these motivations, Wang *et al.* applied CS to develop a super-resolution LCTF-based hyperspectral imaging approach [24]. In Ref. [24], the spectral images were modulated in the spatial domain by a high-resolution random coded aperture, and then projected onto a low-resolution detector. However, random coding is suboptimal due to the sparse sensing matrix of the system, and thus the reconstruction performance can be improved with proper code design. Remarkably, the coded aperture determines the structure of the projection matrix in compressive sampling; thus it plays an important role in improving the reconstruction accuracy of the hyperspectral data cube [25,26]. The design of coded apertures has therefore attracted wide attention from researchers, remaining an important open issue in this field [27,28].

In the past, a set of numerical approaches has been developed to optimize the coded apertures based on the restricted isometry property [29,30] or empirical design rules [31,32]. However, the coded apertures in these methods are pre-designed before the data acquisition and cannot self-adapt to the characteristics of the underlying signal. In addition, the optimization methods entail high computational loads to the design process of spectral imaging systems. Recently, side information has been used to exploit non-local similarity [33], structural sparsity [34], rank minimization [35], etc. [36–39] to aid the reconstruction of undersampled signals. Learning from this concept, other approaches have been proposed to design the coded apertures based on *a priori* information of the underlying spectral images [40,41]. Nevertheless, these approaches require additional optical paths or auxiliary sensors to detect the side information of the spectral scene, which inevitably increase the complexity of the systems. In Ref. [42], Yang *et al*. proposed an adaptive CS sampling strategy based on the dictionary learned from the training data. However, the designed projection matrix in this method is not binary, which makes it difficult to implement in hardware.

This paper proposes a two-tone adaptive CS (TACS) framework that can be easily implemented by coded apertures to enhance the spatial resolution and image quality of compressive spectral imaging systems. The sketch of the proposed hyperspectral imager is illustrated in Fig. 1. The light rays emitted from the target are incident upon the LCTF, which is used to successively select different spectral channels. The LCTF is considered an ideal spectral filter with narrow bandwidth that outputs a monochromatic image corresponding to its center wavelength [43]. The output of the LCTF is collected by the imaging lens 1, and then spatially modulated by a high-resolution coded aperture that is generated according to the TACS method. In hardware, the coded aperture patterns are implemented by the digital micromirror device (DMD). The DMD consists of an array of micromirrors that are individually controllable to generate different binary coded patterns [44–46]. Finally, the imaging lens 2 focuses the reflected light rays onto a low-resolution complementary metal oxide semiconductor (CMOS) detector. The CMOS detector is used to collect the compressive measurements, from which a high-resolution spectral image selected by the LCTF can be reconstructed. In order to obtain the 3D spectral data cube, the center wavelength of the LCTF needs to be tuned gradually to scan different spectral slices. Note that the proposed imager encodes the spectral images in the spatial domain, and thus spatial super-resolution is achieved while the number of acquired spectral channels remains the same. Because most of the hyperspectral images exhibit strong correlation among spectral channels, the spectra are assumed to be smooth [26,47]. Therefore, the recovered images from the previous spectral channels can be used as the reference images to construct the adaptive coded apertures for the following spectral channels. Given an *a priori* spectral image, a pair of complementary coded aperture patterns is generated by a reverse thresholding operation to respectively extract the structural characteristics of bright and dark regions. Since no extra side information of the spectral scene is needed, the proposed method does not increase the complexity of the imaging system. In addition, the computational complexity to calculate the adaptive coded apertures is negligible, compared to that of the conventional coded aperture optimization methods and the reconstruction process of the spectral images.

In order to further improve the reconstruction performance, multiple snapshots are carried out to increase the number of measurements. In the data acquisition process, the center wavelength of the LCTF is first fixed, and then different coded aperture patterns are switched successively on the DMD. In this paper, different coded aperture patterns are generated from one reference image using the random dithering method [48]. It is worth highlighting that the proposed TACS framework is proved theoretically to be valid based on the mutual-coherence criterion. In a statistical sense, the projection matrix for TACS is demonstrated to improve the sensing efficiency of the underlying signal. Simulation and experimental results verify the superiority of the proposed TACS coding method in terms of the imaging performance over the random coding method. In addition, comparison with other adaptive coding methods is provided to further assess the TACS coding method.

It should be noted that for the TACS method, each measurement encompasses the information of one spectral channel; thus only spatial compression is conducted. In order to compress the hyperspectral data cube in both spatial and spectral domains, the underlying concepts are extended to a multi-channel TACS method. During one integration time interval of the detector, the LCTF is switched for several times to encompass a series of spectral channels into one snapshot. In each snapshot, different spectral channels are modulated by different coded aperture patterns, then multiplexed and integrated on the CMOS detector. By doing so, each measurement includes the information of multiple spectral channels, thus increasing the compression capacity of the system. A set of simulations is conducted to prove the feasibility of the proposed multi-channel TACS method.

The remainder of this paper is organized as follows. Section 2 introduces the TACS framework. The proposed two-tone adaptive compressive hyperspectral imaging system is described in Section 3. Simulation and experimental results of the TACS method are provided in Section 4 and Section 5, respectively. The multi-channel TACS method is proposed and assessed in Section 6. Section 7 concludes the paper with some remarks.

## 2. TWO-TONE ADAPTIVE CS FRAMEWORK

#### A. CS Principles

It is known that most natural signals or images are sparse in some representation basis. Suppose $\overrightarrow{X}\in {\mathbb{R}}^{N\times 1}$ is a $K$-sparse signal in the basis $\mathbf{\Psi}=[{\overrightarrow{\psi}}_{1},{\overrightarrow{\psi}}_{2},\dots ,{\overrightarrow{\psi}}_{N}]\in {\mathbb{R}}^{N\times N}$; thus, it can be expressed as $\overrightarrow{X}=\mathbf{\Psi}\overrightarrow{\mathrm{\Theta}}$, where $\mathbf{\Psi}$ is the sparse basis, and $\overrightarrow{\mathrm{\Theta}}\in {\mathbb{R}}^{N\times 1}$ is the sparse coefficient vector including only $K$ ($K\ll N$) significant elements. Commonly used sparse bases include Fourier transform basis, discrete cosine basis (DCT), wavelet basis, and so on [49]. Let $\overrightarrow{Y}$ represent the compressive measurements of $\overrightarrow{X}$ given by $\overrightarrow{Y}=\mathbf{\Phi}\overrightarrow{X}=\mathbf{\Phi}\mathrm{\Psi}\overrightarrow{\mathrm{\Theta}}$, where $\mathbf{\Phi}={[{\overrightarrow{\varphi}}_{1},{\overrightarrow{\varphi}}_{2},\dots ,{\overrightarrow{\varphi}}_{L}]}^{T}\in {\mathbb{R}}^{L\times N}$ is the projection matrix with $L\ll N$. According to CS theory, the sparse signal $\overrightarrow{X}$ can be reconstructed from a few compressive measurements by solving the following inverse problem [20]:

It then becomes natural to ask what properties the projection matrix $\mathbf{\Phi}$ needs to satisfy to recover the signal successfully and accurately. Consider first the most general case where no *a priori* information of $\overrightarrow{X}$ is known. It has been shown that if $\mathbf{\Psi}$ is incoherent to $\mathbf{\Phi}$, $\overrightarrow{X}$ can be successfully recovered when the number of measurements satisfies $L=C\xb7K\xb7\mathrm{log}N\ll N$, where $C\ge 1$ is an oversampling factor [53]. The mutual coherence between $\mathbf{\Psi}$ and $\mathbf{\Phi}$ can be evaluated by

However, in some other scenarios, some *a priori* information of $\overrightarrow{X}$ is known beforehand. For instance, an approximate (not exact) observation of the original signal is available, which can be exploited to improve the coding and reconstruction performance. To this end, Ma *et al.* introduced a design rule for the projection matrices based on the approximate observation of original signals [55]. First, the columns of the basis $\mathbf{\Psi}$ are separated into two sets: $\mathrm{\Upsilon}=\{{\overrightarrow{\psi}}_{l(1)},{\overrightarrow{\psi}}_{l(2)},\dots ,{\overrightarrow{\psi}}_{l(K)}\}$ and $\overline{\mathrm{\Upsilon}}$, where $l(i)$ indicates the location of the column corresponding to the $i$th non-zero coefficient, and $\overline{\mathrm{\Upsilon}}$ is the complementary set of $\mathrm{\Upsilon}$. Accordingly, the mutual-coherence metric is divided into two parts: ${\mu}_{\mathrm{\Upsilon}}=\underset{{\overrightarrow{\psi}}_{j}\in \mathrm{\Upsilon}}{\mathrm{max}}\{|\u27e8{\overrightarrow{\varphi}}_{i},{\overrightarrow{\psi}}_{j}\u27e9{|}^{2}\}$ and ${\mu}_{\overline{\mathrm{\Upsilon}}}=\underset{{\overrightarrow{\psi}}_{j}\in \overline{\mathrm{\Upsilon}}}{\mathrm{max}}\{|\u27e8{\overrightarrow{\varphi}}_{i},{\overrightarrow{\psi}}_{j}\u27e9{|}^{2}\}$. Given some *a priori* information of $\overrightarrow{X}$, it shows that a good projection matrix should maximize the difference between ${\mu}_{\mathrm{\Upsilon}}$ and ${\mu}_{\overline{\mathrm{\Upsilon}}}$ [55].

#### B. TACS Projection Matrix

Monotone adaptive projection matrices were proposed in computational lithography to satisfy the aforementioned design rule, i.e., to maximize the difference between ${\mu}_{\mathrm{\Upsilon}}$ and ${\mu}_{\overline{\mathrm{\Upsilon}}}$ [55]. Suppose the observation of the original signal is given by

where $\overrightarrow{\epsilon}\in {\mathbb{R}}^{N\times 1}$ is the noise vector. The elements of $\overrightarrow{\epsilon}$ are independent identical random variables obeying the Gaussian distribution $\mathcal{N}(0,{\sigma}_{X}^{2})$ with zero-mean and variance ${\sigma}_{X}^{2}$. The monotone adaptive projection matrix with $\pm 1$ elements can be constructed by thresholding the observation $\overrightarrow{S}$. However, the negative elements in the projection matrix cannot be physically implemented by the coded aperture used in the imaging system. That is because transmissive or reflective coded apertures modulate the amplitude of the incident wavefront without changing the phase.To overcome this limitation, this paper proposes a TACS projection matrix with non-negative elements. The rows of the projection matrix $\mathbf{\Phi}$ are independently generated by applying two-tone thresholding operations on the observation $\overrightarrow{S}$. Supposing the measurement number $L$ is an even number, then the element of $\mathbf{\Phi}$ in the $i$th row and $j$th column is defined as

The projection matrix defined in Eq. (4) constitutes two sub-projection matrices with all elements equal to 0 or 1. Figure 2 provides an intuitive illustration of different projection matrices ($N=401,L=100$) for the one-dimensional signal in Fig. 2(a). Figure 2(b) shows the conventional random projection matrix with Bernoulli sampling. Figure 2(c) illustrates the TACS projection matrix. In Figs. 2(b) and 2(c), the white and black pixels have the values of 1 and 0, respectively. Note that the TACS projection matrix extracts some structural characteristics of the original signal in Fig. 2(a), while the random projection matrix does not. In particular, the TACS projection matrix includes two sub-matrices. The top-half and bottom-half sub-matrices respectively capture the structural characteristics of the signal components beyond and below the threshold values. Thus, the overall compressive measurements capture the features of the entire signal.

Next, we use the design rule described in Subsection 2.A to assess the merit of the TACS projection matrix. Note that the voxels in hyperspectral images represent light intensities and thus they are always non-negative. This property will be used in the proof. In Appendix A, the proposed TACS projection matrix is proved to make the mean values of ${\mu}_{\mathrm{\Upsilon}}$ and ${\mu}_{\overline{\mathrm{\Upsilon}}}$ satisfy the following properties:

Next, the superiority of the TACS projection matrix is verified by numerical simulations. Two signals with different dimensions are used as the original signals to be measured. Table 1 compares the mean values of ${\mu}_{\mathrm{\Upsilon}}$ and ${\mu}_{\overline{\mathrm{\Upsilon}}}$ obtained by the random projection matrices and TACS projection matrices. For each projection method, we repeat the simulations 100 times. In contrast to the random projection matrices, the TACS projection matrices further increase the difference between ${\mu}_{\mathrm{\Upsilon}}$ and ${\mu}_{\overline{\mathrm{\Upsilon}}}$, and satisfy the design rule better. Thus, the TACS projection matrix is apt to retain more information of the original signal than the random projection matrix, and benefits in improving the reconstruction performance.

## 3. SUPER-RESOLUTION HYPERSPECTRAL IMAGER USING TACS CODED APERTURE

As shown in Fig. 1, the LCTF-based hyperspectral imager consists of an LCTF, a DMD, and a CMOS detector. The input spatio-spectral data cube, ${f}_{1}(x,y,\lambda )$, is scanned in the spectral domain by tuning the center wavelength of the LCTF. This paper assumes that the LCTF is an ideal spectral filter, the output of which is considered a monochromatic image corresponding to the center wavelength of the LCTF. Suppose the transmission function of the LCTF is denoted as ${T}_{s}^{{n}_{\lambda}}(\lambda )$. The output monochromatic image is expressed as ${f}_{1}^{{n}_{\lambda}}(x,y)={f}_{0}(x,y,\lambda )\xb7{T}_{s}^{{n}_{\lambda}}(\lambda )$, where the superscript ${n}_{\lambda}\text{\hspace{0.17em}}({n}_{\lambda}=1,2,\dots ,{N}_{\lambda})$ represents the order number of the spectral channel. Then, the spectral images passing through the LCTF are spatially modulated by the binary coded aperture patterns, which are realized by the DMD. The DMD consists of an array of micromirrors. The tilt angle of each micromirror can be independently adjusted to change the direction of the reflected light rays. The block/unblock coded aperture patterns can be generated by flipping the corresponding micromirrors, and only the light rays reflected from the unblock pixels are collected to the main light path. The compressive measurement obtained on the detector plane can be written as

In order to further improve the hyperspectral imaging performance, multiple snapshots are taken to increase the number of measurements. As shown in Fig. 3, the center wavelength of the LCTF is first fixed to select a certain spectral channel. In each spectral channel, the coded aperture pattern is switched for $L$ times to capture $L$ different compressive measurements. Afterwards, we tune the center wavelength of the LCTF to the next spectral channel and repeat the measurement process mentioned above. After scanning all the spectral channels, the measurement procedure is terminated. In this case, the coded aperture pattern is denoted as ${T}_{c}^{l}(x,y)$, where $l=1,2,\dots ,L$ indexes the order number of snapshots. Thus, the $l$th compressive measurement in the ${n}_{\lambda}$th spectral channel, referred to as ${g}^{{n}_{\lambda},l}(x,y)$, is formulated as

To simplify the analysis of the system, the imaging model in Eq. (8) is reformulated into a discrete form. The hyperspectral data cube is gridded into ${N}_{x}\times {N}_{y}\times {N}_{\lambda}$ voxels. Each voxel is denoted as ${F}_{{n}_{x},{n}_{y},{n}_{\lambda}}$, where ${n}_{x},{n}_{y}$ represent the spatial coordinates (${n}_{x}=1,2,\dots ,{N}_{x}$ and ${n}_{y}=1,2,\dots ,{N}_{y}$). The requirement for super-resolution is that the pitch resolution of the DMD should be higher than that of the CMOS detector. Specifically, let $\mathrm{\Delta}c$ and $\mathrm{\Delta}d$ be the pitch sizes on the coded aperture and the detector, respectively. Then, the ratio of resolution between the coded aperture and the detector is defined as $R=\mathrm{\Delta}d/\mathrm{\Delta}c$. Assume $R$ is a positive integer larger than 1. That is, the dimension of the coded aperture is ${N}_{x}\times {N}_{y}$, and the dimension of the detector is ${M}_{x}\times {M}_{y}$, where ${N}_{x}/{M}_{x}={N}_{y}/{M}_{y}=R$. Thus, the overall compression ratio is defined as ${\gamma}_{o}={\gamma}_{c}\xb7L={(1/R)}^{2}\xb7L$, where ${\gamma}_{c}$ refers to the compression ratio for one snapshot. In the $l$th snapshot, the measurement on the $({m}_{x},{m}_{y})$th pixel of the detector is represented by ${G}_{{m}_{x},{m}_{y}}^{{n}_{\lambda},l}$. The discrete version of the imaging model in Eq. (8) can be written as

Alternatively, the hyperspectral data cube can be expressed as its vector representation across ${N}_{\lambda}$ spectral channels. Let ${\overrightarrow{f}}_{{n}_{\lambda}}\in {\mathbb{R}}^{{N}_{x}\xb7{N}_{y}\times 1}$ denote monochromatic image in the ${n}_{\lambda}$th spectral channel, which is sparse in a basis $\mathbf{\Psi}\in {\mathbb{R}}^{({N}_{x}\xb7{N}_{y})\times ({N}_{x}\xb7{N}_{y})}$, such that ${\overrightarrow{f}}_{{n}_{\lambda}}=\mathbf{\Psi}{\overrightarrow{\mathrm{\Theta}}}_{{n}_{\lambda}}$. Assume ${\overrightarrow{g}}_{{n}_{\lambda}}\in {\mathbb{R}}^{L\xb7{M}_{x}\xb7{M}_{y}\times 1}$ is the vectorized representation of the measurement $G$ in the ${n}_{\lambda}$th spectral channel. Following this notation, the imaging model in Eq. (9) can be rewritten as

Thus, the coefficient vector of the hyperspectral image in the ${n}_{\lambda}$th spectral channel can be reconstructed from the following inverse optimization problem:

Next, we describe how to generate the TACS coded apertures during the measurement process. Because the hyperspectral images exhibit strong correlation among spectral channels, that is, the spectra are smooth, the images in the adjacent spectral channels should have similar structural characteristics [56]. Therefore, we can divide the entire spectra into several sub-groups, each of which includes several adjacent spectral channels. As shown in Fig. 4, for each sub-group, one reference spectral channel is selected and reconstructed using a random coded aperture. After that, the reconstructed reference image is used as the *a priori* information to construct the TACS coded aperture for all spectral channels in this sub-group.

To explain this process more clearly, let us take a special case, i.e., ${M}_{x}\xb7{M}_{y}=1$. The images within the ${N}_{u}$th and ${N}_{v}$th spectral channels belonging to the same sub-group are denoted by ${\mathbf{I}}_{{N}_{u}}\in {\mathbb{R}}^{{N}_{x}\times {N}_{y}}$ and ${\mathbf{I}}_{{N}_{v}}\in {\mathbb{R}}^{{N}_{x}\times {N}_{y}}$, respectively. Assume the ${N}_{u}$th spectral channel is the reference channel. Then, the reconstruction of ${\mathbf{I}}_{{N}_{u}}$ denoted by ${\widehat{\mathbf{I}}}_{{N}_{u}}$ is used as the *a priori* information to design the TACS coded aperture for the spectral image ${\mathbf{I}}_{{N}_{v}}$. Define an operator $\mathrm{vec}(\xb7)$ that transforms an image into its vectorized representation by stacking all the columns. Due to the similarity between the adjacent spectral images, we have $\mathrm{vec}({\widehat{\mathbf{I}}}_{{N}_{u}})\approx \mathrm{vec}({\mathbf{I}}_{{N}_{v}})+\overrightarrow{\epsilon}$, where $\overrightarrow{\epsilon}$ denotes the error term. Compared to Eq. (3), $\mathrm{vec}({\mathbf{I}}_{{N}_{v}})$ and $\mathrm{vec}({\widehat{\mathbf{I}}}_{{N}_{u}})$ can be regarded as the original signal $\overrightarrow{X}$ and its approximate observation $\overrightarrow{S}$, respectively. Let $N={N}_{x}\xb7{N}_{y}$, and we can generate $L$ different adaptive projection vectors with length of $N$ according to Eq. (4). After that, these projection vectors are inversely stacked into $L$ different 2D TACS coded aperture patterns with dimension of ${N}_{x}\times {N}_{y}$. In practice, we can reconstruct every $R\times R$ sub-region on the spectral images independently, and then stitch them up to obtain the full spectral images. In this case, ${\mathbf{\Phi}}_{{n}_{\lambda}}^{i}$ in Eq. (11) is generated in accordance with the same process as mentioned above. Each row of ${\mathbf{\Phi}}_{{n}_{\lambda}}^{i}$ is denoted by ${\varphi}_{{n}_{\lambda}}^{i,l}\in {\mathbb{R}}^{1\times {R}^{2}}$, which represents the transmission function for the $i$th spatial pixel on the detector in the $l$th snapshot. That is, ${\mathbf{\Phi}}_{{n}_{\lambda}}^{i}=[{({\varphi}_{{n}_{\lambda}}^{i,1})}^{T},{({\varphi}_{{n}_{\lambda}}^{i,2})}^{T},\dots {({\varphi}_{{n}_{\lambda}}^{i,L})}^{T}{]}^{T}$. Then, ${\varphi}_{{n}_{\lambda}}^{i,l}$ can be stacked into an $R\times R$ matrix denoted by ${\mathbf{\Pi}}_{{n}_{\lambda}}^{i,l}$. Note that ${\mathbf{\Pi}}_{{n}_{\lambda}}^{i,l}$ represents the $R\times R$ sub-region on the coded aperture for the $l$th snapshot. Stitching up the sub-matrices ${\mathbf{\Pi}}_{{n}_{\lambda}}^{1,l},{\mathbf{\Pi}}_{{n}_{\lambda}}^{2,l},\dots ,{\mathbf{\Pi}}_{{n}_{\lambda}}^{{M}_{x}\xb7{M}_{y},l}$ together, we can obtain the $l$th coded aperture pattern.

## 4. SIMULATION RESULTS

#### A. Simulation Implementation of the TACS Coding Method

In this subsection, two sets of simulations are conducted using the real hyperspectral data, where the reconstruction performance obtained by TACS coded apertures and random coded apertures is compared. In the following simulations, the Lego figures are used as the target, and the original hyperspectral data cube consists of $256\times 256$ pixels in the spatial domain and 24 spectral bands. More specifically, the center wavelengths of the 24 spectral bands are 450 nm, 459 nm, 467 nm, 476 nm, 485 nm, 493 nm, 502 nm, 511 nm, 520 nm, 528 nm, 537 nm, 546 nm, 554 nm, 563 nm, 572 nm, 580 nm, 589 nm, 598 nm, 607 nm, 615 nm, 624 nm, 633 nm, 641 nm, and 650 nm, respectively. The intensity of the test data is normalized to the range of [0,1]. The coded aperture includes $256\times 256$ pixels, while the detector constitutes $32\times 32$ pixels. Thus, every $8\times 8$ pixels on the coded aperture correspond to one detector pixel and ${\gamma}_{c}={(1/8)}^{2}$. The measurement error on the detector is emulated by the white Gaussian noise with signal-to-noise ratio (SNR) level of 30 dB. In the reconstruction process, the 2D-IDCT basis is used to sparsely represent the spectral images. All of the calculations are carried out using MATLAB R2016a on a server with Intel Xeon E3-1505M v5 2.8 GHz processor, and 64 GB memory.

According to Section 3, we first divide the entire spectra evenly into four sub-groups and reconstruct one reference spectral channel in each sub-group. At this point, the number of snapshots $L$ is set to 32. Figures 5(a) and 5(b) show an original reference image and its reconstruction result using a random coded aperture. Taking the recovered reference image as the *a priori* information, the TACS coded aperture patterns are generated for all spectral channels in the same sub-group. A pair of the TACS coded aperture patterns is shown in Figs. 5(c) and 5(d). It can be easily observed that these two coded aperture patterns are approximately complementary, and respectively extract the structural information within bright and dark regions. Next, the TACS coded apertures are used to modulate and reconstruct the spectral images with multiple snapshots ($L=12$). Thus, the overall compression ratio is ${\gamma}_{o}=0.1875$.

Figure 6 presents the spectral images obtained by the conventional system and the reconstructed spectral images obtained by different kinds of coded apertures for comparison. Figure 6(a) illustrates the high-resolution original images of four spectral channels in the data cube. Figure 6(b) shows the low-resolution images captured by the conventional system, which cannot achieve super-resolution without using the theory of CS. The low-resolution images are simulated by downsampling the original images along both horizontal and vertical directions with a scaling factor of 8, and their spatial resolution is $32\times 32$ pixels. And they are presented to demonstrate the resolution improvement brought by the compressive imaging system. Figures 6(c) and 6(d) show the reconstructed spectral images consisting of $256\times 256$ pixels using TACS coded apertures and random coded apertures, respectively. Note that in Fig. 6(d), the spectral images are reconstructed with 12 snapshots to make a fair comparison. The peak SNRs (PSNRs) of the reconstructed images are also presented in the figure. The proposed TACS coded apertures are shown to effectively improve the reconstruction quality compared to the random coded apertures under the same compression ratio. In order to clearly illustrate the improvement, the specific regions around the eyes are magnified in Fig. 6. Figure 7 plots the original and reconstructed spectral signatures for three representative points on the target. The three representative points are selected from different colorful regions, as shown in Fig. 7(a). The intensities in Figs. 7(b)–7(d) indicate the normalized values of the spectra at those three spatial points. The TACS coded apertures are proved to achieve superior fidelity of the spectral reconstructions over the random coded apertures. It shows that these reconstructed spectral signatures are smooth, which implies that the proposed method is reasonable to use the reconstructed images from previous spectral channels as *a priori* information.

#### B. Impact of Key Factors on the Reconstruction Performance

The impact of some key factors on the reconstruction performance is studied in this subsection. Three key factors considered are the number of sub-groups, the compression ratio, and the reconstruction algorithm. Figure 8(a) illustrates the average reconstructed PSNRs in all spectral channels using the TACS method with different sub-group numbers. In addition, the PSNR curve obtained by random coded apertures is also provided for comparison. For each curve, the reconstruction simulations are repeated three times, and the average PSNRs of the reconstructed images are calculated. In general, the quality of the reconstructed images is enhanced when more sub-groups are used. Increasing the number of sub-groups will decrease the number of adjacent spectral channels in each sub-group. Thus, the similarity among the spectral images within each sub-group will also be improved. Since all spectral images in a sub-group are modulated and reconstructed based on one reference image, increasing the number of sub-groups will improve the reconstruction accuracy. However, dividing the entire spectra into more sub-groups will also increase the computational complexity. That is because one reference spectral channel is required to be reconstructed using a random coded aperture for each sub-group. Given the number of sub-groups ${S}_{g}$, we need to reconstruct ${S}_{g}$ reference spectral channels. Moreover, different sub-groups use different sets of adaptive coded apertures. Then ${S}_{g}$ adaptive coded apertures need to be generated. Thus, the computational complexity to generate the adaptive coded apertures is proportional to the number of sub-groups.

Afterwards, we repeat the simulations in Fig. 6 using different compression ratios. The curves of average reconstructed PSNRs over the four spectral channels with respect to different compression ratios are shown in Fig. 8(b). As can be noticed, the TACS coded apertures outperform the random coded apertures in the reconstruction performance under different compression ratios. In addition, the advantage of TACS coded apertures becomes more obvious as the compression ratio reduces. That is because the TACS coded apertures can extract the structural characteristics of the target and retain more information of the spectral data cube in the compressive measurements. For instance, when the compression ratio reduces to 0.1, the TACS coded apertures obtain 5.16 dB gain on the average PSNR over the random coded apertures.

Furthermore, simulations are conducted using the TACS coded apertures to compare the reconstruction quality and computational efficiency of different reconstruction algorithms. The settings of the simulation are the same as those in Fig. 6. For each algorithm, we repeat the reconstructions three times, and calculate the average PSNRs and runtime. Figures 8(c) and 8(d) present the average PSNRs and runtime corresponding to the GPSR algorithm, TwIST algorithm, and SpaRSA algorithm, respectively. It is observed that the GPSR algorithm can achieve the best reconstruction performance. Although the computational efficiency of the GPSR algorithm is lower than that of TwIST and SpaRSA algorithms, it is still fast to handle the reconstruction problem. The TwIST algorithm leads to the shortest runtime, but the reconstruction performance is much worse than that of the other two algorithms, especially in the spectral channels with shorter wavelengths.

#### C. Comparison with Other Adaptive Coding Methods

This subsection provides the simulation results of two other adaptive coding methods to further illustrate the superiority of the TACS coded apertures. In Ref. [40], Galvis *et al*. estimated the edge of the scene using the image obtained from a red–green–blue (RGB) sensor and subsequently designed the structured coded aperture. They applied two different blue-noise coded patterns to the edge and non-edge regions in order to improve the reconstruction quality of the feature boundaries. Yang *et al.* proposed an adaptive sampling strategy for hyperspectral images based on dictionary learning in Ref. [42]. Specifically, they first learned the over-complete dictionary from the training data, and then computed a singular value decomposition from the dictionary. And a small number of left singular vectors were used eventually as the rows of the projection matrix.

Based on the foundation of their work, we simulate these two methods using our proposed framework. Adaptive coded patterns are generated by replacing the RGB images in Galvis’s method and the training data in Yang’s method with the recovered images from previous spectral channels. Figures 9(a) and 9(b) respectively present examples of the coded patterns generated by Galvis’s method and Yang’s method.

Next, the data cubes are recovered from a small set of measurements, according to the same process and settings as Subsection 4.A. Figure 10(a) illustrates the original images of four spectral channels in the data cube. The reconstructed spectral images using the TACS coding method, Galvis’s method, and Yang’s method are respectively shown in Figs. 10(b)–10(d). The average PSNRs for the entire reconstructed data cubes using Galvis’s and Yang’s methods are 27.64 dB and 30.39 dB, respectively. As can be noticed, the reconstruction performance using Galvis’s method is inferior to the TACS coding method. That is because the content information of the RGB image in the non-edge regions is not fully utilized. In addition, the proposed TACS coding method outperforms Yang’s method. But the gap is not so obvious since the dictionary is also adaptively learned from the *a priori* information in Yang’s method. However, the process of dictionary learning is time-consuming and the coded patterns in Yang’s method are not binary, thus making it inapplicable for hardware implementation.

## 5. EXPERIMENTAL RESULTS

This section demonstrates the proposed TACS coded apertures on an experimental testbed of a staring hyperspectral imager developed by our group. As shown in Fig. 11, the testbed consists of an RL127-WHI-IC broadband ring light source (Edmund), a VNIR-5/20-20-S LCTF (Wayho Technology), two AC254-050-A imaging lenses (Thorlabs) with a focal length of 50.2 mm, a DLP9500 DMD (Texas Instruments), and an acA2040-90μm monochromatic CMOS camera (Basler) to capture the measurements. The CMOS camera consists of $1024\times 1024$ pixels with a pitch size of 11.0 μm. The DMD includes $1920\times 1080$ micromirrors with a pitch size of 10.8 μm.

The system should be calibrated beforehand. During the calibration process, the distances between optical components were adjusted to obtain one-to-one correspondence between the coded aperture pixels and detector pixels. In addition, the effect of dark noise was eliminated, and the non-uniformity of the light source was corrected. Taking into account the effects of system impulse response and stray light, the coded aperture patterns were first measured and recorded when the target was replaced by a diffuse plate. Then, the normalized detected patterns were regarded as the actual transmission functions of the coded apertures. When all of the micromirrors on the DMD were turned on, the measurements of the CMOS detector without spatial coding were considered the original images of the target.

As described in Subsection 4.A, the entire spectra are evenly divided into four sub-groups. The real target under detection is shown in Fig. 12(a). In the measurement stage, a resized region including $400\times 400$ pixels on the DMD was used to implement the coded apertures. Figure 12(b) illustrates an example of random coded aperture patterns used to reconstruct the reference spectral image, and Figs. 12(c) and 12(d) illustrate the examples of TACS coded aperture patterns generated according to the reference image. Compared with the random coded apertures, TACS coded apertures extract the structural characteristics of the scene effectively. To reconstruct the reference spectral channels, 32 snapshots were carried out using random coded apertures. Then, 12 snapshots were taken with the TACS coded aperture patterns to reconstruct all of the spectral images within the same sub-group. The center wavelength of the LCTF was tuned for 24 times in total to obtain 24 spectral bands, from 515 nm to 630 nm with an interval of 5 nm. In order to emulate the low-resolution detector, all $8\times 8$ pixels on the detector were combined into one macro-pixel, and then $50\times 50$ macro-pixels were used to acquire the measurement data.

As a comparison, we also built the testbed of the conventional spectral imaging system without using CS theory. In the conventional system, all the micromirrors on the DMD were set to unblock, so that the incident light was directly reflected to the main optical path. In order to acquire the data cube, the LCTF was switched for 24 times to obtain 24 spectral images, each of which consists of $50\times 50$ pixels. For the conventional system, the time to collect the full data is 10 s, while it costs 166 s to acquire all compressive measurements of 24 spectral bands using 12 TACS coded aperture patterns. That is because multiple snapshots are taken to improve the reconstruction quality of super-resolution images for the TACS method.

Figure 13 illustrates the spectral images obtained by different methods using the experimental testbed. The original images in four spectral channels are presented in Fig. 13(a), and the spatial resolution of these images is $400\times 400$ pixels. Figure 13(b) shows the images containing $50\times 50$ pixels acquired by the conventional system. The conventional system does not utilize the CS theory; thus its spatial resolution is determined by the low-resolution detector. The reconstructed spectral images using TACS coded apertures and random coded apertures both with 12 snapshots are respectively shown in Figs. 13(c) and 13(d). The corresponding PSNRs of the reconstructed images are also presented in the figure. The average PSNRs for the entire reconstructed data cube using TACS and random coded apertures are 23.80 dB and 20.59 dB, respectively. To make the comparison more clear, the error patterns of the recovered images with respect to the original images are illustrated in Figs. 13(e) and 13(f). In more detail, Figs. 13(e) and 13(f) show the color maps of error patterns corresponding to the TACS coded apertures and random coded apertures, respectively. Every pixel in the error pattern represents the absolute intensity error of the corresponding pixel between the reconstructed image and the original image. In addition, the mean square errors of all error patterns are presented in Fig. 13. It is observed that the TACS coded apertures achieve superior reconstruction quality over the random coded apertures. Appendix C provides the comparison among these different methods using the experimental testbed in detail.

Figure 14 plots the original and reconstructed spectra of two representative points on the target located at P1 and P2 in Fig. 12(a). The spectra are characterized by the reflectances, which are obtained through spectral calibration and have no units. And the original spectral reflectances are measured by an HR4000 grating spectrometer (Ocean Optics). It is shown that the reconstructed spectra using the TACS coded apertures are more consistent with the original spectra. As illustrated in Fig. 14, the spectra are smooth, which confirms the correlation characteristic among the neighboring spectral channels again.

## 6. MULTI-CHANNEL TACS METHOD

In the above sections, only one spectral channel is captured during one integration time interval of the detector. This section proposes a multi-channel TACS method to introduce the spectral compression and maintain the reconstruction quality without changing the structure of the imaging system. During one integration time interval of the detector, the LCTF is tuned for several times to encompass a series of spectral channels into one snapshot, each of which is modulated by different coded aperture patterns loaded on the DMD. Then, the coded images in these spectral channels are projected and integrated on the CMOS detector. In this way, spectral images in multiple channels, rather than just one channel, are involved in each measurement, thereby increasing the compression capacity of the system. Different from Section 3, the overall compression ratio is determined by

where $Q$ represents the number of times to switch the LCTF during each integration time interval, and $Q<{N}_{\lambda}$. Compared to the single-channel method described in Section 3, the compression capacity of the system is increased by $Q$-fold.Suppose the spectral images within the ${N}_{1}\text{th},{N}_{2}\text{th},\dots ,{N}_{Q}\text{th}$ channels are collected during one integration time interval, and all of them belong to the same sub-group. Then, the imaging model based on the multi-channel method is reformulated as follows:

Next, we provide the simulation results of the proposed multi-channel method. The reference spectral channels are obtained and reconstructed as described in Subsection 4.A. And all parameters are the same as those used in Subsection 4.A. In particular, we capture three spectral channels during each integration time interval, that is $Q=3$. Thus, the overall compression ratio is ${\gamma}_{om}=0.0625$, and the measurement number is reduced to one third of that used for the single-channel method. Figure 15(a) shows the original spectral images with center wavelengths of 554 nm, 563 nm, and 572 nm. The simulated low-resolution images obtained by the conventional system are shown in Fig. 15(b) to illustrate the improvement in spatial resolution. The reconstructed spectral images using the multi-channel method are presented in Fig. 15(c). Figures 15(d) and 15(e) illustrate the reconstructed spectral images using the single-channel method with TACS coded apertures and random coded apertures, respectively. The average PSNR for the entire reconstructed data cube using the multi-channel method is 28.98 dB. Compared to the single-channel method with random coded apertures, the proposed multi-channel method can improve the reconstruction quality. However, the reconstruction performance is inferior to the single-channel method with TACS coded apertures, since the reduction of measurements will degrade the image quality. It can be concluded that the proposed multi-channel method can improve the compression capacity of the system, while maintaining the reconstruction quality to a certain extent. In Appendix D, a side-by-side comparison of the three coding methods is summarized. In the future, experiments will be done to verify the multi-channel method.

## 7. CONCLUSION

This paper developed a novel TACS coding method in spectral imaging and demonstrated it based on the LCTF-based hyperspectral imager for the first time. CS theory was used to obtain the high-resolution hyperspectral data cube that can be recovered from a small set of measurements on the low-resolution detector. Meanwhile, the proposed TACS coded apertures can achieve superior reconstruction performance over the random coded apertures, since the TACS coded apertures can capture the structural characteristics of the underlying target. Moreover, it was proven that the TACS coding method satisfies the mutual-coherence metric better than the traditional random coding method. Using the proposed TACS coding, an improvement up to 4.81 dB on the average reconstructed PSNR is observed in the simulations. In addition, simulation results of other adaptive coding methods are provided for further comparison. Then experiments on the testbed were carried out and the superiority of the proposed TACS coded apertures was demonstrated. Finally, the multi-channel TACS method was proposed to compress the hyperspectral data cube in the spatial and spectral domains simultaneously. The developed super-resolution staring hyperspectral imager was shown to provide promising image quality using the cost-effective low-resolution detectors.

## APPENDIX A: PROOF OF EQ. (5)

The proof of the first inequality in Eq. (5) is as follows:

Substituting Eq. (A9) and Eq. (A10) into Eq. (A5), we can get

First case: if ${\overrightarrow{\varphi}}_{ij}=[1+\mathrm{sgn}({\overrightarrow{S}}_{j}-{\mathrm{\Lambda}}_{ij})]/(2\sqrt{N})$, then we have

## APPENDIX B: PROOF OF EQ. (6)

For the 2D image $\mathbf{I}\in {\mathbb{R}}^{{N}_{x}\times {N}_{y}}$, we have $\mathrm{vec}(\mathbf{I})=\mathbf{\Psi}\overrightarrow{\mathrm{\Theta}}$. Compared to Eq. (3), $\mathrm{vec}(\mathbf{I})$ can be regarded as the original signal $\overrightarrow{X}$. If the 2D-IDCT basis is employed as the sparse basis, then $\mathbf{\Psi}\in {\mathbb{R}}^{({N}_{x}\xb7{N}_{y})\times ({N}_{x}\xb7{N}_{y})}$ is given by $\mathbf{\Psi}={\mathbf{\Omega}}_{1}\otimes {\mathbf{\Omega}}_{2}$, where ${\mathbf{\Omega}}_{1}\in {\mathbb{R}}^{{N}_{x}\times {N}_{x}}$ and ${\mathbf{\Omega}}_{2}\in {\mathbb{R}}^{{N}_{y}\times {N}_{y}}$ are the 1D-IDCT transformation matrices, and $\otimes $ represents the Kronecker product.

For convenience, assume that ${N}_{x}={N}_{y}=n$ and ${n}^{2}=N$. In particular, if ${N}_{x}\ne {N}_{y}$, we can take the square block on the image and perform IDCT transformation on it in sequence. Now denote the 1D-IDCT basis $\mathrm{\Omega}=[{\overrightarrow{\omega}}_{1},{\overrightarrow{\omega}}_{2},\dots ,{\overrightarrow{\omega}}_{n}]\in {\mathbb{R}}^{n\times n}$; thus we can formulate the 2D-IDCT basis $\mathbf{\Psi}\in {\mathbb{R}}^{N\times N}$ as

Therefore, we should first analyze each element in the 1D-IDCT basis before discussing the sum of ${\overrightarrow{\psi}}_{ij}$. The element of $\mathbf{\Omega}$ in the $k$th row and $j$th column can be expressed as

The real part of $\mathrm{exp}(-\frac{a\pi i}{2n})\sum _{k=1}^{n}\mathrm{exp}(\frac{ka\pi i}{n})$ is zero, which means $\sum _{k=1}^{n}\mathrm{cos}\text{\hspace{0.17em}}\frac{(2k-1)a\pi}{2n}=0$. So,

## APPENDIX C: SUMMARY OF THE COMPARISON AMONG DIFFERENT METHODS USING THE EXPERIMENTAL TESTBED

Table 2 provides a side-by-side comparison among different methods using the experimental testbed in Section 5, including the conventional system, and the proposed system using random coded apertures and TACS coded apertures. The premise of calculating the PSNR is that the sizes of the images stay identical. However, the spectral images acquired by the conventional system and the proposed system are of different sizes, thus the PSNR is not given here. Instead, PSNRs of the reconstructed images corresponding to the original high-resolution images are computed. For the conventional system, it takes 24 snapshots to tune the LCTF to obtain 24 spectral channels. While $24\times 12=288$ snapshots are required for the proposed system using random coded apertures. In particular, for the TACS coding method, additional $4\times 32=128$ snapshots are required to obtain four reference spectral images beforehand, each of which is modulated by 32 random coded aperture patterns. Although the number of snapshots for the TACS method has increased, we can use a detector of $50\times 50$ pixels to acquire images of $400\times 400$ pixels. In addition, the reconstruction quality is improved compared to random coded apertures. Note that the time to collect the full data for using the TACS method includes the time to observe and reconstruct four reference channels, as well as the time to calculate four sets of TACS coded apertures corresponding to the four sub-groups. The optical efficiency of the conventional system is mainly determined by the transmittance of the LCTF. For compressive imaging systems, the optical efficiency is also affected by the transmittance of the DMD. The ratio of block/unblock micromirrors is almost 1:1 for both TACS and random coded aperture patterns. Thus, the transmittance of the DMD is about 50%, and the optical efficiency of the proposed compressive system using TACS coded apertures and random coded apertures is almost the same. Furthermore, it is about 50% of that of the conventional system. But the difference will not be huge, because the LCTF has a relatively low transmittance due to an inherent defect of the narrowband filter device.

## APPENDIX D: SUMMARY OF THE COMPARISON AMONG THE THREE CODING METHODS

Table 3 summarizes the comparison among the three coding methods, including the single-channel random coding method, single-channel TACS coding method, and multi-channel TACS coding method. In Section 6, the original data cube consists of 24 spectral images with spatial resolution of $256\times 256$ pixels, and 12 snapshots are used to obtain the measurements using different coding methods. For the single-channel method with random coded apertures and TACS coded apertures, the number of snapshots has been introduced in Appendix C. As for the multi-channel TACS method, three spectral channels are captured during each integration time interval, and $24/3\times 12=96$ snapshots are required. By adding additional $4\times 32=128$ snapshots to obtain four reference spectral images, the total number becomes 224. As shown in Table 3, the multi-channel TACS method has reduced the number of snapshots and the compression ratio, since it introduces the spectral compression. Although the reconstruction performance is inferior to the single-channel method with TACS coded apertures, it is still superior to the single-channel random coding method, while the latter does not conduct spectral compression.

## Funding

National Natural Science Foundation of China (61371132, 61471043, 61527802); International S&T Cooperation Program of China (2014DFR10960).

## Disclosures

The authors declare no conflicts of interest.

## REFERENCES

**1. **D. Bannon, “Hyperspectral imaging: cubes and slices,” Nat. Photonics **3**, 627–629 (2009). [CrossRef]

**2. **S. R. Domingue, D. G. Winters, and R. A. Bartels, “Hyperspectral imaging via labeled excitation light and background-free absorption spectroscopy,” Optica **2**, 929–932 (2015). [CrossRef]

**3. **D. Haboudane, J. R. Miller, E. Pattey, P. J. Zarco-Tejada, and I. B. Strachan, “Hyperspectral vegetation indices and novel algorithms for predicting green LAI of crop canopies: modeling and validation in the context of precision agriculture,” Remote Sens. Environ. **90**, 337–352 (2004). [CrossRef]

**4. **A. Gowen, C. O’Donnell, P. Cullen, G. Downey, and J. Frias, “Hyperspectral imaging—an emerging process analytical tool for food quality and safety control,” Trends Food Sci. Technol. **18**, 590–598 (2007). [CrossRef]

**5. **G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. **19**, 010901 (2014). [CrossRef]

**6. **F. A. Kruse, J. W. Boardman, and J. F. Huntington, “Comparison of airborne hyperspectral data and EO-1 hyperion for mineral mapping,” IEEE Trans. Geosci. Remote Sens. **41**, 1388–1400 (2003). [CrossRef]

**7. **R. O. Green, M. L. Eastwood, C. M. Sarture, T. G. Chrien, M. Aronsson, B. J. Chippendale, J. A. Faust, B. E. Pavri, C. J. Chovit, M. Solis, M. R. Olah, and O. Williams, “Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS),” Remote Sens. Environ. **65**, 227–248 (1998). [CrossRef]

**8. **R. Basedow, D. Carmer, and M. Anderson, “HYDICE system, implementation and performance,” Proc. SPIE **2480**, 258–267 (1995). [CrossRef]

**9. **B. K. Ford, M. R. Descour, and R. M. Lynch, “Large-image-format computed tomography imaging spectrometer for fluorescence microscopy,” Opt. Express **9**, 444–453 (2001). [CrossRef]

**10. **F. A. Best, H. E. Revercomb, D. C. Tobin, R. O. Knuteson, J. K. Taylor, D. J. Thielman, D. P. Adler, M. W. Werner, S. D. Ellington, J. D. Elwell, D. K. Scott, G. W. Cantwell, G. E. Bingham, and W. L. Smith, “Performance verification of the geosynchronous imaging fourier transform spectrometer (GIFTS) on-board blackbody calibration system,” Proc. SPIE **6405**, 64050I (2006). [CrossRef]

**11. **X. Lin, G. Wetzstein, Y. Liu, and Q. Dai, “Dual-coded compressive hyperspectral imaging,” Opt. Lett. **39**, 2044–2047 (2014). [CrossRef]

**12. **J. J. Puschell, “Hyperspectral imagers for current and future missions,” Proc. SPIE **4041**, 121–132 (2000). [CrossRef]

**13. **Q. Li, X. He, Y. Wang, H. Liu, D. Xu, and F. Guo, “Review of spectral imaging technology in biomedical engineering: achievements and challenges,” J. Biomed. Opt. **18**, 100901 (2013). [CrossRef]

**14. **S. C. Gebhart, R. C. Thompson, and A. Mahadevan-Jansen, “Liquid-crystal tunable filter spectral imaging for brain tumor demarcation,” Appl. Opt. **46**, 1896–1910 (2007). [CrossRef]

**15. **S. J. Woltman, G. D. Jay, and G. P. Crawford, “Liquid-crystal materials find a new order in biomedical applications,” Nat. Mater. **6**, 929–938 (2007). [CrossRef]

**16. **M. D. Evans, C. Thai, and J. C. Grant, “Development of a spectral imaging system based on a liquid crystal tunable filter,” Trans. ASAE **41**, 1845–1852 (1998). [CrossRef]

**17. **O. Aharon and I. Abdulhalim, “Liquid crystal lyot tunable filter with extended free spectral range,” Opt. Express **17**, 11426–11433 (2009). [CrossRef]

**18. **Y. August, C. Vachman, Y. Rivenson, and A. Stern, “Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains,” Appl. Opt. **52**, D46–D54 (2013). [CrossRef]

**19. **R. M. Willett, M. F. Duarte, M. A. Davenport, and R. G. Baraniuk, “Sparsity and structure in hyperspectral imaging: sensing, reconstruction, and target detection,” IEEE Signal Process. Mag. **31**, 116–126 (2014). [CrossRef]

**20. **D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory **52**, 1289–1306 (2006). [CrossRef]

**21. **E. J. Candés, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory **52**, 489–509 (2006). [CrossRef]

**22. **L. Zhang, D. Liang, B. Li, Y. Kang, Z. Pan, D. Zhang, and X. Ma, “Study on the key technology of spectral reflectivity reconstruction based on sparse prior by a single-pixel detector,” Photon. Res. **4**, 115–121 (2016). [CrossRef]

**23. **H. Arguello, H. F. Rueda, and G. R. Arce, “Spatial super-resolution in code aperture spectral imaging,” Proc. SPIE **8365**, 83650A (2012). [CrossRef]

**24. **X. Wang, Y. Zhang, X. Ma, T. Xu, and G. R. Arce, “Compressive spectral imaging system based on liquid crystal tunable filter,” Opt. Express **26**, 25226–25243 (2018). [CrossRef]

**25. **G. R. Arce, D. J. Brady, L. Carin, H. Arguello, and D. S. Kittle, “Compressive coded aperture spectral imaging: an introduction,” IEEE Signal Process. Mag. **31**, 105–115 (2014). [CrossRef]

**26. **A. Rajwade, D. Kittle, T. Tsai, D. J. Brady, and L. Carin, “Coded hyperspectral imaging and blind compressive sensing,” SIAM J. Imaging Sci. **6**, 782–812 (2013). [CrossRef]

**27. **H. Arguello and G. R. Arce, “Code aperture optimization for spectrally agile compressive imaging,” J. Opt. Soc. Am. A **28**, 2400–2413 (2011). [CrossRef]

**28. **L. Wang, T. Zhang, Y. Fu, and H. Huang, “HyperReconNet: joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging,” IEEE Trans. Image Process. **28**, 2257–2270 (2019). [CrossRef]

**29. **H. Arguello and G. R. Arce, “Restricted isometry property in coded aperture compressive spectral imaging,” in *IEEE Statistical Signal Processing Workshop (SSP)* (IEEE, 2012), pp. 716–719.

**30. **H. Arguello and G. R. Arce, “Colored coded aperture design by concentration of measure in compressive spectral imaging,” IEEE Trans. Image Process. **23**, 1896–1908 (2014). [CrossRef]

**31. **Z. Wang and G. R. Arce, “Variable density compressed image sampling,” IEEE Trans. Image Process. **19**, 264–270 (2010). [CrossRef]

**32. **J. Hahn, C. Debes, M. Leigsnering, and A. M. Zoubir, “Compressive sensing and adaptive direct sampling in hyperspectral imaging,” Digit. Signal Process. **26**, 113–126 (2014). [CrossRef]

**33. **L. Wang, Z. Xiong, G. Shi, F. Wu, and W. Zeng, “Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging,” IEEE Trans. Pattern Anal. Mach. Intell. **39**, 2104–2111 (2017). [CrossRef]

**34. **J. Zhang, D. Zhao, F. Jiang, and W. Gao, “Structural group sparse representation for image compressive sensing recovery,” in *Data Compression Conference* (IEEE, 2013), pp. 331–340.

**35. **Z. Zha, X. Liu, X. Huang, H. Shi, Y. Xu, Q. Wang, L. Tang, and X. Zhang, “Analyzing the group sparsity based on the rank minimization methods,” in *IEEE International Conference on Multimedia and Expo (ICME)* (IEEE, 2017), pp. 883–888.

**36. **X. Yuan, T. Tsai, R. Zhu, P. Llull, D. Brady, and L. Carin, “Compressive hyperspectral imaging with side information,” IEEE J. Sel. Top. Signal Process. **9**, 964–976 (2015). [CrossRef]

**37. **X. Wang and J. Liang, “Side information-aided compressed sensing reconstruction via approximate message passing,” in *IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)* (IEEE, 2014), pp. 3330–3334.

**38. **J. F. C. Mota, N. Deligiannis, and M. R. D. Rodrigues, “Compressed sensing with side information: geometrical interpretation and performance bounds,” in *IEEE Global Conference on Signal and Information Processing (GlobalSIP)* (IEEE, 2014), pp. 512–516.

**39. **J. Yang, X. Liao, X. Yuan, P. Llull, D. J. Brady, G. Sapiro, and L. Carin, “Compressive sensing by learning a Gaussian mixture model from measurements,” IEEE Trans. Image Process. **24**, 106–119 (2015). [CrossRef]

**40. **L. Galvis, D. Lau, X. Ma, H. Arguello, and G. R. Arce, “Coded aperture design in compressive spectral imaging based on side information,” Appl. Opt. **56**, 6332–6340 (2017). [CrossRef]

**41. **X. Ma, H. Zhang, X. Ma, G. R. Arce, T. Xu, and T. Mao, “Snapshot compressive spectral imaging based on adaptive coded apertures,” Proc. SPIE **10658**, 1065803 (2018). [CrossRef]

**42. **M. Yang, F. Hoog, Y. Fan, and W. Hu, “Adaptive sampling by dictionary learning for hyperspectral imaging,” IEEE J. Sel. Top. Appl. Earth Observ. Remote Sensing **9**, 4501–4509 (2016). [CrossRef]

**43. **M. A. López-Alvarez, J. Hernández-Andrés, and J. Romero, “Developing an optimum computer-designed multispectral system comprising a monochrome CCD camera and a liquid-crystal tunable filter,” Appl. Opt. **47**, 4381–4390 (2008). [CrossRef]

**44. **M. P. Edgar, G. M. Gibson, and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics **13**, 13–20 (2018). [CrossRef]

**45. **N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, “Single-pixel infrared and visible microscope,” Optica **1**, 285–289 (2014). [CrossRef]

**46. **H. Partanen, A. Friberg, T. Setälä, and J. Turunen, “Spectral measurement of coherence Stokes parameters of random broadband light beams,” Photon. Res. **7**, 669–677 (2019). [CrossRef]

**47. **Y. Fu, Y. Zheng, I. Sato, and Y. Sato, “Exploiting spectral-spatial correlation for coded hyperspectral image restoration,” in *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)* (IEEE, 2016), pp. 3727–3736.

**48. **R. A. Ulichney, “Dithering with blue noise,” Proc. IEEE **76**, 56–79 (1988). [CrossRef]

**49. **E. J. Candés and T. Tao, “Near-optimal signal recovery from random projections: universal encoding strategies,” IEEE Trans. Inf. Theory **52**, 5406–5425 (2006). [CrossRef]

**50. **M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE J. Sel. Top. Signal Process. **1**, 586–597 (2007). [CrossRef]

**51. **J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. Image Process. **16**, 2992–3004 (2007). [CrossRef]

**52. **S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, “Sparse reconstruction by separable approximation,” IEEE Trans. Signal Process. **57**, 2479–2493 (2009). [CrossRef]

**53. **E. Candès and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Probl. **23**, 969–985 (2007). [CrossRef]

**54. **E. J. Candès, J. K. Romberg, and T. Tao, “Stable signal recovery from incomplete and inaccurate measurements,” Commun. Pure Appl. Math. **59**, 1207–1223 (2006).

**55. **X. Ma, D. Shi, Z. Wang, Y. Li, and G. R. Arce, “Lithographic source optimization based on adaptive projection compressive sensing,” Opt. Express **25**, 7131–7149 (2017). [CrossRef]

**56. **H. Zhang, W. He, L. Zhang, H. Shen, and Q. Yuan, “Hyperspectral image restoration using low-rank matrix recovery,” IEEE Trans. Geosci. Remote Sens. **52**, 4729–4743 (2014). [CrossRef]