1. Introduction

Multispectral (MS) and hyperspectral (HS) infrared (IR) sensing continues to be a pivotal tool in remote sensing. The role of MS/HS sensing in a wide spectrum of applications has been increasing steadily with the advancement in sensor technology as well as data-processing and interpretation techniques. Conventionally, a MS sensing system is realized by integrating an IR broadband sensor with some sort of an “optical filter wheel,” where each filter admits a single IR spectral band. Current state-of-the-art IR detectors include the HgCdTe-based (MCT) photodetector due to its superior sensitivity (high detectivity and quantum efficiency). MCT detectors have also been integrated with arrays of diffractive optics at the detector level for MS sensing. Examples of such diffractive optics are lenslet arrays [1], micro-electro-mechanical systems (MEMS) [2] and acousto-optic tunable filters (AOTFs) [3]. In recent years, many other detector technologies have emerged, some of which have been competing with the performance of HgCdTe detector. Such technologies are the micro-bolometer [4], quantum-well IR photodetectors (QWIPs) [5] and InAs quantum-dots IR photodetectors (QDIPs) [6], to name a few. As in QWIPs, QDIPs are also cryogenically cooled photodetectors; however, their operation principle is based on intersubband transitions in quantum dots, which can result in a lower dark current compared to QWIPs with good three-dimensional confinement of the QDs and the increased carrier lifetime resulting from reduced scattering processes.

One of the successful QDIP designs is the dots-in-a-well (DWELL) photodetector [7–9], in which quantum dots are embedded in a quantum well. The DWELL technology combines the advantages of QWIPs and QDIPs: they include operating-wavelength tailoring, normal incidence operation, increased lifetime and three-dimensional quantum confinement. Additionally, the DWELL photodetector offers a unique property of spectral tunability that is continuously controllable by the applied bias voltages. As a result of the quantum-confined Stark effect [10], a single DWELL photodetector can be thought of as a MS spectral detector, albeit with overlapping spectral bands [11,12]. Figure 1 shows spectral responses of a recent DWELL detector developed by our group; this device is used later in this paper to demonstrate the proposed sensing algorithm.

 

Fig. 1 Bias-tunable spectral bands of a DWELL photodetector for various applied bias voltages in the range −3 to 3 V.

Download Full Size | PDF

As seen from Fig. 1, the spectral bandwidth of the raw DWELL’s spectral response is too broad (i.e., over 2 μm) for many practical MS/HS applications that require finer spectral resolutions (object detection and identification, spectroscopy, etc.). To enhance the resolution of the DWELL beyond what is available at each fixed bias, we must exploit the continuous bias-dependent tunability property of the DWELL and explore novel multi-bias modes of sensing through post processing. In recent years, our group has reported post-processing algorithms that offer two functionalities beyond those offered by the single-bias mode of the DWELL. The underlying idea is to sense an object of interest repeatedly at multiple applied biases and then form a linear superposition of the bias-dependent photocurrents according to weights that are designed for specific MS sensing tasks. The first functionality, termed spectral tuning, allows performing algorithmic spectrometry [13–16], which has been demonstrated by our group and others in reconstructing the spectra of targets of interest without utilizing any physical optics or spectrometer. Specifically, for an arbitrarily specified narrowband tuning filter, the algorithmic spectral-tuning technique yields an optimal set of weights that can be used to add the bias-dependent spectral responses of the DWELL. The resulting superposition spectral response is the best approximation of the desired shape of the specified narrowband tuning filter. The bandwidth can be as narrow as 0.5 μm, which is one fourth of the full-wave-at-half-maximum (FWHM) spectral bandwidth of the DWELL’s spectral response. A reconstruction of a target’s spectrum at each wavelength is then obtained by forming a weighted linear superposition of bias-dependent photocurrents. Such “superposition photocurrent” represents the best approximation of the ideal photocurrent that would be obtained if we were to use a broadband detector to probe the same target of interest through a physical narrowband spectral filter.

The second functionality, termed spectral matched-filtering [17], is too based upon the principle of forming a superposition. However, the objective there is to perform target classification [18] instead of spectral reconstruction. Specifically, for a given spectrum, representing a class of targets of interest, the spectral matched-filtering technique finds an optimal set of weights to be used to form a weighted superposition of the DWELL’s bias-dependent spectral responses approximating the spectrum of interest. A matched-output for the target’s spectrum is obtained by forming a weighted linear superposition of the bias-dependent photocurrents. The superposition photocurrent represents the best approximation of the ideal photocurrent that would be obtained if we were to use a broadband detector through a spectral filter that is matched to the target’s spectrum. Both algorithms take into account the bias-dependent signal-to-noise ratios of the DWELL’s photocurrents [13,17].

The two functionalities described above were designed without restricting the number of bias-dependent photocurrents to be used in forming the superposition photocurrent. For practical implementation, it may be necessary to limit the number of data acquisitions (or equivalently, the number of applied biases used) due to hardware (memory and processors), cost and/or total acquisition-time constraints. The delay associated with acquiring such a high number of photocurrents sequentially is proportional to the number of biases, making the method inadequate for dynamic targets. It is therefore critical that we extend the sensing algorithms so that only a minimum number of biases are used. The ability to utilize a small number of biases can be exploited by a smart-pixel read-out circuitry in order to enable on-chip implementation of the algorithm.

In this paper we report a new multispectral sensing algorithm to substantially compress the number of necessary biases, and hence the amount of data to be sensed, subject to a prescribed performance level. In essence, the algorithm identifies a minimal set of biases to enable sensing only the relevant spectral information for remote-sensing applications of interest. The remainder of this paper is organized as follows. In Section 2 we review the basic concepts underlying our original algorithmic spectral sensing approach and identify the technical challenges associated with constraining the number of required biases. In Section 3 we describe the generalized, data-compressive spectral sensing algorithm. In Section 4 we perform the case study on optimal bias selection. In Section 5, we demonstrate experimentally the efficacy of our approach in the context of target spectrometry and classification. The conclusions are stated in Section 6.

2. Review of algorithmic spectral sensing and moving on to reducing the sensed data

In this section, we review germane aspects of our original algorithmic MS sensing approach drawing freely from our earlier work [13]. The DWELL’s spectral bands are denoted by the functions R ₁(λ),…,R_K(λ), corresponding to the applied bias voltages v ₁,…,v_K. Let us consider an arbitrary target of interest with unknown spectrum, p(λ), that is probed by the DWELL photodetector at the bias values v ₁,…,v_K. The output of the DWELL photodetector is represented by a vector of bias-dependent photocurrents, I = [i₁,…,i_K]^T; the m ^th photocurrent, i_m, corresponds to the m ^th bias v_m. Mathematically, i_m is expressed by

The spectral-tuning (ST) algorithm [13,14] uses the vector I to estimate the transmittance (or reflectance) spectrum of the unknown target. A brief description of the ST algorithm is given next. Firstly, the user specifies a series of hypothetical narrowband tuning filters, r(λ;λ_n), n = 1, …, L, that would be used to sample the target’s transmittance spectrum at wavelengths λ_{1… .}λ_L. Next, the ST algorithm generates a weight vector, w _n = [w ₁,…,w_K], for each tuning filter r(λ;λ_n). The weights are calculated so that when w _n is linearly combined with the spectral responses R ₁,…,R_K, the superposition spectral response will approximate the r(λ;λ_n). The vector of weights, w _n, can be computed using a closed-form formula (Eq. (18) in [13]):

2.1 Challenges in reducing the number of required biases

To reiterate, the reduction in the number of required biases is needed for two reasons: (1) to minimize the substantial redundancy in the bias-dependent photocurrents as a target is probed by the DWELL detector at the different biases and (2) to make the approach amenable to near real-time implementation by reducing the data-acquisition time. There are two challenges in reducing the number of require biases that this paper aims to surmount. Firstly, if we restrict the number of biases to a small value, there needs to be a viable algorithm for selecting the actual biases from an often-large number of available biases. The challenge here is that the complexity of a direct search approach is exponential due to the combinative nature of the problem. Secondly, even if the first challenge is overcome and we are able to generate a small set of biases for each one of the narrowband (hypothetical) tuning filters r(λ;λ_n), we may obtain a different set of reduced biases for each filter. Thus, an aggregated set of biases (obtained by taking the union of the small number of biases for each filter) that guarantees good performance for all the filters may no longer be small.

To help appreciating the second challenge, consider the example were we are interested in approximating three spectral filters (n = 3) as shown in Fig. 2 . Suppose that we have a total of 30 DWELL spectral responses corresponding to the biases in the range −3 to 3 V in steps of 0.2 V. With an approximation-error metric for performance defined and specified (to be described in details in Section 3), we would need only eight biases for each tuning filter from 30 biases. Our calculations based on the results to be presented in Section 3 (the MBS approach) show that the reduced bias sets for the tuning filters (a), (b) and (c) are{-2.2, −1.2, −0.8, −0.2, 0.2, 0.4, 0.6, 0.8 V}, {-3.0, −2.8, −2.6, −1.8, −1.4, −0.6, −0.4, 1.4 V} and {-2.0, −1.6, −0.8, 0.2, 1.4, 1.8, 2.4, 2.6 V}, respectively. Thus, to approximate all three tuning filters with the same prescribed approximation error, then we would need 21 biases in total.

 

Fig. 2 Example of three different narrowband tuning filter approximations centered at (a) 7.4 μm, (b) 8.8 μm and (c) 10.2 μm, the algorithm requires 21 out of 30 biases. The biases used are {-3.0, −2.8, −2.6, −2.2, −2.0, −1.8, −1.6, −1.4, −1.2, −0.8, −0.6, −0.4, −0.2, 0.2, 0.4, 0.6, 0.8, 1.4, 1.8, 2.4, 2.6}.

Download Full Size | PDF

In the following section we will provide a solution that addresses both of the aforementioned challenges.

3. Uniformly-accurate compressive spectral-sensing algorithm

We begin by defining an extension of the ST algorithm in a generalized setting for which the set of biases and the hypothetical spectral filters (to be approximated by the DWELL spectral responses) are arbitrarily specified. Consider the collection, F_DWELL = {R ₁, …, R_K}, of the DWELL spectral responses corresponding to a maximal set of biases B _DWELL = {v ₁,…,v_K}; namely, R_i(λ) is the spectral response of the DWELL detector when voltage v_i is applied to it. Let F _MS = {f ₁, …, f_M} be a collection of hypothetical multispectral sensing filters designed for specific sensing problems of interest and let b⊂{1,…, K} be the index set for a specified subset of biases from B _DWELL. For each filter f_i, let${\widehat{f}}_i^{(b)}={\displaystyle \sum _{j\in b}{w}_{i,j}^{(b)}{R}_j(\lambda )}$ be its approximation using the set of biases identified by b. In this approximation, the weight vector w _i ^{( b )} = [w_{i ,1} ^{( b )}, …, w_{i ,| b|} ^{( b )}] is calculated according to Eq. (3) with the proviso that the matrices A and Φ are now restricted to the set of biases specified by b, which we denote as A ^{( b )} and Φ ^{( b )}. More precisely,

3.1 Bias-selection algorithms

Two bias-selection algorithms are reported here: the Minimal-Bias-Set (MBS) algorithm, which gives optimal results using an exhaustive search approach, and the Approximate Minimal-Bias-Set (AMBS) algorithm, which offers a suboptimal solution, based on a greedy search approach, but offers huge computational advantage over the MBS algorithm. (A minimal collection of biases may not be unique.)

The procedure of MBS algorithm is straightforward. It searches among all the minimal number of required biases q* and a corresponding q*-bias collection B _min is identified by the index set b _min for which the resulting error metric $P^{(b_{\min })}$is below the prescribed error threshold$\theta \ge P^{(\left\{1,\mathrm{...},K\right\})}$. More precisely, the exhaustive-search method for identifying the minimal bias subset B _min is described through the following steps.

Minimal-Bias-Set Algorithm

Note that since $\theta \ge P^{(\left\{1,\mathrm{...},K\right\})}$ the algorithm described in Steps 1-4 must terminate in at most K steps. Also note that in general$P^{\text{(}{b}_q^{*}\text{)}}\ge P^{\text{(}{b}_{q+1}^{*}\text{)}}$, q = 1, …, K-1.

This MBS algorithm is optimal but it is computationally feasible only when q is reasonably small (e.g., q = 4 and K = 30 as in the example considered in Section 4.) since the identification of each $b_q^{*}$involves $\left(\begin{array}{c}K\\ q\end{array}\right)$calculations of $P^{\text{(}{b}_q\text{)}}$. For large q values the number of bias combinations to consider becomes enormous, which results in unrealistically large computing times. As an alternative, we can employ a greedy approach we referred to as AMBS, which is suboptimal, where the biases for the q + 1 are selected by augmenting the q biases from an earlier stage of the selection process by a single bias that is selected optimally from the remaining K-q biases. The number of searches for each q is therefore reduced from $\left(\begin{array}{c}K\\ q\end{array}\right)$ to K-q. To avoid falling in local minima early on in the selection process, we start the process by first performing the exhaustive-search bias selection process for a small q value (typically q = 3 in our examples) and then employ the greedy approach. The AMBS algorithm in determining a suboptimal minimal bias subset,${\tilde{B}}_{\min }$, is described through the following steps.

Approximate Minimal-Bias-Set Algorithm

Note that since$\theta \ge P^{(\left\{1,\mathrm{...},K\right\})}$, the algorithm described in Steps 1–3 must terminate in at most K steps.

The AMBS approach falls in the more general category of matching pursuit algorithms reported by Cotter et al. [19] and Davis et al. [20]. Both approaches are based upon a greedy principle and share the common objective of searching for a sparse solution to represent the signal based upon a suboptimal forward search. In both approaches, a search is made through a “dictionary” in an iterative fashion rather than solving the optimal approximation problem. However, there are two key differences in the implementation of the search processes used in the AMBS and that used by the matching pursuit algorithms. The AMBS algorithm selects the vector (or subset) from a given dictionary based upon minimizing the “first-order residual,” which simply corresponds to the error between the true signal and the projected signal. On the other hand, the matching pursuit algorithm chooses the vector from the set of dictionary vectors iteratively by sub-decomposing the residual to represent the original signal, thereby considering “higher-order residuals,” as explained in [19,20]. Another key difference is that the AMBS involves an important initialization step, based on exhaustive search, for finding a good initial value in order to avoid falling in local minima early on in the selection process. The greedy process then follows the initial step.

3.2 Uniformly-accurate compressive spectral sensing algorithm

The uniformly-accurate compressive spectral sensing (UCSS) algorithm is summarized in Fig. 3 . There are three inputs specified by the user. The first input is the collection, F_DWELL and the corresponding maximal set of biases B _DWELL. The second input is the collection F _MS of hypothetical multispectral sensing filters for the specific sensing problems of interest. The third and final input is the user-prescribed worst-case error threshold, θ, for the error metric P ^{( b )}.

 

Fig. 3 Illustration of the remote-sensing applications of data compressive UCSS algorithm.

Download Full Size | PDF

Here the threshold θ is selected such that it is achievable, namely, θ ≥ P ^{({1, …, K })}. The set of indices, b _min⊂{1,…, K}, is obtained from either MBS or AMBS algorithms described in Subsection 3.A, and it defines a minimal set of biases B _MS. The optimal collection of weight vectors corresponding to with b _min and F _MS is${\text{W}}_{\text{MS}}=\text{{}{\mathbf{w}}_{\text{1}}{}^{\text{(}{b}_{\text{min}}\text{)}}\text{, }\dots \text{, }{\mathbf{w}}_M{}^{\text{(}{b}_{\text{min}}\text{)}}\}$ (here M is the number of spectral filters in F_MS). Note that each weight vector is of length |B _min|. In the final stage of the UCSS algorithm, the photocurrents from the spectrally tunable detector sensing a target at the minimal bias-set B _MS; these photocurrents are the most relevant spectral data set for any specific application represented by F _MS. The photocurrents are then linearly combined according to the subset of weight vectors from${\text{W}}_{\text{MS}}$, corresponding to the spectral filters in F _MS, to yield the desired features equivalent to those that we would have obtained had we used a broadband detector in conjunction with the spectral filters in F _MS.

3.3 Generalization to linear space generated by F_MS

Suppose that we are interested in sensing using a hypothetical filter,$\tilde{f}$, a linear superposition of individual filters in the collection F _MS:$\tilde{f}(\lambda )={\displaystyle \sum _{i=1}^M{\beta }_i{f}_i(\lambda )}$ where β_i’s are scaling factors that are chosen to control the shape of $\tilde{f}(\lambda )$). For example, if M = 2, β ₁ = −1 and β ₂ = 1, then $\tilde{f}(\lambda )=f_2(\lambda )-f_1(\lambda ),$ which yields the differences of the spectral features at λ ₂ and λ ₁. Is it possible to extend the ST algorithm to accommodate this scenario without the need for redoing the bias-selection optimization problem (Subsection 3.A) for the extended filter set$F_{MS}\cup \left\{\tilde{f}\right\}$? Indeed, the linear nature of the sensing problem at hand dictates that the required weight vector${\tilde{\mathbf{w}}}^{\text{(}{b}_{\text{min}}\text{)}}$associated with $\tilde{f}$ is nothing but a linear superposition of the scaling factors of the individual filter elements in$F_{MS}$:

4. Case study on optimal bias selection

4.1 Specification of sensing filters and their approximations by a minimal bias set

We experimentally measured the bias-dependent spectral responses of the DWELL photodetector, F_DWELL = {R ₁(λ), R ₂(λ),…, R ₃₀(λ)}, with 30 different biases corresponding to the bias set B _DWELL = {-3, −2.8, −2.6, …, 3 V}. We also set the error threshold, θ, to 8%, and further specified F _MS as the collection of six spectral sensing filters {f ₁(λ), f ₂(λ),…, f ₆(λ)}. Specifically, f ₁(λ), f ₂(λ) and f ₃ (λ) are defined as three disjoint hypothetical narrowband triangular sensing filters centered at 7.4 μm, 8.8 μm and 10.2 μm, each with a full-width at half maximum of 0.5 μm. We select the filters f ₄(λ), f ₅(λ) and f ₆(λ) to be the actual transmittances of three optical filters in the ranges 7.5-10.5 μm, 8.0-9.0 μm and 8.5-11.5 μm. For the generalization in Subsection 3.B, we specified two linearly superpositioned filters: a spectral integrator ${\tilde{f}}_1\text{(}\lambda \text{) }$and a spectral differentiator${\tilde{f}}_2\text{(}\lambda \text{)}$. The filter ${\tilde{f}}_1\text{(}\lambda \text{) }$ is the sum of f ₁(λ), f ₂(λ) and f ₃ (λ), and the filter${\tilde{f}}_2\text{(}\lambda \text{)}$is the difference between f ₂(λ) and f ₁(λ), as shown in the dotted lines in Fig. 4(c) . The UCSS algorithm was invoked and a minimal set of four biases was obtained by using the MBS algorithm: B _MS = {-3, −0.8, 1.0, 2.8 V} (with the corresponding set of indices, b _min). The corresponding collection of six weight vectors${\text{W}}_{\text{MS}}=\text{{}{\mathbf{w}}_{\text{1}}{}^{\text{(}{b}_{\text{min}}\text{)}}\text{, }\dots \text{, }{\mathbf{w}}_6{}^{\text{(}{b}_{\text{min}}\text{)}}\}$was also found, resulting in a relative error metric $P^{(b_{\min })}$ = 6.7%, which satisfies the prescribed error threshold of θ = 8%. Approximations of the member of F _MS are shown in solid blue lines of Fig. 4(a) for f ₁(λ), f ₂(λ) and f ₃ (λ), and in solid blue lines in Fig. 4(b) for f ₄(λ), f ₅(λ) and f ₆(λ). Since an error metric is only 6.7%, shapes of approximated F _MS using minimal four biases are very similar to the reference (the approximated F _MS using entire 30 biases) shown in solid red lines of Fig. 4. This demonstrates that the use of minimal biases selected by the MBS algorithm does not sacrifice performance. Also note that as compared to the result in Fig. 2 by the original ST algorithm (which uses 21 biases), the use of the MBS algorithm has significantly reduced the number of required biases down to four, resulting in a reduction by a factor of 7 in the required biases for sensing. The weight vector ${\tilde{\mathbf{w}}}_1^{(b_{\text{min}})}$associated with the spectral integrator ${\tilde{f}}_1\text{(}\lambda \text{) }$is obtained by solving Eq. (6) with${\text{W}}_{\text{MS}}$ and the scale factors${\beta }_1=\mathrm{...}={\beta }_3=1$and${\beta }_4=\mathrm{...}={\beta }_6=0$. Similarly, for the spectral differentiator${\tilde{f}}_2\text{(}\lambda \text{)}$, the weight vector ${\tilde{\mathbf{w}}}_2^{(b_{\text{min}})}$is found by solving Eq. (6) with ${\text{W}}_{\text{MS}}$and the scale factors${\beta }_1=-1$, ${\beta }_2=1$and${\beta }_3=\dots ={\beta }_6=0$. Approximations of ${\tilde{f}}_1\text{(}\lambda \text{)}$ and ${\tilde{f}}_2\text{(}\lambda \text{)}$ are shown in Fig. 4(c).

 

Fig. 4 The MBS algorithm is used to approximate the specified spectral-filter collection F _MS: (a) f ₁(λ), f ₂(λ) and f ₃ (λ) are hypothetical narrowband triangular sensing filters and (b) f ₄(λ), f ₅(λ) and f ₆(λ) are spectral matched filters using only minimal four biases B _MS out of K = 30 biases, B _DWELL. The successful approximations using minimal four biases are shown in blue, which corresponds to the error metric $P^{(b_{\min })}$ = 6.7% as compared to the approximations using all 30 biases shown in red. The approximations (in blue) of two superposition filters, the spectral integrator ${\tilde{f}}_1\text{(}\lambda \text{)}$ and the spectral differentiator${\tilde{f}}_2\text{(}\lambda \text{)}$, are shown in (c) along with the approximations using all 30 biases in red.

Download Full Size | PDF

Performance of the AMBS algorithm for F_MS

In this subsection, we evaluate the performance of the AMBS algorithm for approximating the specified collection F _MS. The results from the AMBS are also compared to those of the MBS. For evaluation purposes, we applied the AMBS algorithm to three different cases by specifying three different error thresholds: (i) θ = 8%, (ii) θ = 6% and (iii) θ = 5%. Results for all three cases are given in Tables 1 , 2 and 3 , respectively. We observed that the minimal bias set identified by the AMBS algorithm does not exactly match that obtained by the MBS algorithm in all three cases. However, the error metrics $P^{(B_{\min ,AMBS})}$ (7.1%, 5.4% and 4.6%) for the AMBS are all within 0.5% of $P^{(B_{\min ,MBS})}$ (6.7%, 5.1% and 4.4%) for the MBS, demonstrating almost identical performance. Also note that for all three cases, the search time by the AMBS algorithm is faster than the MBS algorithm. Particularly, in case (iii), the search time by the AMBS algorithm is 69 times faster than the MBS algorithm. Thus, the AMBS algorithm can be a good alternative to the MBS algorithm since it can generate comparable results with less computational effort.

Table 1. Summary of Results for Case (i) Comparing between MBS and AMBS Algorithms for the Approximations of F_MS

View Table | View all tables in this article

Table 2. Summary of Results for Case (ii), Comparing between MBS and AMBS Algorithms for the Approximations F_MS

View Table | View all tables in this article

Table 3. Summary of Results for Case (iii), Comparing between MBS and AMBS Algorithms for the Approximations of F_MS

View Table | View all tables in this article

4.2. Flexibility in the minimal bias selection

In Subsection 4.A we identified a minimal set of four biases. However, we have seen some level of tolerance to these bias values with a minimal penalty in performance. In this subsection we generate four groups of biases that offer a more flexible specification of the minimal set of required biases. In particular, an alternative minimal set of biases can be obtained by selecting a bias from each group of biases.

To introduce flexibility in the bias selection, we allowed the MBS algorithm to find the top-twenty ranked bias sets instead of single minimal bias set B _MS. The tolerance in the error metric is set to 0.2% as compared to the original error metric of$P^{(b_{\min })}=6.7\%$. With this procedure, we generated 10 biases in total (there are at most 80 biases that can be generated but many of these where duplicates). We can then list all these 10 biases and identify four groups. The significance of each bias out of the 10 biases is determined by the number of times it is selected by the top-twenty bias sets. The significance of the 10 biases is illustrated by the histogram shown in Fig. 5 . By visual inspection, four different bias groups G ₁, G _2, G ₃, and G ₄, are identified and listed in Table 4 . Note that the originally selected optimal biases are members of these groups, as identified by thick text in Table 4.Our ability to identify the populated bias group, for example G ₄, is attributable to the similarity in the DWELL’s spectral responses at these three biases and their comparable SNRs. The corresponding spectral responses are compared in Fig. 6 , showing the similarity among them. It is interesting to note that the collections of biases, {-2.8, −2.6, −2.4, −2.2, −2, −1.8, −1.6 V}, {-0.6,-0.4, −0.2, 0.2, 0.4, 0.6 V} and {1.2, 1.4, 1.6, 1.8, 2.0, 2.2, 2.4 V} are never selected due to the fact they have little overlap with the members of F _MS as well as their relatively low SNRs. We have verified that the SNRs for the bias collection {-2.8, −2.6, −2.4, −2.2, −2, −1.8 V} are much lower (< 80) than those for −3 V (> 300), which explains why −3 V is always selected while its neighboring biases are not selected. Moreover, the biases −0.2, 0.2, 0.4, and 0.6 V are never selected because their SNRs (< 10) are the lowest among all the biases.

 

Fig. 5 The histogram illustrates the significance of each bias member in the set of 10 biases. By visual inspection, we identified four distinct bias groups.

Download Full Size | PDF

Table 4. Identified Members in Four Bias Groups for Approximating The Specified Filter Collection F_MS*

View Table | View all tables in this article

 

Fig. 6 Similarity of the DWELL’s spectral responses at 2.6, 2.8 and 3V.

Download Full Size | PDF

5. Experimental results on spectrometry and classification

In order to experimentally demonstrate the multispectral sensing capability of the UCSS algorithm, we have applied the collection, F _MS, of filters described in Subsection 4.A to two common remote-sensing applications. The first application is spectrometry, termed algorithmic spectrometry here. It aims to reconstruct samples of the spectra of any unknown target of interest at prescribed tuning wavelengths without the use of any physical dispersive elements or optics. This is done by means of forming a weighted linear superposition of the measured bias-dependent photocurrents, measured by the DWELL detector, according to a predetermined set of weights obtained from the UCSS algorithm. The measured photocurrents are obtained by probing the unknown target by the DWELL detector using a minimal bias set provided by the MBS algorithm. The result of this weighted-superposition process is a set of “superposition photocurrents” that represent samples of the transmittance at desired tuning wavelengths. In addition to sampling the spectrum of the unknown target, we can also extract more general spectral features, such as an spectral average over multiple wavelengths or slope of the spectrum at specified wavelengths, by performing weighted superposition using other predetermined weights (also from the UCSS algorithm) applied to the same bias-dependent photocurrent.

The second application is the classification of a probed unknown object as that having one of multiple known transmittance spectra (the spectra are selected from the members of F _MS), based on the concept of algorithmic spectral matched filtering. The idea of spectral matched filtering is to use multiple weight vectors (as many as the number of candidate transmittance spectra) obtained from the UCSS algorithm that can be used by a “classifier” to perform a weighted linear superposition of the measured bias-dependent photocurrents. The measured photocurrents in this case results from probing the unknown target whose transmittance spectrum is any one of multiple possible spectra. The result is a set of extracted “superposition features,” which the classifier further converts to the “label” of the unknown object (label of its spectrum). Details of the experimental procedure and results for these two remote-sensing applications are given next.

5.1 Experimental results on target spectrometry

Three spectral filters, f ₁(λ), f ₂(λ) and f ₃(λ) (members of F _MS), are selected to sample the transmittance of the unknown target centered at λ₁ = 7.4 μm, λ₂ = 8.8 μm and λ₃ = 10.2 μm. The unknown target was selected as the spectral filter in the range 7.5-9.5 μm, whose transmittance spectrum is shown in Fig. 7 , solid red line.

 

Fig. 7 Three spectral filters, f ₁(λ), f ₂(λ) and f ₃ (λ) in the filter collection F _MS are used to sample the unknown target, whose transmittance is shown in red. For reference, the ideal triangular spectral filters are also shown in dashed line. Approximated filters in blue line were obtained by the UCSS algorithm using minimum four biases −3.0, −0.8, 1.0, 2.8 V selected by the MBS algorithm.

Download Full Size | PDF

We measured the photocurrent vector, I _spec, as the DWELL photodetector sequentially probed the unknown filter target using the minimal set of four biases {-3.0, −0.8, 1.0, 2.8 V} selected by the MBS algorithm as described in Section 4. For comparison, the photocurrent measurement was also repeated for the following auxiliary bias sets: the best-five bias set {-3, −1.4, −0.8, 1, 2.8 V}, the best-six bias set {-3, −1.4, −0.8, 0.8, 2.2, 3 V} and the complete bias set consisting of all 30 biases. Note that in the best-five and best-six bias set cases the biases were also selected using the MBS algorithm described in Section 3 by constraining the number of biases to 5 and 6, respectively. Specifically, the measured ${\mathbf{I}}_{\text{spec}}$ is linearly combined with each weight vector, yielding a superposition photocurrent ${\widehat{I}}_i={({\mathbf{w}}_i{}^{(b_{\text{min}})})}^T{\mathbf{I}}_{\text{spec}}$, where i = 1,2, 3. As referred to [14], this superposition photocurrent ${\widehat{I}}_i$, termed “experimental reconstruction” best approximates the transmittance of unknown target that we would have obtained if we look at the same target through the ideal triangular spectral filter. Recall that in Subsection 4.A, the UCSS algorithm generated three weight vectors: ${\mathbf{w}}_{\text{1}}{}^{\text{(}{b}_{\text{min}}\text{)}}\text{, }{\mathbf{w}}_2{}^{\text{(}{b}_{\text{min}}\text{)}}\text{ and }{\mathbf{w}}_3{}^{\text{(}{b}_{\text{min}}\text{)}}$ corresponding to f_i(λ), i = 1, 2 and 3.

The experimental reconstructions using minimal four biases are shown in Fig. 8 (blue circle) and represent the sampled transmittances of target at λ₁, λ₂ and λ₃. We also generated the estimated transmittances resulting from sampling the true target transmittance by ideal triangular filters centered at λ₁, λ₂ and λ_3, shown in Fig. 8 (red square), and used them as a reference for accurate comparison, Results show that both the reconstruction and the reference at λ₁ and λ₃ are close to zero. These values are consistent with the true target transmittance shown in Fig. 7 (red) since λ₁ and λ₃ correspond to the stopband where the transmittance is zero. At λ₂ = 8.8 μm, the reconstructed transmittance is within 30% error as compared to the corresponding reference (0.123 and 0.171 in Table 5 ). Also for a comparison, the true target transmittance at 8.8 μm in Fig. 7 (red) is 0.381, which is the ground truth. Note that the ultimate goal of our algorithmic sensing approach is to estimate this true transmittance in the best way possible; the use of the narrowest “triangular filter” is just a one way for achieving this goal.

 

Fig. 8 Experimentally reconstructed transmittances (blue circle) at 7.4 μm, 8.8 μm and 10.2 μm extracted by the UCSS algorithm using minimum four biases −3.0, −0.8, 1.0, 2.8 V selected by the MBS algorithm were obtained. Results are compared to the sampled transmittances by the ideal triangular spectral filters (red square) considered as the reference.

Download Full Size | PDF

Table 5. Comparison of Experimental Reconstruction of the Transmittance at Three Wavelengths Using the Minimal Four Biases by the MBS Algorithm and the Associated Reconstruction Errors to those Using other Bias Selections by the MBS Algorithm (Best-5 Biases, Best-6 Biases and all 30 Biases)

View Table | View all tables in this article

Results from the other bias selections (best-five and best-six bias sets) by the MBS algorithm are also shown in Table 5. Here, we observed that the reconstructions at 7.4 μm and 10.2 μm are close to zero for all bias selections. At 8.8 μm, the reconstructions for all bias selections are within 8%. Thus, the use of minimal four biases does not sacrifice the performance of UCSS algorithm in successfully extracting the narrowband feature. Note that in Table 5 we find the error between the experimental reconstruction and the ideal reconstruction (using ideal triangular filters) starts to increase at some point as more biases are used. As we explained in our prior work [13], this observation is not contradictory to the optimality of the algorithm since sets of weights determined in the spectral tuning algorithm do not guarantee minimizing the error between the actual target spectrum and the reconstruction. Instead, what these weights do guarantee is that the error between the ideal triangular tuning filter and the approximate triangular tuning filter is minimized. Indeed, the error in the synthesized triangular filters do decrease monotonically in the number of biases used, achieving a minimum error when all 30 biases are used. Note that the quality of the reconstructed transmittance not only depends on the quality of approximation of the triangular filter but also on actual transmittance (its variation as a function wavelength within the passband of the triangular filter). We also suspect that for the case of reconstructing spectral content at λ₃ = 10.2 μm, when the algorithm uses all 30 biases, those biases beyond the fifth bias selected have weak signal content and their inclusion simply adds more noise to the estimate, hence increasing the reconstruction error.

Moving onto the superposition filter case (as described in Subsection 4.A), the UCSS algorithm found two weight vectors, ${\tilde{\mathbf{w}}}_1^{(b_{\text{min}})}$and ${\tilde{\mathbf{w}}}_2^{(b_{\text{min}})}$, that approximated the spectral integrator${\tilde{f}}_1\text{(}\lambda \text{) }$and the spectral differentiator${\tilde{f}}_2\text{(}\lambda \text{)}$, respectively. Each weight vector is linearly combined with$I_{\text{spec}}$, obtaining the reconstructed spectral features${\widehat{\tilde{\mathbf{I}}}}_i={({\tilde{\mathbf{w}}}_i{}^{(b_{\min })})}^T{\mathbf{I}}_{\text{spec}}$. Recall that for${\tilde{f}}_1\text{(}\lambda \text{)}$, ${\widehat{\tilde{\mathbf{I}}}}_1$approximately represents the sum of reconstructed transmittances at λ₁, λ₂ and λ₃, as illustrated in Fig. 9(a) . The average of reconstructed transmittances can be obtained after dividing ${\widehat{\tilde{\mathbf{I}}}}_1$ by the number of center wavelengths (i.e., dividing by 3). In the case of${\tilde{f}}_2\text{(}\lambda \text{) }$, ${\widehat{\tilde{\mathbf{I}}}}_2$ represents the difference in the transmittance values at λ₁ and λ₂, as shown in Fig. 9(b). As a result, the slope of the transmittance curve can be approximated by dividing ${\widehat{\tilde{\mathbf{I}}}}_2$ by λ ₂ - λ ₁.

 

Fig. 9 Applications of two linearly superpositioned filters (i.e., (a) the spectral integrator ${\tilde{f}}_1\text{(}\lambda \text{) }$and (b) the spectral differentiator${\tilde{f}}_2\text{(}\lambda \text{)}$) to the spectrometry problem of unknown filter target. Approximations${\widehat{\tilde{f}}}_1(\lambda )$and${\widehat{\tilde{f}}}_2(\lambda )$ can extract the spectral average and slope of unknown target, respectively.

Download Full Size | PDF

The experimentally extracted values of the averaged transmittance values (captured by${\widehat{\tilde{f}}}_1(\lambda )$) and the approximated slope of transmittance (captured by${\widehat{\tilde{f}}}_2(\lambda )$) are listed in Table 6 . The experimental reconstructions are compared to the values obtained by using ideal spectral integrator and differentiator (shown in dotted line of Fig. 4(c)). For ${\widehat{\tilde{f}}}_1(\lambda )$, the estimate of the averaged transmittance is within 14% error as compared to the ideal value (i.e., 0.058 in Table 6). For ${\widehat{\tilde{f}}}_2(\lambda )$, the estimated slope is within 40% error as compared to the reference (i.e., 0.121 in Table 6). In addition, we observed that the use of the minimal four biases by the MBS algorithm yields consistent results with less than 9% error as compared to values for the other (larger) bias selections

Table 6. Experimentally Extracted Averaged Transmittance Captured by${\widehat{\tilde{f}}}_1(\lambda )$and Slope of Transmittance Captured by${\widehat{\tilde{f}}}_2(\lambda )$ for Different Bias Selections: Minimum Four Biases, Best-Five Biases, Best-Six Biases and All 30 Biases*

View Table | View all tables in this article

5.2 Experimental results on target classification

Here, the target spectral filters comprising the classes of spectra are selected as f ₄(λ), f ₅(λ) and f ₆(λ) (7.5-10.5 μm, 8.0-9.0 μm and 8.5-11.5 μm). The photocurrent vector ${\mathbf{I}}_{\text{class}}$was measured as the DWELL photodetector was exposed to radiation transmitted through three target filters, f ₄(λ), f ₅(λ) and f ₆(λ) using the same bias sets used in the spectrometry problem of Subsection 5.A. For each filter, photocurrent measurements were repeated at least 20 times and averaged to minimize the temporal variability of DWELL photodetector. Recall that the use of the three weight vectors,${\mathbf{w}}_{\text{4}}{}^{\text{(}{b}_{\text{min}}\text{)}}\text{, }{\mathbf{w}}_5{}^{\text{(}{b}_{\text{min}}\text{)}}\text{ and }{\mathbf{w}}_6{}^{\text{(}{b}_{\text{min}}\text{)}}$ in Subsection 4.A had resulted in optimal matching of the reconstructed transmittances to the actual transmittances f ₄(λ), f ₅(λ) and f ₆(λ). We denoted the corresponding reconstructed matched filters as ${\widehat{f}}_4\text{(}\lambda \text{) , }{\widehat{f}}_5\text{(}\lambda \text{)}$and${\widehat{f}}_6\text{(}\lambda \text{)}$. For the classification problem, each matched filter is labeled with a specific class number: Class-1 corresponding to f ₄(λ), Class-2 corresponding to f ₅(λ), and Class-3 corresponding to f ₆(λ). In the classifier, ${\mathbf{w}}_{\text{4}}{}^{\text{(}{b}_{\text{min}}\text{)}}\text{, }{\mathbf{w}}_5{}^{\text{(}{b}_{\text{min}}\text{)}}\text{ and }{\mathbf{w}}_6{}^{\text{(}{b}_{\text{min}}\text{)}}$are linearly combined with the incoming test data,${\mathbf{I}}_{\text{class}}$, resulting in three synthesized features: $F_{\text{1}}={\text{(}{\mathbf{w}}_4{}^{\text{(}{b}_{\text{min}}\text{)}}\text{)}}^T{\mathbf{I}}_{\text{class}}$, $F_{\text{2}}={\text{(}{\mathbf{w}}_5{}^{\text{(}{b}_{\text{min}}\text{)}}\text{)}}^T{\mathbf{I}}_{\text{class}}$ and $F_{\text{3}}={\text{(}{\mathbf{w}}_6{}^{\text{(}{b}_{\text{min}}\text{)}}\text{)}}^T{\mathbf{I}}_{\text{class}}$. We denote the feature vector formed by these synthesized features by $\mathbf{F}=(F_{\text{1}},F_{\text{2}},F_{\text{3}})$. Finally, the classifier assigns this F to class$i^{*}$whose feature value, $F_{i^{*}}$, is the highest among the three features; more precisely, $i^{*}=\underset{i\in \{1,2,3\}}{\arg \max }{F}_i$.

With the minimal four-bias set used, the results show that the classifier has correctly assigned all three test data (${\mathbf{I}}_{\text{class}}$) to their respective classes, as shown in Fig. 10 . In our experimental demonstration, our classifier yielded 100% accuracy. This perfect classification was obtained owing to the fact that the three target spectral filters were reasonably separable. However, if targets are not separable to begin with (i.e., if the extracted features from multiple targets are similar), then we would expect the accuracy of classifier to be reduced.

 

Fig. 10 Classification results for identifying three experimental test data, (I)_class. The classifier has successfully assigned the data to Class-1 (see (a)), the data to Class-2 (see (b)), and the data to Class-3 (see (c)).

Download Full Size | PDF

When we use the best-five biases (gray bars in Fig. 11 ), the best-six biases (blue bars in Fig. 11) and all 30 biases (green bars in Fig. 11), we also obtain 100% accuracy. This implies that the use of the minimal four biases in the classification problem produced equivalent performance as compared to the result using all the 30 biases.

 

Fig. 11 Comparison of classification results for minimal four biases (white) to other bias selections: best-five biases (gray), best-six biases (blue) and all 30 biases (green) for identifying the three experimental test data, (I)_class to (a) Class-1, (b) Class-2 and (c) Class-3. Note that the use of minimum four biases obtained by the MBS algorithm in the UCSS algorithm achieved almost identical result compared to the case using all 30 biases.

Download Full Size | PDF

It is important to mention, that we have observed that the temporal variation of the test data affects the outcome of the classifier if insufficient number of photocurrent measurements is available. For example, over 30% classification error was obtained when we used only 9 photocurrent measurements (per class and averaged). However, when we use 10 or more photocurrent measurements, the classification error was highly improved; for example, with 16 or more photocurrents measurements, 100% classification was achieved.

6. Conclusions

In this paper we reported a novel data compressive spectral sensing algorithm in conjunction with the bias-dependent spectrally tunable DWELL photodetector that identifies and employs a minimal set of required biases subject to a specified performance level. The identification of a minimal bias set enables the detector to sense only the most relevant and least noisy bias-dependent spectral bands for specific sensing applications. Moreover, the minimal bias set provides a uniformly accurate solution across the collection of specified spectral sensing filters, which captures the corresponding multispectral features for remote-sensing applications of interest. We implemented the algorithm to approximate the collection of six spectral sensing filters and the algorithm identified the minimal set of only four biases for successful approximation of the filter collection. By sensing using the DWELL at these four biases only, we successfully performed two remote-sensing applications that utilize the six spectral sensing filters; these applications were spectrometry of unknown filter target and the classification of three filter targets. In the spectrometry problem, we were able to successfully reconstruct three samples of the transmittance of an unknown test target. In addition, we are able to reconstruct the average of the transmittance across three wavelengths and the slope of the transmittance spectrum at a given wavelength. For the classification problem, we were able to use the DWELL measurement using the four applied biases to successfully classify three spectral filters selected from the collection of six spectral filters.

It is to be noted that in essence, what our approach is capable of doing is to synthesis the effect of an arbitrary optical filter by solely using the optoelectronic properties of the DWELL. The ability to do so successfully gives optical filtering a fresh perspective. Our approach can potentially be used beyond the DWELL sensor; it can be applicable to traditional multi-color infrared detectors, especially if there is overlap in the spectral bands. For example, our approach can potentially be applied to quantum-well detectors which already demonstrated voltage tunable multicolor detection reported in [21]. From a device perspective, this work helps us understand rigorously the reach of the spectral diversity of the DWELL device.

We wish to point out that the MBS and AMBS algorithms can be further enhanced by introducing an extra preliminary stage that eliminates insignificant spectral bands, based on certain SNR requirement, before applying either the MBS or AMBS algorithms. This can be achieved by building such de-selection process in the metrics used by the MBS and AMBS algorithms.

Finally, effort is underway to implement this new data-compressive DWELL-based sensing paradigm in a focal-plane-array (FPA) platform using a novel custom-designed readout integrated circuit, which can directly output spectral signatures or object classes in near real-time spectral sensing.