Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Tunable image projection spectrometry

Open Access Open Access

Abstract

We present tunable image projection spectrometry (TIPS), a Fourier-domain line-scan spectral imager with a tunable compression ratio. Compared to state-of-the-art spatial-domain pushbroom hyperspectral cameras, TIPS requires much fewer measurements and provides a higher light throughput. Using a rotating Dove prism and a cylindrical field lens, TIPS scans an input scene in the Fourier domain and captures a subset of multi-angled one-dimensional (1D) en face projections of the input scene, allowing a tailored data compression ratio for a given scene. We demonstrate the spectral imaging capability of TIPS with a hematoxylin and eosin (H&E) stained pathology slide. Moreover, we showed the spectral information obtained can be further converted to depths when combining TIPS with a low-coherence full-field spectral-domain interferometer.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

A hyperspectral camera captures both spatial $({x,\; y} )$ and spectral information $(\lambda )$ of input scenes, providing both intrinsic and discriminative spectral characteristics of objects for target recognition and classification. Originally being developed for remote sensing, hyperspectral imaging has found broad applications in other fields as well, such as biomedical imaging and machine vision [110]. Based on the data acquiring mechanism, hyperspectral cameras are generally stratified into three categories: spatial scanning [1114], spectral scanning [1519], and snapshot [2027]. While spatial-scanning hyperspectral cameras capture the spectrum of a spatial point or line at a time, their spectral-scanning counterparts acquire a two-dimensional (2D) image at each wavelength. Despite a relatively simple optical architecture, spatial-scanning hyperspectral imagers bear a lengthy acquisition—to measure a ($x,\; y,\; \lambda $) datacube, the system must perform a complete scan of all spatial locations or spectral wavelengths. In contrast, snapshot hyperspectral imagers can acquire all datacube voxels in parallel, thereby maximizing the light throughput. However, current snapshot hyperspectral systems are plagued by various problems, such as low image quality [2830], extensive computation [3134], and complex configuration [3537].

The primary challenge for measuring a hyperspectral datacube is dimensionality reduction because most electronic detectors are in two-dimensional (2D), one-dimensional (1D), or zero-dimensional (0D) format. Direct mapping a three-dimensional (3D) hyperspectral datacube $({x,\; y,\; \lambda } )$ to a 2D detector array often leads to a trade-off between spatial, spectral, and temporal resolutions. Compressed-sensing-based techniques solves this problem by utilizing the sparsity of natural scenes and measuring a hyperspectral datacube with much fewer measurements than that required by the Nyquist-Shannon sampling theorem [27,31,3841]. Nonetheless, their applications are restricted by a static optical architecture and a fixed compression ratio. For example, an imager with a high compression ratio cannot be applied to a complex object, while an imager with a low compression ratio is ineffective in measuring a simple scene.

To address this unmet need, we herein present tunable image projection spectrometry (TIPS), a Fourier-domain line-scan spectral imaging method with a tunable compression ratio. Unlike the conventional line-scan hyperspectral cameras, TIPS is built on an optical architecture with multiplexing advantages [4244]. By using a rotating Dove prism and a cylindrical field lens, TIPS measures the en face projections of an object, converting a 2D image to a 1D line. The resultant line images are further spectrally dispersed and captured by an area camera. We demonstrated the spectral imaging capability of TIPS with a hematoxylin and eosin (H&E) stained pathology slide. Additionally, we show that the combination of TIPS with a low-coherence full-field interferometer can further convert spectral information to depths, thereby enabling volumetric imaging with a tunable compression ratio.

2. System

The schematic of TIPS is shown in Fig. 1. The object plane is located at the front focal plane of an objective lens L1 ($f$ = 50 mm, $f/\# = 2$). L1 and L2 ($f$ = 50 mm, $f/\# = 2$) together form a 4f system, and a Dove prism (Thorlabs, PS992M) serves as the system stop and locates at the Fourier plane. The Dove prism is mounted on a motorized rotation stage (Thorlabs, PRM1Z8). When the Dove prism is rotated by an angle of $\theta $, the image of the object is rotated by $2\theta $. We positioned a cylindrical lens ($f$ = 15 mm, invariant axis is along y axis) 35 mm after L2. Here the cylindrical lens plays two roles: generating 1D en face projection along y axis like those in standard computed tomography (CT) and serving as a field lens to reimage the system stop to a slit plane (Thorlabs, S15K) along x axis. The tracing of chief and marginal rays in the two orthogonal planes is illustrated in Fig. 1(b). It is worth noting that along the y axis, the object plane is conjugated to the slit plane, while alone x axis the stop plane is conjugated to the slit plane. In $y$-$z$ plane, the cylindrical lens is effectively a plane-parallel plate (PPP) and transparent to the object. The focus shift and spherical aberration introduced by the cylindrical lens are compensated by shifting the slit plane along the optical axis. In $x$-$z$ plane, the cylindrical lens integrates the intermediate image along x axis to form en face projection. Moreover, it demagnifies the pupil image to increase the light throughput. After passing the slit, the line image is spectrally dispersed by a diffraction grating (600 groves/mm) and reimaged by another 4f system (L3 and L4, $f$ = 50 mm, $f/\# $ = 1.4) to an area camera (Thorlabs, CS235MU).

 figure: Fig. 1.

Fig. 1. Optical system of a tunable image projection spectrometer (TIPS). (a) System schematic. (b) Chief ray and marginal ray path in x-z and y-z plane.

Download Full Size | PDF

The image reconstruction pipeline is shown in Fig. 2. At each Dove prism rotation angle, we capture a dispersed projection $({x,\; \lambda } )$. To minimize the correlations between projections and maximize information content for reconstruction, projections are sampled uniformly in an angular range [0, $\pi $]. Upon completion of acquisition of a $({x,\; \lambda ,\; \theta } )$ datacube at selected angles, the datacube is rearranged to a wavelength-dependent sinogram stack ${({x,\; \theta } )_\lambda }$. Next, we construct a 2D spatial image $({x,\; y} )$ using the sinogram $({x,\; \theta } )$ at each wavelength through filtered inverse Radon transformation. Other algorithms such as fast iterative shrinkage-thresholding algorithm (FISTA) [45] or deep learning can be used to further increase the image quality at the expense of an increased computation cost [46]. The final spectral datacube $({x,\; y,\; \lambda } )$ is computed by converting sinograms at all wavelengths to the corresponding spatial images. Detailed mathematic model of imaging formation and reconstruction can be found in Appendix (Section 5).

 figure: Fig. 2.

Fig. 2. Image reconstruction pipeline.

Download Full Size | PDF

3. Results

3.1 Spatial resolution, tunable imaging

To measure spatial resolution, we directly imaged a negative USAF resolution target at the object plane (i.e., the front focal plane of L1) under monochromatic illumination at 532 nm. Like the sparse-view computed tomography (CT), TIPS measures only a subset of projections, whose total number is smaller than the pixel resolution of the input image [47]. Therefore, the corresponding inverse problem is under-determined and, generally, the more projections the system acquires, the higher the image quality. Reconstruction results with the total number of projections = 90, 45, and 15 are shown in Fig. 3(a), (b), and (c), respectively. In each panel, the sinogram is shown at upper-left, and the reconstructed image is shown at the bottom, and a boxed zoom-in area from the reconstructed image is shown at upper-right. We define the data compression ratio $\gamma $ as:

$$\gamma = \frac{{{N_y}}}{{{N_\theta }}}, $$
where ${N_y}$ is the number of pixels along the y axis in the recovered image, and ${N_\theta }$ is the number of projections. The reconstructed image has a resolution of $300 \times 300$ pixels, corresponding compression ratios 3.3, 6.6, and 20 in Fig. 3(a), (b) and (c), respectively. The spatial resolutions were measured by calculating the visibility of bar features in each reconstructed image with a threshold 0.2, and the results are 7 µm, 11 µm, and 16 µm, respectively (in Appendix). It is worth noting that a USAF target is generally considered a non-sparse object. Therefore, the image quality is more sensitive to the projection number than a sparse object, evidenced by the degraded image quality as the compression ratio increases. By contrast, when imaging a sparse object like a butterfly pattern, a much fewer number of projections are sufficient to recover a high-fidelity image (detailed in Appendix). Therefore, the tunable compression ratio allows an effective measurement for a given object.

 figure: Fig. 3.

Fig. 3. Reconstructed USAF resolution target with (a) 90 projections, (b) 45 projections, and (c) 15 projections. In each panel, upper-left: sinogram; bottom: reconstructed image; upper-right: a boxed area in the reconstructed image.

Download Full Size | PDF

The spatial resolution of TIPS is fundamentally limited by diffraction like in a conventional optical system. However, because TIPS captures only a limited number of projections, the image quality is practically determined by the sparsity of the object like that in sparse-view CT. In TIPS, we can tune the number of projection measurements based on the prior knowledge of object sparsity and, thereby, provide balanced spatiotemporal resolutions.

When imaging the USAF target, we set the exposure time to ∼ 15 ms for each frame. With an additional 200 ms for rotating the dove prism between adjacent steps, the total acquisition time is ∼ 20 seconds.

3.2 Spectral resolution, spectral imaging

To evaluate spectral resolution, we recorded the system’s responses on the camera under monochromatic illumination at different wavelengths. Our results show that a 1 nm spectral bandwidth occupies 5.5 pixels on the camera, corresponding to a spectral sampling of 0.18 nm, and spectral resolution is 0.36 nm (detailed in Appendix). Next, we tested the system on a lung cancer hematoxylin and eosin (H&E) stained pathology slide illuminated by a broadband halogen lamp (Amscope, HL250-AS) in the transmission mode. Four represented reconstructed images at 530 nm, 550 nm, 570 nm, and 590 nm are shown in Fig. 4(a). The color of each image was rendered according to the corresponding wavelength based on CIE 1931 observer. A video shows the sweeping of wavelengths of the spectral image stack from 510 to 590 nm, with a step size = 1 nm (Visualization 1). As shown in the figure, hematoxylin (area 1) and eosin (area 2) have a significant spectral discrepancy in the 540-590 nm spectral range. Relative absorption spectra of area 1 and area 2 are shown in Fig. 4(b). The relative absorption at each wavelength is calculated by subtracting the transmission intensity from the background intensity.

 figure: Fig. 4.

Fig. 4. Spectral imaging of a lung cancer hematoxylin and eosin (H&E) slide in transmission mode. (a) Spectral images at 530 nm, 550 nm, 570 nm, and 590 nm. (b) Relative absorption spectra of area 1 and area 2. The relative absorption at each wavelength is calculated by subtracting the transmission intensity from the background intensity.

Download Full Size | PDF

3.3 Depth imaging with full-field spectral domain interferometer

We further demonstrated the combination of TIPS with a low-coherence full-field spectral domain interferometer [48,49]. The system layout is shown in Fig. 5(a), which consists of a Michelson interferometer and TIPS. We illuminate the sample with a low-coherent light source (a halogen lamp; Amscope, HL250-AS) filtered with a 40 nm bandpass filter centered at 550 nm (Thorlabs, FB550-40), yielding a 3.8 µm depth resolution and a 210 µm depth range in the final reconstructed volumetric image. The focal length of L1 and L2 are 50 mm and 150 mm, respectively. The scattered light from the sample combines with the light reflected from the reference mirror at a beam splitter, forming an interferogram at the input plane of TIPS. After acquiring the corresponding spectral datacube, we compute the depths by applying a Fourier transform to the spectrum acquired at each spatial location. To evaluate the system, we imaged two crossed hairs extending along orthogonal directions at two different depths. The results are shown in Fig. 5(b), where the reconstructed en face images are shown in the first row, B-scan (i.e., cross-sectional) image is shown in the middle, and a 3D volumetric image is shown at the bottom. The measured depths of hair 1 and 2 are 26.6 µm and 98.8 µm, respectively. A video shows sweeping of depths of the en face image stack from 3.8 to 190 µm, with a step size = 3.8 µm (Visualization 2).

 figure: Fig. 5.

Fig. 5. Depth imaging with full-field spectral domain interferometer. (a) System layout. (b) Reconstructed results of two intersecting hairs at two depths. First row, en face reconstructed results; middle, B-scan cross-sectional image; bottom, a 3D volumetric image of two hairs.

Download Full Size | PDF

4. Discussion

Measuring a line projection at a time, TIPS is a counterpart of conventional pushbroom spectrometry in the Fourier domain. However, TIPS outperforms pushbroom spectrometers in both light throughput and signal-to-noise ratio (SNR).

  • a. Light throughput (Jacquinot advantage). In conventional pushbroom imaging spectrometers, a slit is positioned at a conjugate image plane to measure only one line of the image at a time. The line image is then spectrally dispersed by a diffraction grating or a prism. Given widefield illumination, the light throughput equals the ratio of the slit width to the conjugate imaging field of view (FOV) at the slit plane, i.e., $\eta = {d_{slit}}/\textrm{FOV}$. On the other hand, the slit width determines the image resolution and the space-bandwidth-product (SBP), SBP=${({\textrm{FOV}/{d_{slit}}} )^2}$. Therefore, there is a trade-off between the light throughput and the image resolution and SBP. In contrast, in TIPS, the slit locates at the pupil plane at the x-z plane. The light throughput equals the ratio of the slit width to the conjugate pupil diameter (D) at the slit plane, i.e., $\eta = {d_{slit}}/D$, where D relates the stop size (${D_0}$) through $D = {D_0}/M$. Here M is pupil demagnification ratio in the x-z plane, and $M = {f_{L2}}/{f_{CL}}$, where ${f_{L2}}$ and ${f_{CL}}$ are the focal lengths of lens L2 and the cylindrical lens, respectively. Because ${f_{CL}}$ does not contribute to the image magnification in the y-z plane, we can use a cylindrical lens with a short focal length to increase the pupil demagnification ratio, reducing the conjugate pupil diameter (D) at the slit plane and, thereby, increasing the light throughput. Moreover, since TIPS scans in the Fourier domain, the slit width is decoupled with the image resolution, eliminating the trade-off between the light throughput and image resolution/SBP.
  • b. SNR (Fellgett advantage). TIPS performs measurement through multiplexing signals in the spatial domain. If the measurement contains only signal-independent noise sources, such as random thermal noise or readout noise in the cameras, the multiplexing approach can enhance the SNR of the recovered signal, which is known as the Fellgett advantage [50,51]. Like that in Fourier transform infrared spectroscopy (FTIR), TIPS can improve the SNR by a factor of $N\sqrt P $ compared with its spatial-scanning counterpart (derivation provided in Appendix). Here N denotes the image dimension in pixels, and P is the number of projections.
We summarize the competitive advantage of TIPS over pushbroom imaging spectrometry in Table 1.

Tables Icon

Table 1. Comparison of TIPS and pushbroom scanner in Jacquinot advantage, Fellgett advantage and compression ratio (${\boldsymbol M} = {{\boldsymbol f}_{{\boldsymbol L}2}}/{{\boldsymbol f}_{{\boldsymbol CL}}}$, pupil demagnification ratio in TIPS; ${\boldsymbol \eta } = {{\boldsymbol d}_{{\boldsymbol slit}}}/{\mathbf FOV}$, light throughput in a pushbroom hyperspectral imager; ${\boldsymbol N}$, number of integrated pixel along ${\boldsymbol x}$ axis in TIPS;${\boldsymbol P}$, number of projections that used for image reconstruction in TIPS).

In summary, we developed a new category of spectral imaging methods, TIPS, which can provide a tailored data compression ratio for a given target. TIPS also outperforms conventional pushbroom imaging spectrometers in both light throughput and SNR. Seeing its advantages, we expect TIPS can find broad applications in various disciplines.

Appendix

A. Image formation and reconstruction model

The system layout is shown in Fig. 1 and a photograph of the real system in shown in Fig. 6. For each wavelength, the image formation can be formulated as:

$${{\boldsymbol f}^\theta }(\lambda )= {\boldsymbol T}{{\boldsymbol R}^\theta }g(\lambda )+ \sigma ,$$
where $g(\lambda )$ is the ground truth image at wavelength $\lambda $, ${{\boldsymbol R}^\theta }$ is the image rotation matrix introduced by dove prism at an angle of $\theta /2$, ${\boldsymbol T}$ is the transfer function of the optical system, $\sigma $ denotes the system noise, and ${{\boldsymbol f}^\theta }(\lambda )$ is the monochromatic projection corresponding to angle $\theta $. Projections are sampled uniformly in an angular range [0, π]. Therefore, the whole image formation process is modeled as:
$${\boldsymbol f}(\lambda )= {\boldsymbol T}\left[ {\begin{array}{c} {{{\boldsymbol R}^0}}\\ \vdots \\ {{{\boldsymbol R}^{\boldsymbol \pi }}} \end{array}} \right]g(\lambda )+ \sigma = {\boldsymbol A}g(\lambda )+ \sigma , $$
where $\boldsymbol{A}$ is the system operator describing the rotation and compression of the ground truth image. The final spectral datacube can be reconstructed using the following equation:
$$\mathop {\textrm{argmin}}\limits_{\hat{g}} \|{\boldsymbol f}\; - \; {\boldsymbol A}g \|_2^2 + \rho \|\varphi {(g )\|_1},$$
where $\varphi (g )$ is the regularization function that sparsifies the image, and $\rho $ is the hyperparameter that weights the regularization term. Equation (3) can be solved using an established iterative algorithm [47].

 figure: Fig. 6.

Fig. 6. A photograph of TIPS.

Download Full Size | PDF

B. Image reconstruction for sparse object

To demonstrate the TIPS in imaging sparse object, we imaged a butterfly pattern that printed on a glass substrate, and it was directly put at the object plane of TIPS. Four representative reconstructed images with different projection numbers are shown in Fig. 7(a). The normalized mean-squared error versus projection number for both butterfly pattern and the USAF resolution target (Fig. 3) are shown in Fig. 7(b). The reconstructed results based on 90 projections are used as ground truth. The results indicate that 30 projections are sufficient to faithfully reconstruct the butterfly pattern, with only 16% errors are remaining.

 figure: Fig. 7.

Fig. 7. Sparse object imaging. (a) En face reconstruction results with different projection numbers for a printed butterfly pattern. (b) Normalized mean-squared error versus projection number for both butterfly pattern and the USAF resolution target (Fig. 3).

Download Full Size | PDF

C. Spatial resolution

To quantify spatial resolutions in Fig. 3, we calculated the visibility in each reconstructed image. A representative reconstructed image using 90 projections in Fig. 3(a) is shown in Fig. 8(a). We plotted the intensity along the yellow dashed line, and the result is shown in Fig. 8(b). The image visibility is defined as $\frac{{{I_{max}} - {I_{min}}}}{{{I_{max}} + {I_{min}}}}$, and I is the intensity. For each group of bars, we calculated the visibility using the intensity of peaks and valleys. Using a threshold 0.2, the spatial resolution in Fig. 3(a), (b) and (c) are 7 µm, 11 µm, and 16 µm, respectively.

 figure: Fig. 8.

Fig. 8. Spatial resolution. (a) A reconstructed image using 90 projections in Fig. 3(a). (b) Intensity plot along the yellow dashed line in (a). For each group of bars, we calculate the visibility using the intensity of peaks and valleys.

Download Full Size | PDF

D. Spectral resolution and sampling

To calibrate the spectral response, we placed a pinhole (Thorlabs, P20D) at the object plane of TIPS, and illuminated it with monochromatic light of different wavelengths. The corresponding pixel locations of the projections were recorded. The results were fitted with a linear polynomial, as shown in Fig. 9(a). The slope of the line indicates the spectral sampling. Because 1 nm bandwidth in wavelength occupied 5.5 pixels on the camera, the spectral sampling of the system is 0.18 nm. The spectral resolution was measured as the full-width half-maximum (FWHM) of the spectral response when the illumination wavelength is 532 nm. Here, we limited the source wavelength using a 1 nm bandpass filter (Thorlabs, FL532-1). The raw measurement result is shown in Fig. 9(b). The raw measurement has a FWHM of ∼9 pixels. However, this width is a convolution of the geometric image of the pinhole on the camera (∼3 pixels), the bandwidth of the light source (corresponding to ∼6 pixels on the camera), and the system spectral resolution. Because the width of a convoluted function can be computed as [52]:

$$w({{f_1}\ast {f_2}\ast {f_3}} )= w({{f_1}} )+ w({{f_2}} )+ w({{f_3}} )- 2,$$
where w denotes the width of the function, ${\ast} $ denotes the convolution operator, and ${f_i}$ (i = 1:3) denotes the individual function in a discrete form, the width of the spectral resolution on the camera is derived to be 2 pixels. Given a 0.18 nm spectral sampling, the spectral resolution is, therefore, 0.36 nm.

 figure: Fig. 9.

Fig. 9. Spectral calibration. (a) Spectral response locations in pixel of different wavelengths on the camera. (b) Raw response under 532 nm illumination (bandwidth = 1 nm).

Download Full Size | PDF

E. Validation of system performance

We quantitatively compared the spectral responses of TIPS to a standard fiber spectrometer (STSIS-L-25-400-SMA, Ocean Optics, Inc.). We first captured an image of a color checker target (ColorChecker Passport Photo 2, X-Rite) using TIPS and recovered the spectrum. Next, we measured the spectrum of the same checker using the fiber spectrometer. The normalized spectra obtained by TIPS and the fiber spectrometer are shown in Fig. 10(a), matching well with each other. The full spectral range of TIPS is ∼150 nm (450-600 nm), which is limited by the bandpass filter in the system. In Fig. 4(b), we showed the results only in a selected spectral range (510-590 nm) because this range shows the most significant difference between the spectra of hematoxylin and eosin. On the camera, we read out only part of the chip that corresponds to the spectral range of the system.

 figure: Fig. 10.

Fig. 10. Accuracy of spectral and depth measurements. (a) Comparison between normalized spectra measured by TIPS and a standard fiber spectrometer (ground truth). (b) Comparison between derived depths from interferometry and ground truth.

Download Full Size | PDF

To validate the depth accuracy in the combined system (a spectral-domain interferometer and TIPS), we first positioned a planar specular-reflection object (a mirror) at a plane conjugated with the reference mirror in the interferometer. Next, we translated the reference mirror along the optical axis from 25.4 µm to 203.2 µm with a step size of 25.4 µm. At each step, the recovered depth from interferometry matches well with the ground truth (Fig. 10(b)). The mean squared error is 1 µm.

F. Fellgett advantage of TIPS

In a traditional detector-noise-dominated spectral system, the image is represented as:

$$s({x,y} )= g({x,y} )+ n({x,y} ),\; $$
where $s({x,y} )$ is the measured 2D image, $g({x,y} )$ is the ground truth image, and $n({x,y} )$ is signal-independent gaussian noise, with a standard deviation $\sigma $. Assuming the mean of the noise is 0, we have:
$${\sigma ^2} = \langle n{({x,y} )^2}\rangle - \langle n{({x,y} )\rangle^2} = \; \langle n{({x,y} )^2\rangle},$$
where $\langle \cdot \rangle$ denotes the mean. Consider a uniform object with intensity I, the signal-to-noise ratio (SNR), therefore, can be calculated as:
$$\frac{S}{N} = \frac{I}{\sigma }.$$
In TIPS, the input scene is first rotated by a dove prism, then intensity along x axis is integrated by the cylindrical lens. The image formation in TIPS is analogous to that in a standard computed tomography, which can be described by Radon transform:
$$g_\theta^{\prime}(r )= \smallint \smallint g({x,y} )\delta ({xcos(\theta )+ ysin(\theta )- r} )dxdy,\; $$
where $g({x,y} )$ is the 2D ground truth image, and $g_\theta^{\prime}(r )$ is the integrated image at projection angle $\theta $, and r denotes the projected line, which satisfies $r = xcos(\theta )+ ysin(\theta )$. The signal recorded by the detector array is:
$$s_\theta^{\prime}(r )= g_\theta^{\prime}(r )+ {n_\theta }(r ),$$
whe $s_\theta^{\prime}(r )$ is measured 1D projection at an object angle $\theta $, ${n_\theta }(r )$ is a noise vector that has the same number of elements as a projection. For simplicity, we use direct back projection to reconstruct the image. We assume N pixels are integrated along each projection direction. Considering a uniform object with intensity I, we have:
$$s_\theta^{\prime}(r )= NI + {n_\theta }(r ).$$
Reconstructed with M projection measurements, the signal intensity is:
$$\hat{g}({x,y} )= NMI + \mathop \sum \nolimits_{k = 1}^M \hat{n}({x,y,k} ),$$
where $\hat{g}({x,y} )$ is estimated reconstruction image, and $\hat{n}({x,y,k} )$ is the back-projected 2D noise image at projection number = $k$. Define $m = \mathop \sum \nolimits_{k = 1}^M \hat{n}({x,y,k} )$ and the standard deviation of m is ${\sigma _m}$. Therefore, the SNR in TIPS is:
$$\frac{S}{N} = \frac{{NMI}}{{{\sigma _m}}}.$$
For each k, the noise at each spatial pixel $({{x_0},{y_0}} )$ can be considered as an independent variable with a same Gaussian probability density function (PDF), which has a zero mean and a standard deviation $\sigma $. The characteristic function of m is:
$$\; \; \; {\varphi _m}(w )= {\varphi _{k = 1}}(w ){\varphi _{k = 2}}(w )\ldots {\varphi _{k = M}}(w ). $$
The above equation can be further simplified as:
$${\varphi _m}(w )= {e^{ - \frac{{{w^2}({{\sigma^2}} )}}{2}}}{e^{ - \frac{{{w^2}({{\sigma^2}} )}}{2}}} \ldots {e^{ - \frac{{{w^2}({{\sigma^2}} )}}{2}}} = {e^{ - \frac{{{w^2}({M{\sigma^2}} )}}{2}}}. $$
From above equation, we can see that the characteristic function of m is the characteristic function of a gaussian function with a variance = $M{\sigma ^2}$. Therefore, ${\sigma _m} = \; \sqrt M \sigma $. The SNR in TIPS can be calculated as:
$$\frac{S}{N} = \frac{{N\sqrt M I}}{\sigma }. $$
The resultant Fellgett’s advantage factor is $N\sqrt M $.

Funding

National Institutes of Health (R01EY029397).

Acknowledgement

We thank Rishyashring R. Iyer and Stephen A. Boppart, M.D., Ph.D. at University of Illinois Urbana-Champaign for their insightful discussion.

Disclosures

The authors declare no conflicts of interest.

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request. Code used for this work are available from the corresponding author upon reasonable request.

References

1. W. Jahr, B. Schmid, C. Schmied, F. O. Fahrbach, and J. Huisken, “Hyperspectral light sheet microscopy,” Nat. Commun. 6(1), 7990 (2015). [CrossRef]  

2. S. Karpf, M. Eibl, W. Wieser, T. Klein, and R. Huber, “A time-encoded technique for fibre-based hyperspectral broadband stimulated Raman microscopy,” Nat. Commun. 6(1), 6784 (2015). [CrossRef]  

3. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19(1), 010901 (2014). [CrossRef]  

4. Q. Pian, R. Yao, N. Sinsuebphon, and X. Intes, “Compressive hyperspectral time-resolved wide-field fluorescence lifetime imaging,” Nat. Photonics 11(7), 411–414 (2017). [CrossRef]  

5. F. Yesilkoy, E. R. Arvelo, Y. Jahani, M. Liu, A. Tittl, V. Cevher, Y. Kivshar, and H. Altug, “Ultrasensitive hyperspectral imaging and biodetection enabled by dielectric metasurfaces,” Nat. Photonics 13(6), 390–396 (2019). [CrossRef]  

6. K. Monakhova, K. Yanny, N. Aggarwal, and L. Waller, “Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array,” Optica 7(10), 1298–1307 (2020). [CrossRef]  

7. C. Ma, X. Cao, R. Wu, and Q. Dai, “Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system,” Opt. Lett. 39(4), 937–940 (2014). [CrossRef]  

8. C. Deng, X. Hu, J. Suo, Y. Zhang, Z. Zhang, and Q. Dai, “Snapshot hyperspectral imaging via spectral basis multiplexing in Fourier domain,” Opt. Express 26(25), 32509–32521 (2018). [CrossRef]  

9. J. Jung, K. Kim, J. Yoon, and Y. Park, “Hyperspectral optical diffraction tomography,” Opt. Express 24(3), 2006–2012 (2016). [CrossRef]  

10. A. F. H. Goetz, G. Vane, J. E. Solomon, and B. N. Rock, “Imaging spectrometry for earth remote sensing,” Science 228(4704), 1147–1153 (1985). [CrossRef]  

11. J. Qin, M. S. Kim, K. Chao, D. E. Chan, S. R. Delwiche, and B.-K. Cho, “Line-scan hyperspectral imaging techniques for food safety and quality applications,” Appl. Sci. 7(2), 125 (2017). [CrossRef]  

12. M. Abdo, V. Badilita, and J. Korvink, “Spatial scanning hyperspectral imaging combining a rotating slit with a Dove prism,” Opt. Express 27(15), 20290–20304 (2019). [CrossRef]  

13. K. Uto, H. Seki, G. Saito, Y. Kosugi, and T. Komatsu, “Development of a low-cost hyperspectral whiskbroom imager using an optical fiber bundle, a swing mirror, and compact spectrometers,” IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing 9(9), 3909–3925 (2016). [CrossRef]  

14. Y. Yu, Y. Tang, K. Chu, T. Gao, and Z. J. Smith, “High-resolution low-power hyperspectral line-scan imaging of fast cellular dynamics using azo-enhanced raman scattering probes,” J. Am. Chem. Soc. 144(33), 15314–15323 (2022). [CrossRef]  

15. V.-D. Tuan, “A hyperspectral imaging system for in vivo optical diagnostics,” IEEE Eng. Med. Biol. Mag. 23(5), 40–49 (2004). [CrossRef]  

16. L. Gao and R. T. Smith, “Optical hyperspectral imaging in microscopy and spectroscopy – a review of data acquisition,” J. Biophotonics 8(6), 441–456 (2015). [CrossRef]  

17. P.-H. Cu-Nguyen, A. Grewe, M. Hillenbrand, S. Sinzinger, A. Seifert, and H. Zappe, “Tunable hyperchromatic lens system for confocal hyperspectral sensing,” Opt. Express 21(23), 27611–27621 (2013). [CrossRef]  

18. N. Gat, “Imaging spectroscopy using tunable filters: a review,” Proc. SPIE 4056, 50–64 (2000). [CrossRef]  

19. M. C. Phillips and N. Hô, “Infrared hyperspectral imaging using a broadly tunable external cavity quantum cascade laser and microbolometer focal plane array,” Opt. Express 16(3), 1836–1845 (2008). [CrossRef]  

20. L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact Image Slicing Spectrometer (ISS) for hyperspectral fluorescence microscopy,” Opt. Express 17(15), 12293–12308 (2009). [CrossRef]  

21. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express 17(8), 6368–6388 (2009). [CrossRef]  

22. X. Yuan, T. Tsai, R. Zhu, P. Llull, D. Brady, and L. Carin, “Compressive hyperspectral imaging with side information,” IEEE J. Sel. Top. Signal Process. 9(6), 964–976 (2015). [CrossRef]  

23. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008). [CrossRef]  

24. T. He, Q. Zhang, M. Zhou, T. Kou, and J. Shen, “Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning,” Opt. Express 30(6), 9790–9813 (2022). [CrossRef]  

25. M. E. Pawlowski, J. G. Dwight, T.-U. Nguyen, and T. S. Tkaczyk, “High performance image mapping spectrometer (IMS) for snapshot hyperspectral imaging applications,” Opt. Express 27(2), 1597–1612 (2019). [CrossRef]  

26. J. Ma, D.-W. Sun, H. Pu, Q. Wei, and X. Wang, “Protein content evaluation of processed pork meats based on a novel single shot (snapshot) hyperspectral imaging sensor,” J. Food Eng. 240, 207–213 (2019). [CrossRef]  

27. Q. Cui, J. Park, Y. Ma, and L. Gao, “Snapshot hyperspectral light field tomography,” Optica 8(12), 1552–1558 (2021). [CrossRef]  

28. M. Descour and E. Dereniak, “Computed-tomography imaging spectrometer: experimental calibration and reconstruction results,” Appl. Opt. 34(22), 4817–4826 (1995). [CrossRef]  

29. M. R. Descour, C. E. Volin, E. L. Dereniak, T. M. Gleeson, M. F. Hopkins, D. W. Wilson, and P. D. Maker, “Demonstration of a computed-tomography imaging spectrometer using acomputer-generated hologram disperser,” Appl. Opt. 36(16), 3694–3698 (1997). [CrossRef]  

30. J. R. Weber, J. D. Anthony, J. T. Bruce, J. C. David, R. J. William, W. W. Daniel, H. B. Gregory, H. Mike, L. Alexander, and K. B. M. D. Devin, “Multispectral imaging of tissue absorption and scattering using spatial frequency domain imaging and a computed-tomography imaging spectrometer,” J. Biomed. Opt. 16(1), 011015 (2011). [CrossRef]  

31. G. R. Arce, D. J. Brady, L. Carin, H. Arguello, and D. S. Kittle, “Compressive coded aperture spectral imaging: an introduction,” IEEE Signal Process. Mag. 31(1), 105–115 (2014). [CrossRef]  

32. M. E. Gehm, R. John, D. J. Brady, R. M. Willett, and T. J. Schulz, “Single-shot compressive spectral imaging with a dual-disperser architecture,” Opt. Express 15(21), 14013–14027 (2007). [CrossRef]  

33. H. Arguello, C. V. Correa, and G. R. Arce, “Fast lapped block reconstructions in compressive spectral imaging,” Appl. Opt. 52(10), D32–D45 (2013). [CrossRef]  

34. H. Zhang, X. Ma, X. Zhao, and G. R. Arce, “Compressive hyperspectral image classification using a 3D coded convolutional neural network,” Opt. Express 29(21), 32875–32891 (2021). [CrossRef]  

35. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). [CrossRef]  

36. L. Gao, R. T. Smith, and T. S. Tkaczyk, “Snapshot hyperspectral retinal camera with the Image Mapping Spectrometer (IMS),” Biomed. Opt. Express 3(1), 48–54 (2012). [CrossRef]  

37. T.-U. Nguyen, M. C. Pierce, L. Higgins, and T. S. Tkaczyk, “Snapshot 3D optical coherence tomography system using image mapping spectrometry,” Opt. Express 21(11), 13758–13772 (2013). [CrossRef]  

38. Y. Xue, K. Zhu, Q. Fu, X. Chen, and J. Yu, “Catadioptric hyperspectral light field imaging,” in Proceedings of the IEEE International Conference on Computer Vision (2017), pp. 985–993.

39. Y. Rivenson, A. Stern, and B. Javidi, “Overview of compressive sensing techniques applied in holography [Invited],” Appl. Opt. 52(1), A423–A432 (2013). [CrossRef]  

40. D. J. Brady, K. Choi, D. L. Marks, R. Horisaki, and S. Lim, “Compressive Holography,” Opt. Express 17(15), 13040–13049 (2009). [CrossRef]  

41. R. Horisaki, J. Tanida, A. Stern, and B. Javidi, “Multidimensional imaging using compressive Fresnel holography,” Opt. Lett. 37(11), 2013–2015 (2012). [CrossRef]  

42. M. R. Descour, “Throughput advantage in imaging Fourier-transform spectrometers,” Proc. SPIE 2819, 285–290 (1996). [CrossRef]  

43. R. G. Sellar and G. D. Boreman, “Comparison of relative signal-to-noise ratios of different classes of imaging spectrometer,” Appl. Opt. 44(9), 1614–1624 (2005). [CrossRef]  

44. J. A. Decker, “Experimental realization of the multiplex advantage with a Hadamard-transform spectrometer,” Appl. Opt. 10(3), 510–514 (1971). [CrossRef]  

45. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imaging Sci. 2(1), 183–202 (2009). [CrossRef]  

46. X. Feng and L. Gao, “Ultrafast light field tomography for snapshot transient and non-line-of-sight imaging,” Nat. Commun. 12(1), 2179 (2021). [CrossRef]  

47. H. Kudo, T. Suzuki, and E. A. Rashed, “Image reconstruction for sparse-view CT and interior CT-introduction to compressed sensing and differentiated backprojection,” Quantum Imaging Med. Surg. 3(3), 147–161 (2013). [CrossRef]  

48. X. Tian, X. Tu, J. Zhang, O. Spires, N. Brock, S. Pau, and R. Liang, “Snapshot multi-wavelength interference microscope,” Opt. Express 26(14), 18279–18291 (2018). [CrossRef]  

49. R. R. Iyer, M. Žurauskas, Q. Cui, L. Gao, R. Theodore Smith, and S. A. Boppart, “Full-field spectral-domain optical interferometry for snapshot three-dimensional microscopy,” Biomed. Opt. Express 11(10), 5903–5919 (2020). [CrossRef]  

50. P. R. Griffiths, “Fourier transform infrared spectrometry,” Science 222(4621), 297–302 (1983). [CrossRef]  

51. T. Hirschfeld, “Fellgett's advantage in UV-VIS multiplex spectroscopy,” Appl. Spectrosc. 30(1), 68–69 (1976). [CrossRef]  

52. A. V. Oppenheim, J. Buck, M. Daniel, A. S. Willsky, S. H. Nawab, and A. Singer, Signals & Systems (Pearson Educación, 1997).

Supplementary Material (2)

NameDescription
Visualization 1       A video shows the sweeping of wavelengths of the hyperspectral image stack of a H&E slide from 510 to 590 nm, with a step size = 1 nm in research article "Tunable Image Projection Spectrometry".
Visualization 2       A video shows the sweeping of depths of the en face image stack of two intersecting hairs from z = 3.8 to 190 nm, with a step size = 3.8 nm in research article "Tunable Image Projection Spectrometry".

Data availability

The data that support the plots within this paper and other findings of this study are available from the corresponding author upon reasonable request. Code used for this work are available from the corresponding author upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1.
Fig. 1. Optical system of a tunable image projection spectrometer (TIPS). (a) System schematic. (b) Chief ray and marginal ray path in x-z and y-z plane.
Fig. 2.
Fig. 2. Image reconstruction pipeline.
Fig. 3.
Fig. 3. Reconstructed USAF resolution target with (a) 90 projections, (b) 45 projections, and (c) 15 projections. In each panel, upper-left: sinogram; bottom: reconstructed image; upper-right: a boxed area in the reconstructed image.
Fig. 4.
Fig. 4. Spectral imaging of a lung cancer hematoxylin and eosin (H&E) slide in transmission mode. (a) Spectral images at 530 nm, 550 nm, 570 nm, and 590 nm. (b) Relative absorption spectra of area 1 and area 2. The relative absorption at each wavelength is calculated by subtracting the transmission intensity from the background intensity.
Fig. 5.
Fig. 5. Depth imaging with full-field spectral domain interferometer. (a) System layout. (b) Reconstructed results of two intersecting hairs at two depths. First row, en face reconstructed results; middle, B-scan cross-sectional image; bottom, a 3D volumetric image of two hairs.
Fig. 6.
Fig. 6. A photograph of TIPS.
Fig. 7.
Fig. 7. Sparse object imaging. (a) En face reconstruction results with different projection numbers for a printed butterfly pattern. (b) Normalized mean-squared error versus projection number for both butterfly pattern and the USAF resolution target (Fig. 3).
Fig. 8.
Fig. 8. Spatial resolution. (a) A reconstructed image using 90 projections in Fig. 3(a). (b) Intensity plot along the yellow dashed line in (a). For each group of bars, we calculate the visibility using the intensity of peaks and valleys.
Fig. 9.
Fig. 9. Spectral calibration. (a) Spectral response locations in pixel of different wavelengths on the camera. (b) Raw response under 532 nm illumination (bandwidth = 1 nm).
Fig. 10.
Fig. 10. Accuracy of spectral and depth measurements. (a) Comparison between normalized spectra measured by TIPS and a standard fiber spectrometer (ground truth). (b) Comparison between derived depths from interferometry and ground truth.

Tables (1)

Tables Icon

Table 1. Comparison of TIPS and pushbroom scanner in Jacquinot advantage, Fellgett advantage and compression ratio ( M = f L 2 / f C L , pupil demagnification ratio in TIPS; η = d s l i t / F O V , light throughput in a pushbroom hyperspectral imager; N , number of integrated pixel along x axis in TIPS; P , number of projections that used for image reconstruction in TIPS).

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

γ = N y N θ ,
f θ ( λ ) = T R θ g ( λ ) + σ ,
f ( λ ) = T [ R 0 R π ] g ( λ ) + σ = A g ( λ ) + σ ,
argmin g ^ f A g 2 2 + ρ φ ( g ) 1 ,
w ( f 1 f 2 f 3 ) = w ( f 1 ) + w ( f 2 ) + w ( f 3 ) 2 ,
s ( x , y ) = g ( x , y ) + n ( x , y ) ,
σ 2 = n ( x , y ) 2 n ( x , y ) 2 = n ( x , y ) 2 ,
S N = I σ .
g θ ( r ) = g ( x , y ) δ ( x c o s ( θ ) + y s i n ( θ ) r ) d x d y ,
s θ ( r ) = g θ ( r ) + n θ ( r ) ,
s θ ( r ) = N I + n θ ( r ) .
g ^ ( x , y ) = N M I + k = 1 M n ^ ( x , y , k ) ,
S N = N M I σ m .
φ m ( w ) = φ k = 1 ( w ) φ k = 2 ( w ) φ k = M ( w ) .
φ m ( w ) = e w 2 ( σ 2 ) 2 e w 2 ( σ 2 ) 2 e w 2 ( σ 2 ) 2 = e w 2 ( M σ 2 ) 2 .
S N = N M I σ .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.