Abstract
Imaging Fourier-transform spectroscopy (IFTS) is a powerful method for biological hyperspectral analysis based on various imaging modalities, such as fluorescence or Raman. Since the measurements are taken in the Fourier space of the spectrum, it can also take advantage of compressed sensing strategies. IFTS has been readily implemented in high-throughput, high-content microscope systems based on wide-field imaging modalities. However, there are limitations in existing wide-field IFTS designs. Non-common-path approaches are less phase-stable. Alternatively, designs based on the common-path Sagnac interferometer are stable, but incompatible with high-throughput imaging. They require exhaustive sequential scanning over large interferometric path delays, making compressive strategic data acquisition impossible. In this paper, we present a novel phase-stable, near-common-path interferometer enabling high-throughput hyperspectral imaging based on strategic data acquisition. Our results suggest that this approach can improve throughput over those of many other wide-field spectral techniques by more than an order of magnitude without compromising phase stability.
© 2017 Optical Society of America
1. INTRODUCTION
Hyperspectral imaging in microscopy is a powerful method for quantifying the functional and morphological states of cells and tissues. Many biological molecules have distinct spectral signatures, which are sensitive to local biochemical microenvironments. By labeling cellular constituents with different fluorescence probes, their distribution and concentration can be studied qualitatively and quantitatively in vivo [1]. Multicolor spectral karyotyping of human chromosomes is one example [2]. Another is spectral fluorescence resonance energy transfer, which provides more precise and robust measurements of protein–protein interactions [3]. Moreover imaging based on many nonfluorescent analytes utilizing their complex vibronic Raman spectra may also be realized using hyperspectral imaging microscopy [4]. For many of these applications, improving imaging throughput is of high interest. Compared with point scanning techniques, wide-field microscopic modalities provide unmatched imaging speed. Using wide-field fluorescence imaging, Bakal et al. analyzed the morphological properties of large populations of cells to associate their protein expression to mechanistic steps of cell migration [5]. Perlman et al. used wide-field fluorescence microscopy for drug classification by studying the morphological responses of cells [6]. Thus, wide-field microscopy can be used for more informative studies in biology and pharmacology. Enabling fast hyperspectral imaging in wide-field modalities may open doors to new high-throughput imaging studies of complex biological system where tens to hundreds of constituents may need to be resolved.
Hyperspectral imaging can be viewed as a problem where a three-dimensional (3D) data cube (two spatial dimensions and the spectral dimension) is measured using an array of detectors multiplexed, at most, in two dimensions (for example, CCD or CMOS sensors). Therefore multiple (scanning) measurements must be acquired to complete the 3D measurement. Thus, hyperspectral imagers are often categorized as position scanning (pushbroom) versus wavelength scanning types [7]. Position scanning systems multiplex one spatial dimension and the spectral dimension, and scan across the other spatial dimension (pushbroom). Wavelength scanning type multiplexes the two spatial dimensions and scans across the wavelength dimension. Another popular approach is imaging Fourier-transform spectroscopy (IFTS). Similar to wavelength scanning type, IFTS multiplexes the two spatial dimensions, but the scanning is performed in an orthogonal Fourier representation of the wavelength dimension. IFTS systems use an interferometer to make spectral measurements. The interferometer makes two copies of the input image, optically delays one copy (signal arm), and interferes it with the other (reference arm). The optical path-length difference (OPD), between the signal arm and reference arm, is changed successively and the interference pattern (interferogram) is recorded at each OPD for all the pixels in the field of view (FOV) in parallel.
The unique property of Fourier transform spectroscopy is that the measurements are taken in the Fourier space. This measurement scheme satisfies the incoherence property [8] of modern compressive sensing theory. Therefore, when a sparse representation of the spectrum to be measured exists, fewer measurements can be used to fully recover the spectrum [8,9]. IFTS has many different designs and not all are adaptable for compressive data acquisition. Importantly, the interferometer should enable OPD scanning independent of the spatial position of pixels. Any spatial dependency restricts the capability of random incoherent measurements of spectrum for all the pixels in the FOV. Furthermore, in cases where a specific set of OPD positions performs better than a random set (see Section 3.C.1 for an example), it is imperative that the OPD values be constant throughout the FOV at the detector of the interferometer. In a slightly different wording, OPD across the FOV should be uniform for best compressive data acquisition.
Most commonly implemented interferometer designs for IFTS are based on either Michelson interferometry [4,10,11] or Sagnac interferometry [12]. Designs based on Michelson interferometer are straightforward and the OPD is uniform across the whole FOV (Fig. S1.A), allowing compressive spectral recovery from strategically sampling at very few OPD values. However, Michelson designs are not common-path and typically phase-unstable due to their sensitivity to external vibrations. Moreover, in the original form, their pitch and yaw stability are hard to maintain during interferometer mirror translation [13]. Attempts have, however, been made to improve Michelson designs. Miniaturized designs may improve phase stability, but require extremely sophisticated control mechanisms to maintain pitch and yaw stability [14]. An important modification is the replacement of mirrors with retroreflectors, which correct for pitch and yaw stability [15]. There is still a lateral displacement, which in turn produces a shear that has an effect on the interferogram. However, in practice, it is easier to meet the tolerance on the shear than the tilt [15]. In both plane mirror- and retroreflector-based Michelson designs, feedback control is usually needed for accurate OPD stepping. Additionally, active feedback control may also be used to correct for thermal drifts or motion nonlinearity. Müller et al. recently demonstrated a retroreflector-based Michelson design for IFTS microscopy, which has a high acceptance angle and a large travel range with feedback-controlled translation [13]. In contrast to Michelson designs, designs based on Sagnac interferometer are very stable and have been implemented commercially for microscopy applications (see Applied Spectral Imaging, Inc.). However, their acceptance angle is inherently limited [16,17]. As a result, in an imaging configuration, they have a limited FOV and a limited spectral resolution [16]. Therefore, Sagnac is less suitable for applications such as Raman imaging that require high spectral resolution [13]. Moreover, in the Sagnac design, the OPD varies linearly across the FOV at any given instance and requires the zero-OPD line to be scanned across the entire FOV (Fig. S1.A, Supplement 1). This limits the operation of Sagnac to sequential scanning over a large OPD range and it does not support random OPD stepping. Therefore, Sagnac cannot be utilized for compressive spectral measurements. Furthermore, even in sequential operation mode, a flat-field interferometer works up to an order of magnitude faster than the Sagnac for florescence measurements at large FOVs (Section 1 in Supplement 1); in general, the throughput of Sagnac is inversely proportional to the image FOV (see Supplement 1 for a detailed analysis).
In this paper, we introduce a new interferometer design for IFTS, which is near-common-path and has a near-uniform OPD throughout the FOV. The near-common-path design enables higher phase-stable measurements compared with non-common-path architectures. The near-uniform OPD provides a high throughput compared with Sagnac, which has a linear OPD variation across the FOV. OPD uniformity further enables strategic OPD sampling for compressive spectral measurements. The proposed geometry also does not impose any constraints on the OPD scanning range. Therefore, it enables a large OPD range scanning for high-spectral-resolution applications.
2. OPTICAL DESIGN
The optical design of the proposed interferometer is similar to that of the conventional common-path phase microscope [18,19]. The common-path phase microscope uses a transmission grating as a beamsplitter. Gratings, in reflection mode, have also been used as beamsplitters in interferometer designs for (single-point) Fourier-transform spectrometers [20,21]. In our design, two transmission gratings are used to split and combine the image, forming the two arms of the imaging interferometer. The optical design is shown in Fig. 1(A1). At the microscope’s image plane (IP), a first transmission grating (G1, Edmund Optics, #46-073), which splits the incident beam into $+1$ and $-1$ orders, is placed. The two information-bearing beams are then relayed to a second IP using a $4\text{-}f$ system (L1, L2, Thorlabs, AC508-150-A-ML). At the Fourier plane of the $4\text{-}f$ system, a variable OPD is introduced to one arm by changing the effective thickness of a glass block (implementation details are discussed in Section 2.A). The two arms are combined at the second IP using a similar grating (G2, Edmund Optics, #46-073). The combined beams are again relayed to the camera (sCMOS, Hamamatsu, ORCA-Flash4.0) detection plane using a second $4\text{-}f$ system (L3, L4, Thorlabs, AC508-100-A-ML, AC508-200-A-ML). Note that the off-axis orders are blocked at the Fourier plane of the second $4\text{-}f$ system. The two on-axis beams form an interference pattern at the detector plane. The OPD is scanned and the interference patterns on the detector are recoded. The recorded interferograms contain the spectral information from sources at the sample plane on their conjugate positions at the detector plane, i.e., at detector’s pixels (see Section 2.B).
While the Sagnac designs are fully common-path with excellent stability, the proposed design is nearly common-path. The two interferometric arms mostly travel through the same optical components and the same environment except near the Fourier plane (please note that in Fig. 1(A1) the separation of the two arms—about 9 mm at the Fourier plane—was exaggerated for clarity). Therefore, the system is largely insensitive to external vibrations and is stable enough for spectral measurements of weak signals from biological molecules (see Section 4.A).
The next sections introduce implementation strategies for the OPD scanning mechanism, the mathematical formulation of the proposed optical design, and the calibration process of the instrument.
A. Implementations of a Variable-Phase-Delay Element Configured for OPD Scanning
Two independent techniques, one based on a prism setup and another based on a rotating slab, were used to introduce an OPD between the reference and signal arms of the interferometer. A selection can be made depending on the end-user application. The prism-based technique requires fabrication of a special optical element (Fig. S2 in Supplement 1) and, hence, is more expensive. But this technique supports a large OPD scanning range required for applications such as Raman imaging. The second technique is cheaper and can be implemented using off-the-shelf optical elements, but has a limited OPD scanning range.
1. Implementation 1: Prism-Based OPD Scanning
The combination of prisms discussed below is positioned across the reference ($-1$ order) and signal ($+1$ order) beams in the setup at the first Fourier plane (see Fig. 1(A1)). As shown schematically in Fig. 1(A2), a translation scanning mechanism (Physik Instrumente, PIHera P-629.1CD) is employed to move a negative prism with respect to a second stationary positive prism. The shape of the moving prism is complementary to the shape of the wedge of the stationary prism. The scanning mechanism, in operation, varies the thickness of the air gap between the wedges. The variation of the air gap in turn introduces a variable change in the optical path length of the signal beam as compared with that of the reference beam. The negative prism has an additional ${t}_{0}$ thickness over the positive prism to enable negative OPDs. The OPD introduced, $\tau $, can be written as
Here $\theta $, ${n}_{g}$, ${n}_{a}$, $u$, and ${t}_{0}$ are, respectively, the half wedge angle of the prisms, refractive index of glass, refractive index of air, the movement of the negative prism with respect to the touching position, and the additional thickness of the negative prism over the positive one (see Fig. 1(A2)). Here $\mathrm{sin}(\beta )=\mathrm{sin}(\theta ){n}_{g}/{n}_{a}$. According to the above equation, the translation of the negative prism, $u$, is directly proportional to the OPD, $\tau $.
The data presented in Sections 4.C and 4.D were collected using this implementation. The fabrication strategy of the optical elements is discussed in Section 2 of Supplement 1.
2. Implementation 2: OPD Scanning by Rotating an Optically Isotropic Transparent Slab of Material
In an alternative implementation, as shown in Fig. 1(A3), two thin optically isotropic transparent slabs of equal thicknesses are inserted across $+1$ order and $-1$ order diffracted beams. One of the slabs—as shown—is caused to rotate (Thorlabs, CR1/M-Z7) to increase the optical path length for one of the beams. The OPD, $\tau $, introduced can be written as
Here ${t}_{1}$, ${n}_{g}$, and ${n}_{a}$ are, respectively, the thicknesses of the glass slabs, refractive index of glass, and refractive index of air. ${\theta}_{0}$ and $\theta $ are, respectively, the angles between the glass slabs (fixed and rotating) and the normal to the optical axis as shown in Fig. 1(A3).
The data presented in Section 4.B were collected using this implementation.
B. Mathematical Formulation of the Interferometer Setup
The operating principles of the optical setup can be mathematically described as below. For simplicity, the electrical field ${E}_{0}$ at the first image plane (IP in Fig. 1(A1)) is assumed to be a plane wave:
Here $x$, $y$, and $z$ are spatial dimensions where $z$ is the direction of propagation. Denoted by $t$, $c$, and $k$ are the time, speed of light, and wavenumber, respectively. $E$ is the amplitude of the electric field. The electric field after the first grating is
Here ${k}_{0}$ is a constant, which depends on the grating period. Then at the first Fourier plane, a finite phase delay, $\varphi $, is introduced to the signal arm and, hence, the electric field before the second grating is
Adding the phase delay is equivalent to inserting a parallel glass slab of thickness $T$, and, hence,
where ${n}_{g}$ and ${n}_{a}$ are, respectively, the refractive indices of glass and air. After the second grating plane, the electric field isThen at the second Fourier plane, the off-axis orders are blocked and, hence, the electric field just before the detector plane is
Therefore, the intensity at the detector is
The above detected intensity at the detector is a function of $T$ and is called the interferogram for a particular $x$, $y$ position. The first term is a DC term and the second term, denoted by $J$ below, contains the spectral information of the light coming from the conjugate point at the sample:
For a specific pixel ${x}_{1}$, ${y}_{1}$ in the image, Eq. (10) can be rewritten as
For simplicity, ${x}_{1}$, ${y}_{1}$ indexes are dropped and the OPD is denoted by $\tau =T\mathrm{\Delta}n$. The source spectrum is redefined as $S(k)={E}^{2}(k)/8$:
Thus, the Fourier cosine transform of $J(\tau )$ gives the spectral information $S(k)$:
C. Alignment and Calibration
The optical design discussed provides a near-common-path imaging interferometer with a near-uniform OPD across the FOV. However, the phase uniformity (see Section 4.A) depends on the alignment accuracy. Thus, the instrument should be properly aligned and calibrated before acquiring data. The most critical part of the alignment process is to align the two images so that they overlap pixel-by-pixel. In order to satisfy this, the two gratings should be strictly paralleled to each other. During alignment, first the OPD scanning assembly and second the grating are removed. Then the grating lines of the first grating are focused on the detector. This ensures that the first grating is in the IP. Then the second grating is introduced. A target object (usually a USAF target) is placed on the microscope sample stage and the smallest features are placed in the FOV. Then the spatial position and the angle of the second grating are adjusted so that the images from the two arms overlap with each other. Once a coarse alignment is achieved, the second grating is further adjusted so that the number of fringes seen within the FOV is minimized. Then the OPD scanning assembly is inserted and the alignment is fine-tuned. While the overlapping is sensitive to the alignment process, it is common for any imaging interferometer. Once the instrument is aligned, it is calibrated as discussed below.
First, a laser light source was used across all pixels in the FOV and their interferograms were recorded (see Fig. 2(A1) for an interferogram of a representative pixel). Then the following process was independently performed for every pixel. A fast Fourier transform (FFT) of the interferogram was calculated and the positive frequencies were taken. The FFT, in this case, is a delta function with the frequency of the laser light. This information was used to calibrate the frequency axis of the FFT (consequently, the wavelength axis of the power spectrum) as a function of OPD step size. The resulting power spectrum is shown in Fig. 2(A2) and the peak refers to the wavelength of the laser light. However, OPD is a function of refractive index and each wavelength sees a slightly different OPD step size. The red plot in Fig. 2(B2) shows the power spectrum of a broadband light source through a band-pass filter, at a representative pixel, measured using the system after the above calibration (shown in Fig. 2(B1) is the corresponding interferogram). In the same plot, the reference spectrum measured using a commercial spectrometer is shown in black (from Ocean Optics). The mismatch at longer wavelengths is due to the dependence of refractive index (and, hence, of OPD step size) on the wavelength. To correct for this, a second calibration step was performed.
Wavelengths at the spectral features (cutoff values of the pass bands of the filter) of the reference spectrum (Fig. 2(B2), black) and the measured spectrum (Fig. 2(B2), red) were recorded. Then the two sequences of values were fitted to a second-degree polynomial to calibrate the wavelength axis (Fig. 2(C)). Figure 2(B2) (blue) shows the final calibrated spectrum of the broadband light through the filter, measured using the system. It can be noted that the two spectrometers have different sensitivities at different wavelength regions (compare the spectral power axis in Fig. 2(B2), black and blue). This can most likely be due the difference in the quantum efficiency responses of the sensors in the two instruments. One could further calibrate for this difference. However, the current calibration process is good enough for most of the spectra that are not fully broad band (see Fig. 2(D1) and 2(D2)).
The next section introduces the data processing methods used to reconstruct the spectral information form the interferometric measurements taken using the calibrated instrument.
3. DATA PROCESSING METHODS AND SIMULATIONS
For a discrete set of wavenumbers $\mathbf{k}=({k}_{1},{k}_{2},\dots {k}_{N})$ and a discrete set of OPD values $\mathit{\tau}=({\tau}_{1},{\tau}_{2},\dots {\tau}_{M})$, Eq. (12) can be written in the matrix form as
where $\mathbf{J}=J(\mathit{\tau})$, $\mathbf{S}=S(\mathbf{k})$, and $A$ is the corresponding discrete cosine transform matrix. However, common practice is to write $A$ as the discrete Fourier-transform (DFT) matrix for practical purposes. The interferometer takes noisy measurements ${\mathbf{J}}_{\mathbf{n}}$, where $\mathit{\eta}=({\eta}_{1},{\eta}_{2},\dots {\eta}_{M})$ is the noise vector. Recovering $\mathbf{S}$ from ${\mathbf{J}}_{\mathbf{n}}$ is a liner inverse problem and three solution methods—Fourier-transform-based spectral recovery, regularized L2 norm minimization using TwIST (two-step iterative shrinkage/thresholding), and greedy single-wavenumber search—were used in this paper.Fourier-transform-based spectral recovery is the standard method used in FTS and can be used for any spectral measurement in general. Regularized L2 norm minimization using TwIST can also be used for any spectral measurement. Importantly, it does not require sequentially acquired data. Furthermore, it provides a framework to incorporate prior information (such as sparsity) about the spectrum to enable under-sampled data acquisition or spectral de-noising post acquisition. Greedy single-wavenumber search is a specialized method for smooth fluorescence spectra. It measures the wavelength of the emission maximum, which can be used to identify and separate the fluorescence species in a specimen.
A. Fourier-Transform-Based Spectral Recovery
When $A$ is the full DFT matrix, $A$ is orthogonal and invertible. Ignoring the noise term $\mathit{\eta}$, a simple solution can be written as
This solution is completely equivalent to taking an inverse Fourier transform of the AC term of the interferogram and is traditionally used in FTS.
B. Regularized L2 Norm Minimization Using TwIST
A standard solver for the liner inverse problem in Eq. (14) can be formulated as a regularized L2 norm minimization [22]:
Here $\widehat{\mathbf{S}}$ is the estimated spectrum and ${\Vert \xb7\Vert}_{2}$ denotes the L2 norm. $R(\xb7)$ is a regularization function and can be used to incorporate prior information about the spectrum. $\gamma $ is a regularization parameter, which controls the contribution of the prior toward the optimization. This technique can be used when the measurement matrix, $A$, is ill-conditioned (as in compressed sensing) or when the measurement is corrupted by noise.
In compressed sensing, the interferogram is under-sampled and $A$ is a partial form of the DFT matrix with some missing rows. Therefore, the system is ill-conditioned and $R$ is formulated as a sparsity prior:
where ${\Vert \xb7\Vert}_{1}$ denotes the L1 norm. In this context, Eq. (16) can be written asMany efficient numerical solvers have been proposed for the optimization problem above [23–25] and we use a technique called two-step iterative shrinkage/thresholding algorithms (TwIST) originally proposed for image restoration [22].
C. Greedy Single-Wavenumber Search
In applications where the properties of the spectrum are of interest rather than the entire spectrum, the spectrum $\mathbf{S}$ can be approximated by a functional form with a fewer number of parameters. For instance, the most sensible representation for a fluorescence spectrum may be a Gaussian mixture representation. However, this involves a nonlinear representation of $\mathbf{S}$ and, hence, the system cannot be captured by a set of liner equations as in Eq. (14). Linear approximations and solvers for nonlinear forms of Eq. (14) have been proposed [26,27]. However, here as a proof of concept, we present a simpler linear version by assuming the spectrum as a single delta function. In a real fluoresce spectrum, the wavenumber represented by the delta function is the peak wavenumber of the spectrum. This idea can be formulated as
Here ${\Vert \xb7\Vert}_{0}$ denotes the L0 norm. This problem is then solved by exhaustively plugging in $\mathbf{S}$ with a single entry and greedily selecting the spectrum that minimizes the objective function in Eq. (19). The accuracy of the results depends on the selection of measurement positions, which ultimately determines the measurement matrix $A$ as demonstrated in the simulation experiment below.
1. Peak Wavelength Extraction Simulation Experiment
A Fourier-transform interferometer (FTS) was simulated in Matlab (see Section 2.B for the mathematical formulation of the system). A set of spectra were simulated for a set of quantum dots (QDs) with varying peak emission wavelengths spanning the visible spectrum (from 400 nm to 800 nm). Interferograms for each QD spectrum were measured computationally using the simulated FTS. A representative interferogram is shown in Fig. 3(A) and its corresponding spectrum is shown in Fig. 3(B). The interferogram was then randomly under-sampled (with $M$ measurements) and a “greedy single-wavenumber search” was used to recover the peak emission wavelength (${\lambda}_{M}$ in Fig. 3(B)) for the QD. Then the measurement error was calculated according the following definition:
Here ${\lambda}_{\mathrm{GT}}$ is the ground truth peak emission wavelength of the corresponding QD spectrum (see Fig. 3(B)).
The objective of this experiment was to investigate the effect of the selection of $M$ measurements on the measurement error. Even in the absence of any detection noise, an error can be made as the system may be ill-conditioned. Therefore, the experiment was first performed in the absence of any detection noise.
Let ${\mathit{\lambda}}_{\mathrm{QD}}=\{{\lambda}_{{\mathrm{QD}}_{1}},{\lambda}_{{\mathrm{QD}}_{2}},\dots \}$ be the set of ground truth peak emission wavelengths of the QDs. Let $N$ be the full length of the interferogram, ${I}_{{\mathrm{QD}}_{i}}$, and let $\mathit{\tau}=\{{\tau}_{1},{\tau}_{2},\dots {\tau}_{N}\}$ be the set of corresponding OPD positions. According to Nyquist sampling theorem, $N$ depends on the minimum wavelength, ${\lambda}_{\mathrm{min}}$, and the spectral resolution, ${\delta}_{\lambda}$, of the FTS system [28]:
On average for a 400–800 nm spectral window, $N=300$ at 1 nm spectral resolution. Then the following process was performed, interactively, for a sufficiently large number of iterations ($\sim 10,000$):
1: | Procedure Repeat and record $\hat{\mathit{\tau}}$ and $\mathit{\text{Error}}(\hat{\mathit{\tau}})$ |
2: | $\hat{\mathit{\tau}}\leftarrow \mathrm{select}\text{\hspace{0.17em}}M\text{\hspace{0.17em}}\mathrm{entries}\text{\hspace{0.17em}}\text{randomly}(\mathit{\tau})$ |
3: | $\forall \text{\hspace{0.17em}}{\lambda}_{{\mathrm{QD}}_{i}}\in {\mathit{\lambda}}_{\mathrm{QD}}$ |
4: | ${I}_{{\mathrm{QD}}_{i}}(\mathit{\tau})\leftarrow \mathrm{measure}\text{\hspace{0.17em}}\mathrm{using}\text{\hspace{0.17em}}\mathrm{FTS}({\mathrm{QD}}_{i})$ |
5: | ${\widehat{I}}_{{\mathrm{QD}}_{i}}\leftarrow {I}_{{\mathrm{QD}}_{i}}(\hat{\mathit{\tau}})$ |
6: | ${\widehat{\lambda}}_{{\mathrm{QD}}_{i}}\leftarrow \mathrm{greedy}\text{\hspace{0.17em}}\mathrm{single}\text{\hspace{0.17em}}\mathrm{wavenumber}\text{\hspace{0.17em}}\mathrm{search}\text{\hspace{0.17em}}({\widehat{I}}_{{\mathrm{QD}}_{i}})$ |
7: | ${\mathit{\text{Error}}}_{{\mathrm{QD}}_{i}}\leftarrow |{\widehat{\lambda}}_{{\mathrm{QD}}_{i}}-{\lambda}_{{\mathrm{QD}}_{i}}|$ |
8: | $\mathit{\text{Error}}(\hat{\mathit{\tau}})\leftarrow \mathrm{max}(\{{\mathit{\text{Error}}}_{{\mathrm{QD}}_{i}}\})$ |
The above process was performed for $M=\{1,2,3,\dots ,20\}$. For each $M$, the best, average, and worst measurement errors are plotted in Fig. 3(C). As can be seen, the error drops much faster in the best case (reaches a minimum at $M=7$) than in the average case. This implies that there are some OPD position sets, which perform considerably better than the average case. Therefore, an optimized set of OPD positions can be chosen for each $M$, and for a lower $M$, this optimum set performs much better than a blindly selected random set for the same $M$.
Second, the same simulation was performed for the optimum set of OPD positions selected above in the presence of Poisson noise at different signal levels. The signal level was quantified by the number of photons fed to the FTS system at each measurement. As shown in the figure, the same results as in the noiseless conditions can be achieved at 18 measurements with each measurement having 100 photons, i.e., with 1800 photons in total (blue plot in Fig. 3(D)).
Thus, this simulation suggests that for a set of smooth spectra, there exists some OPD position sets, which perform much better than a random set for under-sampled data acquisition.
4. RESULTS AND DISCUSSION
First, the performance parameters of the instrument, namely, OPD uniformity in the FOV and phase stability, are experimentally demonstrated and discussed. Then the system is validated on fluorescence specimens. Hyperspectral imaging for fluorescence beads and fluorescently labeled cells and tissue are presented. Then the instrument is demonstrated for hyperspectral Raman imaging. Exhaustive and under-sampled data acquisition and reconstruction for Raman imaging are also discussed. Finally, compressive measurements of fluorescence spectral features are demonstrated.
A. Performance Parameters
First, we demonstrate the OPD uniformity across the FOV. A reflective USAF target (Thorlabs, R3L3S1N) was illuminated with a 532 nm laser line (Coherent, Verdi V6) and the reflecting laser light was imaged (microscope objective used: Nikon, $40\times $, 0.75 NA, CFI Plan Fluor DLL). The resulting image is shown in Fig. 1(B) and the OPD was nearly uniform throughout the FOV. The physical distance between two dark fringes, $\mathrm{\Delta}X$, was measured to be 36 μm. The equivalent OPD, $\mathrm{\Delta}\mathrm{OPD}$, is the wavelength of the laser line, i.e., 0.5323 μm. Therefore, the OPD tilt across the FOV is
According to our measurements, a similar Sagnac interferometer has a phase tilt of 0.5 (see Section 1 of Supplement 1). Therefore, the phase tilt of the proposed interferometer is an order to two orders of magnitude (35 times for the two systems compared experimentally) less than that of a similar Sagnac. As discussed before, this suggests that even when the conventional Fourier-transform algorithms are used for spectral recovery, the proposed system is about an order of magnitude faster than the Sagnac interferometer for florescence measurements (see Section 1 in the Supplement 1).
Next we demonstrate the phase stability of the instrument in the following experiment. A reflective object was placed in the sample plane of the microscope and the OPD was fixed. The interference pattern at the IP was recorded every 50 ms over a period of 50 s (imaging objective: Nikon, $40\times $, 0.75 NA, CFI Plan Fluor DLL). Then the interference intensity measurements were converted to respective phase angles. Figure 1(C) shows the phase angle change (from its time average) versus time for a representative pixel. The maximum phase shift was less than 50 mrad, which refers approximately to a 4 nm OPD scan, which is well below the minimum OPD step size (200 nm for the visible spectrum, i.e., 400–800 nm) and well within the limits of the OPD scanning error of the actuator for closed-loop geometry ($\sim 12\text{\hspace{0.17em}}\mathrm{nm}$). More importantly, the error is $\lambda /100$ for the shortest wavelength measured, i.e., ${\lambda}_{\mathrm{min}}=400\text{\hspace{0.17em}}\mathrm{nm}$.
B. Hyperspectral Fluorescence Imaging
In this section, we validate the system by measuring known fluorescent spectra in fluorescence beads as well as biological specimens.
First, a specimen with heterogeneous fluorescent beads (Invitrogen fluorescence microscope test slide 2—F36913 FocalCheck) was excited using a 488 nm laser line (Coherent, Sapphire 488-100) and imaged (imaging objective: Nikon, $40\times $, 0.75 NA, CFI Plan Fluor DLL; imaging filters: Di02-R488, BLP01-488R-25) for multiple OPD positions. Five hundred images were taken with a 50 ms exposure time per image. The total measurement time was 50 s (including the OPD scanning time). The FOV imaged is shown in Fig. 4(A1). The specimen contained two types of 6 μm diameter beads. One type was with excitation and emission maxima (ex/em) of 503/511 nm and the other type was with ex/em of 511/524 nm. Figure 4(A2) shows their respective emission spectra (recovered by the FT-based method explained in Section 3.A). The spectra for all five beads were plotted in the same graph for easy comparison. The two types of emission spectra were recovered to a few nanometer accuracy ($<2\text{\hspace{0.17em}}\mathrm{nm}$ error).
Second, two types of biological specimen were imaged and spectra were recovered (using the FT-based method—Section 3.A). Figure 4(B1) shows an in vitro HeLa cell specimen where nuclei were labeled with AlexaFluor488. A 488 nm laser source was used to excite the cells and 300 images were recorded with varying OPD (imaging objective: Nikon, $40\times $, 0.75 NA, CFI Plan Fluor DLL; imaging filters: Semrock, Di02-R488, BLP01-488R-25). The exposure time per image was 100 ms and the total acquisition time, including OPD scanning time, was 45 s. Corresponding spectra for three representative pixels (marked and color-coded in Fig. 4(B1)) are shown in Fig. 4(B2). Next we present spectral measurements for a tissue specimen with multiple fluorophores in the presence of an autofluorescence background. A mouse muscle specimen with two types of fluorophores, hrGFPII (with a nuclear localization signal) and mYFP (without a localization signal), was prepared and imaged (Fig. 4(C1)). Mice were euthanized with an overdose of carbon dioxide and the tibialis anterior muscle was isolated. The muscle was fixed overnight in 4% paraformaldehyde and embedded in 2% low-melting-point agarose. The embedded tissue was frozen in an optimal cutting temperature compound medium and 10 μm sections were cut using a cryotome (Leica Microsystems). The sections were mounted on superfrost glass slides (Fisher Scientific) and embedded in a Vectashield hard set mounting medium (Innovative Biotech, Singapore). The same 488 nm laser source was used to excite mYFP (em/ex of 508/524) and hrGFPII (em/ex of 500/506) and the emission light was imaged in the same imaging conditions as in the previous experiment. Then the spectral cube was recovered for the entire FOV. The spectra of two representative pixels are shown in Fig. 4(C2). The slight mismatch of the spectra can be due to the autofluorescent background from the tissue. It is consistent with the fluorescence signal from flavin adenine dinucleotide (FAD) [29]. Figure 4(C3) shows the spectral images for five representative wavelengths. As expected, hrGFPII shows a bright signal for the 505 nm band, while mYFP shows a bright signal for the 525 nm band. The 565 nm band also shows some autofluorescence signal, most likely FAD [29].
C. Hyperspectral Raman Imaging
In this section, we demonstrate Raman spectral imaging of 4-acetamidophenol (A7302, Sigma-Aldrich), a well-known calibration specimen for Raman spectroscopy [30–32]. First, 4-acetamidophenol powder was put on a quartz coverslip and excited with a high-power 532 nm laser (Coherent, Verdi V6) with light-sheet-based illumination (see Fig. S3). The total power on the sample was 2.7 W. Then multiple images were taken (imaging objective: Olympus, $20\times $, 0.5 NA, UPlanFL N; imaging filters: Semrock, LPD02-532RU-$25\times 36\times 1.1$, two LP03-532RE-25, FF01-612/SP-25) over a large OPD range (over a 400 μm OPD range) to generate a Raman spectral image stack. Over 2900 OPD positions were first imaged at a Nyquist sampling rate. The exposure time per image was 280 ms. The total measurement time (including OPD scanning time) was 16 min. The OPD step size was 132 nm. The conventional Fourier-transform-based algorithm (see Section 3.A) was used to recover the spectrum from the interferograms for the entire FOV. A representative Raman spectrum is shown in Fig. 5(A) (shown in blue). Spectral images at five key spectral bands are shown in Fig. 5(C). Raman spectra usually do not span across the entire visible range and are band-limited. Therefore, the interferogram can be under-sampled without risking aliasing [13]. This is also known as coarsely sampled Fourier-transform spectroscopy [33]. Thus, 10 times under-sampled interferograms were taken (at 1.32 μm step size) by scanning 290 OPD positions. The exposure time per image was the same and the total measurement time was 1.6 min. Then the spectra were reconstructed using TwIST (see Section 3.B and [22]), first with no prior information (Fig. 5(A), brown plot) and then with sparsity priors (Fig. 5(B), yellow plot). Compared with full Nyquist sampled results, there is an increase in noise in the under-sampled reconstruction in Fig. 5(A). This observation is in line with the theoretical simulations presented by Thurman and Fienup [33]. Figure 5(B) shows de-noised recovery of the same spectrum using TwIST along with sparsity priors. Spectral images for key spectral bands are shown in Fig. 5(E). However, it should be noted that this may lead to a loss of weak Raman bands. The sparsity prior’s contribution toward the objective function can be adjusted by the weight, $\gamma $, in Eq. (18). Loss of weaker signal can be controlled by selecting a lower weight. One other way to improve the denoising is by introducing priors on the spatial dimensions, such as smoothness and shape priors, of the spectral cube.
D. Compressed Fluorescence Spectral Feature Imaging
For some fluorescence-based applications, measuring the entire fluorescence spectrum is not necessary and knowing only some features of the spectrum is sufficient. Therefore, as a proof of concept, in this section we concentrate on measuring the peak emission wavelength of a fluorescence spectrum. In most of the labeled multi-fluorescence-based applications, knowing the peak emission wavelength provides sufficient information to identify the species present in the specimen at a particular spatial location. In some applications, the fluorescence spectrum shifts with experimental conditions, such as pH [34], and monitoring the peak emission wavelength is a very good indicator of the amount of spectral shift. Therefore, being able to measure the peak emission wavelength with a fewer number of measurement is of interest in hyperspectral microscopy.
Thus, we demonstrate compressive measurements of peak wavelengths, for fluorescent beads and mouse muscle specimen, using an optimized sampling strategy and greedy single-wavenumber search as described in Section 3.C. The optimum OPD positions were selected by the following method: emission light from green (Thorlabs, M530L3) and red (Thorlabs, M625L3) LEDs were measured throughout the entire FOV with a Nyquist sampling criterion (imaging objective: Nikon, $40\times $, 0.75 NA, CFI Plan Fluor DLL). The spectral cubes were recovered using the FT-based algorithm (Section 3.A) and ground truth peak wavelengths (say ${\lambda}_{\mathrm{GT}}$) were extracted by looking at the recovered spectra. For a specified compression ratio $n/N$ ($n=\text{number of compressive measurements}$, $N=\text{number of Nyquist measurements}$), $n$ measurements were drawn at random OPD positions. These were then used to recover the peak wavelength (say ${\lambda}_{n/N}$) using a greedy single-wavenumber search. The measurement error (defined below) was then calculated as
This was repeated a sufficiently large number of times and the OPD position set, which resulted in the minimum measurement error for both green and red LEDs, was chosen as the optimum. Then interference images for the test specimens were taken (imaging objective: Nikon, $40\times $, 0.75 NA, CFI Plan Fluor DLL; imaging filters: Semrock, Di02-R488, BLP01-488R-25) at optimum OPD positions and the peak wavelengths were recovered using a greedy single-wavenumber search. Figures 6(A1)–6(A6) show the peak wavelengths for, respectively, no compression, i.e., compression $\mathrm{ratio}=n/N=1$ (A1 and A4), compression $\mathrm{ratio}=0.2$ (A2 and A5), and compression $\mathrm{ratio}=0.05$ (A3 and A6) for the fluorescent bead specimen. Figures 6(B1)–6(B6) show similar results for the mouse muscle specimen. For beads, acceptable results were acquired up to the lowest compression ratio of 0.05, whereas for the mouse muscle specimen, the scheme broke down after 0.2 (compare Figs. 6(B5) and 6(B6)). In Figure 6(C), we quantitatively compare our optimized sampling strategy with conventional random sampling used frequently in compressive sensing. The peak wavelength recovered from non-compressed measurements was treated as the ground truth to calculate the measurement error defined in Eq. (24). Agreeing with the qualitative results, for the mouse muscle specimen, the average measurement error of optimized sampling reached the best case of random sampling with $n/N=0.2$. For that compression ratio, the measurement error for the average case in random sampling remained relatively high and was unacceptably high for the worst case. The average and worst case of random sampling seemed to converge after $n/N=0.5$. This suggests that the proposed strategic sampling scheme could recover near-best results for compression ratios above a certain threshold (0.2 for mouse muscle specimen).
5. CONCLUSION
In this paper, we presented a new near-common-path interferometer design for wide-field imaging Fourier-transform spectroscopy with phase stability similar to industry standard Sagnac imaging spectrometers. In Sagnac, there are serious vignetting effects, which limits the FOV [16,17]. In contrast, in our design, the maximum FOV allowed by the microscope optics can be utilized for spectral imaging. Furthermore, in Sagnac interferometers, at a specific OPD scan location, the OPD seen by each pixel varies linearly along one spatial dimension of the FOV. However, in the proposed design, the OPD is uniform throughout the FOV and, hence, OPD scanning is completely decoupled from the spatial dimensions of the data cube. For the same FOV, having uniform OPD speeds up the proposed setup by almost an order of magnitude compared with the Sagnac design for applications such as fluorescence imaging spectroscopy, which require moderate spectral resolution ($\sim 10\text{\hspace{0.17em}}\mathrm{nm}$). Moreover, uniform OPD variation allows random-addressed OPD scanning for all the pixels in the FOV simultaneously. We have demonstrated that this can be utilized in practical applications to measure properties of fluorescence spectra without having to measure the entire spectral range. One other major limitation of the Sagnac design is that the spectral resolution is limited for large FOVs as argued in [13]. Therefore, Sagnac cannot maintain a large FOV for high-spectral-resolution applications such as Raman imaging. In contrast, our system has no restrictions on the size of the FOV with respect to increased spectral resolution. To validate this point, we have demonstrated full-field Raman imaging using the proposed design. We also experimentally demonstrated that under-sampled interferogram acquisition can speed up Raman imaging by an order of magnitude at an expense of SNR as proven theoretically in [33]. However, we further demonstrated that low SNR Raman spectra can be de-noised using sparsity priors.
In conclusion, with our system, one could expect to significantly speed up the IFTS spectral measurements by seamless integration of various compressive sampling schemes without compromising phase stability.
Funding
National Institutes of Health (NIH) (1R01HL121386-01A1, 1U01CA202177-01, 4R44EB012415, 5R01NS051320, 9P41EB015871-26A1); Singapore-MIT Alliance for Research and Technology Centre (SMART) (BioSym IRG); Hamamatsu Corp.; Samsung Advanced Institute of Technology.
See Supplement 1 for supporting content.
REFERENCES
1. C. Buehler, K. H. Kim, U. Greuter, N. Schlumpf, and P. T. So, “Single-photon counting multicolor multiphoton fluorescence microscope,” J. Fluoresc. 15, 41–51 (2005). [CrossRef]
2. E. Schröck, S. Du Manoir, T. Veldman, B. Schoell, J. Wienberg, M. Ferguson-Smith, Y. Ning, D. Ledbetter, I. Bar-Am, D. Soenksen, Y. Garini, and T. Ried, “Multicolor spectral karyotyping of human chromosomes,” Science 273, 494–497 (1996). [CrossRef]
3. C. Thaler, S. V. Koushik, P. S. Blank, and S. S. Vogel, “Quantitative multiphoton spectral imaging and its use for measuring resonance energy transfer,” Biophys. J. 89, 2736–2749 (2005). [CrossRef]
4. C. J. Brenan and I. W. Hunter, “Design and characterization of a visible-light Fourier transform Raman spectrometer,” Appl. Spectrosc. 49, 1086–1093 (1995). [CrossRef]
5. C. Bakal, J. Aach, G. Church, and N. Perrimon, “Quantitative morphological signatures define local signaling networks regulating cell morphology,” Science 316, 1753–1756 (2007). [CrossRef]
6. Z. E. Perlman, M. D. Slack, Y. Feng, T. J. Mitchison, L. F. Wu, and S. J. Altschuler, “Multidimensional drug profiling by automated microscopy,” Science 306, 1194–1198 (2004). [CrossRef]
7. Q. Li, X. He, Y. Wang, H. Liu, D. Xu, and F. Guo, “Review of spectral imaging technology in biomedical engineering: achievements and challenges,” J. Biomed. Opt. 18, 100901 (2013). [CrossRef]
8. E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Probl. 23, 969–985 (2007). [CrossRef]
9. R. G. Baraniuk, “Compressive sensing,” IEEE Signal Process. Mag. 24, 118–121 (2007). [CrossRef]
10. J. Maillard, L. Drissen, F. Grandmont, and S. Thibault, “Integral wide-field spectroscopy in astronomy: the imaging FTS solution,” Exp. Astron. 35, 527–559 (2013). [CrossRef]
11. M. R. Carter, C. L. Bennett, D. J. Fields, and F. D. Lee, “Livermore imaging Fourier transform infrared spectrometer (LIFTIRS),” Proc. SPIE 2480, 380–386 (1995).
12. Z. Malik, D. Cabib, R. Buckwald, A. Talmi, Y. Garini, and S. Lipson, “Fourier transform multipixel spectroscopy for quantitative cytology,” J. Microsc. 182, 133–140 (1996). [CrossRef]
13. W. Müller, M. Kielhorn, M. Schmitt, J. Popp, and R. Heintzmann, “Light sheet Raman micro-spectroscopy,” Optica 3, 452–457 (2016). [CrossRef]
14. Y. Chen and I. W. Hunter, “Design of a miniature hyperspectral imaging Fourier transform spectrometer for endoscopy,” in Imaging Systems and Applications (Optical Society of America, 2016), paper IW1E.2.
15. P. R. Griffiths and J. A. De Haseth, Fourier Transform Infrared Spectrometry (Wiley, 2007), Vol. 171.
16. R. G. Sellar and G. D. Boreman, “Limiting aspect ratios of Sagnac interferometers,” Opt. Eng. 42, 3320–3325 (2003). [CrossRef]
17. Y. Ferrec, J. Taboury, H. Sauer, and P. Chavel, “Optimal geometry for Sagnac and Michelson interferometers used as spectral imagers,” Opt. Eng. 45, 115601 (2006). [CrossRef]
18. G. Popescu, T. Ikeda, R. R. Dasari, and M. S. Feld, “Diffraction phase microscopy for quantifying cell structure and dynamics,” Opt. Lett. 31, 775–777 (2006). [CrossRef]
19. T. R. Hillman, N. Lue, Y. Sung, R. R. Dasari, and Z. Yaqoob, “Near-common-path self-reference quantitative phase microscopy,” IEEE Photon. Technol. Lett. 24, 1812–1814 (2012). [CrossRef]
20. R. Kruger, L. Anderson, and F. Roesler, “All-reflection interferometer for use as a Fourier-transform spectrometer,” J. Opt. Soc. Am. 62, 938–945 (1972). [CrossRef]
21. R. Kruger, L. Anderson, and F. Roesler, “New Fourier transform all-reflection interferometer,” Appl. Opt. 12, 533–540 (1973). [CrossRef]
22. J. M. Bioucas-Dias and M. A. Figueiredo, “A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. Image Process. 16, 2992–3004 (2007). [CrossRef]
23. D. Needell and J. A. Tropp, “CoSaMP: iterative signal recovery from incomplete and inaccurate samples,” Appl. Comput. Harmon. Anal. 26, 301–321 (2009). [CrossRef]
24. W. Dai and O. Milenkovic, “Subspace pursuit for compressive sensing signal reconstruction,” IEEE Trans. Inf. Theory 55, 2230–2249 (2009). [CrossRef]
25. R. Chartrand and W. Yin, “Iteratively reweighted algorithms for compressive sensing,” in 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2008), pp. 3869–3872.
26. T. Blumensath, “Compressed sensing with nonlinear observations and related nonlinear optimization problems,” IEEE Trans. Inf. Theory 59, 3466–3474 (2013). [CrossRef]
27. A. Beck and Y. C. Eldar, “Sparsity constrained nonlinear optimization: optimality conditions and algorithms,” SIAM J. Optim. 23, 1480–1509 (2013). [CrossRef]
28. A. Barducci, D. Guzzi, C. Lastri, P. Marcoionni, V. Nardino, and I. Pippi, “Theoretical aspects of Fourier transform spectrometry and common path triangular interferometers,” Opt. Express 18, 11622–11649 (2010). [CrossRef]
29. M. S. Islam, M. Honma, T. Nakabayashi, M. Kinjo, and N. Ohta, “pH dependence of the fluorescence lifetime of FAD in solution and in cells,” Int. J. Mol. Sci. 14, 1952–1963 (2013). [CrossRef]
30. C. A. Lieber and A. Mahadevan-Jansen, “Automated method for subtraction of fluorescence from biological Raman spectra,” Appl. Spectrosc. 57, 1363–1367 (2003). [CrossRef]
31. P. Caspers, G. Lucassen, R. Wolthuis, H. Bruining, and G. Puppels, “In vitro and in vivo Raman spectroscopy of human skin,” Biospectroscopy 4, S31–S40 (1998). [CrossRef]
32. M. G. Shim, L.-M. Wong Kee Song, N. E. Marcon, and B. C. Wilson, “In vivo near-infrared Raman spectroscopy: demonstration of feasibility during clinical gastrointestinal endoscopy,” Photochem. Photobiol. 72, 146–150 (2000). [CrossRef]
33. S. T. Thurman and J. R. Fienup, “Signal-to-noise ratio trade-offs associated with coarsely sampled Fourier transform spectroscopy,” J. Opt. Soc. Am. A 24, 2817–2821 (2007). [CrossRef]
34. K. D. Piatkevich, V. N. Malashkevich, K. S. Morozova, N. A. Nemkovich, S. C. Almo, and V. V. Verkhusha, “Extended stokes shift in fluorescent proteins: chromophore–protein interactions in a near-infrared tagrfp675 variant,” Sci. Rep. 3, 1847 (2013). [CrossRef]