1. INTRODUCTION

Living cells dynamically organize distinct intracellular organelles to enable the spatiotemporal regulation of biological processes. Visualizing these diverse anatomical and functional complexities within the densely packed cellular space unfolds critical details of the fundamentals of living organisms. In the past decades, optical microscopy technologies have revolutionized cell biology by enabling systematic visualization and analysis of single cells across nano-, micro- and macroscales with a high spatiotemporal resolution, molecular specificity, and sensitivity [1–6]. However, despite major advances, imaging technology innovation has been persistently demanded to cope with the emerging breadth of biological insights. One underlying challenge lies in locating and dissecting complex intracellular multicomponent entities that span high spatiotemporal dimensions and scales.

Conventionally, major fluorescence microscopy techniques produce orthographic views and acquire 3D information in a sequential or scanning fashion [7]. This process inevitably compromises the temporal resolution and increases photo damage for live-cell imaging. In contrast, the emerging light-field microscopy (LFM) techniques can simultaneously record both the 2D spatial and 2D angular information of light, allowing computational synthesis of the volume of a specimen from a single camera frame [8–11]. This 4D imaging scheme offers fast and volumetric acquisition (limited primarily by the camera speed), minimum photo damage for time-lapse observation, and a high scalability and design flexibility. These capabilities allow for functional imaging of the mammalian brain with a cellular-level, milliseconds spatiotemporal resolution across a significant thickness ranging from tens to hundreds of micrometers in various model systems [11–16]. Scaling down to the subcellular level, recent efforts have exploited LFM for imaging intracellular organelles and dynamics in single cells [17], as well as multicellular specimens by scanning [18]. However, the applicability of these developments has been largely limited by the intrinsic uneven sampling of the optical signals in the conventional LFM design. The deficiency causes reconstruction artifacts and aggravated computational cost, which becomes particularly restrictive in high-resolution imaging of delicate subcellular information [17].

Addressing the challenge, the recent development of Fourier light-field microscopy (FLFM) has significantly enhanced the image quality and computational efficiency [19–22]. Specifically, the strategy captures the 4D light field in the Fourier domain, permitting spatially invariant sampling and parallel image formation and retrieval. The advancement has triggered rapidly growing interests [23–26] and versatile configurations such as confocal microscopy [27], endoscopy [28], label-free microscopy [29], single-molecule imaging [30], light-sheet microscopy [31], and miniature microscopy [32,33]. However, current demonstrations have been primarily focused on either large-scale tissues or fixed samples. Such a Fourier-domain light-field methodology, despite the great demand, remains unexplored in the realm of high-resolution live-cell imaging.

Here, we introduce high-resolution FLFM (HR-FLFM), an approach to fast, volumetric and multicolor live-cell imaging. In HR-FLFM, we have devised optical, algorithmic, and instrumental strategies to transform conventional cell imaging and enable uncompromised visualization of subcellular structural and dynamic information. The results present a near-diffraction-limited 3D spatial resolution, an imaging depth extended by five-fold, and a volumetric temporal resolution up to milliseconds. We anticipate HR-FLFM to provide a promising avenue to explore many spatiotemporal-limiting cellular responses in a variety of single-cell studies.

2. SYSTEM DESIGN

A. Experimental Setup

As illustrated in Fig. 1(a), we constructed the HR-FLFM system using an epi-fluorescence microscope (Nikon Eclipse Ti2-U) implemented with a ${100} \times$, 1.45 NA objective lens (Nikon CFI Plan Apochromat Lambda ${100}\; \times$ Oil) and multicolor laser lines (647 and 488 nm, MPB). The objective lens was controlled by a piezo nano-positioner (MCL Nano-F100S), and the sample stage by a micropositioning system (ASI MS2000). The fluorescence emission was collected using a quadband dichroic mirror (ZT405/488/561/647, Chroma) and a corresponding emission filter (ZET405/488/561/647m, Chroma). The native image plane (NIP) of the objective lens was Fourier transformed using a Fourier lens (${f_{{\rm FL}}} = {275}\;{\rm mm}$, Thorlabs). The back focal plane of the Fourier lens was partitioned by a customized microlens array (MLA, RPC Photonics, specified in Section 2.B), forming elemental images by each individual microlens on an sCMOS camera (Hamamatsu ORCA-Flash4.0, pixel size ${P_{{\rm cam}}} = {6.5}\;\unicode{x00B5}{\rm m}$).

 

Fig. 1. High-resolution Fourier light-field microscopy (HR-FLFM). (a) Experimental setup of HR-FLFM. The objective lens (OL) and the tube lens (TL) form wide-field images at the native image plane (NIP). The Fourier lens (FL) transforms the NIP to its back focal plane, where the microlens array (MLA) is positioned. The MLA segments the light field and forms three elemental images at its back focal plane on the sCMOS camera. DC, dichroic cube. CAM, camera. The inset diagram illustrates image formation through the customized MLA for emitters at different axial positions, capturing both the spatial and angular information in an uncompromised manner. (b) Axial stack projection (step ${\rm size} = {100}\;{\rm nm}$) of the experimental point-spread function (PSF) through the microlenses [(i)-(iii)] (effective ${\rm pitch} = {72.5}\;{\unicode{x00B5}{\rm m}}$ in the object space) within an axial range from ${-}{5 {-} 5}\;{\unicode{x00B5}{\rm m}}$, as color-coded in the color scale bar. (c) Lateral displacement of each elemental PSF image [(i)-(iii)] at the camera plane as a function of the axial position, showing a good agreement with the theoretical prediction (dashed line). Scale bar: 10 µm.

Download Full Size | PDF

B. MLA Design

As an aperture-partitioning system, the MLA that segments the Fourier plane plays an essential role in determination of the imaging capability of FLFM such as the 3D resolution, the field of view (FOV), and the depth of focus (DOF). In theory, to facilitate a high-resolution system, the MLA is expected to exhibit a minimal segmentation of the full Fourier aperture. This allows maintaining a sufficient photon budget (i.e., a high SNR) crucial for the reconstruction quality, maximizing the FOV within each elemental image, and utilizing all the high spatial frequencies throughout the entire pupil for an optimum axial (or angular) sensitivity and 3D resolution. Given this consideration, here, we designed a new MLA to achieve the optimum subcellular imaging performance based on our theoretical model of FLFM [22]. Specifically, we exploited a customized hexagonal MLA (pitch $d = {3.25}\;{\rm mm},\;f \text{-} {\rm number} = {36},\;{f_{\rm ML\:}} = {117}\;{\rm mm}$), which segments the full pupil into a minimum of three off-axis elements [Fig. 1(a) and Supplement 1]. Notably, to reduce segmentation, this design circumvents the use of the on-axis microlens normally adopted by FLFM implementations. We reason that the on-axis element contains mainly the DC component of the light field with a low angular sensitivity, thus contributing less significantly to the overall volumetric imaging capability [34].

C. Light-Field Propagation and Image Formation

The image formation of the Fourier scheme, as shown in Fig. 1(a), allows the spatial components contained in the full aperture to be recorded in an uncompromised and well-aliased manner. As a result, the overall light-field propagation can be described by a unified 3D point-spread function (PSF) [Fig. 1(b)], and thereby, the elemental images are formed as a convolution between the object and the 3D PSF.

To model the image formation, we derived a wave-optics framework that describes the light-field propagation through the entire HR-FLFM system (Supplement 1). In brief, first, the framework obtains the 2D native image at the NIP originated from a 3D object in the epifluorescence microscope using the Debye theory [35]. Here, unlike the existing scalar models, we exploited the vectorial wave propagation in the Debye theory to address the high-NA system, as well as the refractive index mismatch between the objective immersion medium and sample solution [36]. This consideration provides a rigorous native image formation using the large-aperture, oil-immersion objective, critical for precise modeling of the subsequent high-resolution light-field propagation. Next, the native image is optically Fourier transformed by the Fourier lens and partitioned and modulated by the microlenses at the MLA plane. Notably, placing the MLA at the Fourier plane allows maintenance of a consistent magnification of the elemental images on the camera plane from objects at varying depths. Finally, the light field from the MLA is propagated to its back focal plane, forming the elemental images on the camera using the Fresnel light propagation [22,37].

 

Fig. 2. Characterization of HR-FLFM using fluorescent beads. (a–b) Raw light-field (a) and 3D reconstructed HR-FLFM (b) images of 200-nm fluorescent beads distributed in 3D agarose gel. The inset images (i)-(iii) in (a) show distinct light-field information captured from the same boxed regions of the three elemental images. (c) Wide-field image of the same volume as in (b), acquired by overlaying 41 axial stacks at a step size of 100 nm, showing consistent sample information at each depth. The inset displays the zoomed-in image of a single wide-field stack at $z = {0}\;{\unicode{x00B5}{\rm m}}$ of the boxed region in (c), showing a limited DOF compared with HR-FLFM. (d)–(e) Zoomed-in images of the corresponding boxed regions in (b) and (c), respectively, and their projected $x - z$ and $y - z$ views. The results show consistent structural information and more compact axial profiles in the HR-FLFM images. (f) FWHM values of the 3D cross-sectional profiles at varying depths over a ${\gt}{3}\;{\unicode{x00B5}{\rm m}}$ axial range, exhibiting 0.3–0.7 µm and 0.5–1.5 µm in the lateral and axial dimensions, respectively. The scatters and lines represent the FWHM measurements of each bead and local-weighted linear regressions (Supplement 1). The corresponding shades are set to enclose 90% of the data points. The depth information in (b)–(c) is coded as in the corresponding color-scale bars. Scale bars: (a)–(c) 10 µm and (a) inset, (d), and (e) 1 µm.

Download Full Size | PDF

To examine the theoretical model, we experimentally acquired and characterized the PSF of HR-FLFM by recording 200-nm dark-red fluorescent beads (T7280, ThermoFisher) at different axial positions [Fig. 1(b)]. As observed, the results present no parallax of elemental images in the lateral translation of the emitters. Meanwhile, the measurements showed a radial displacement of each elemental image by ${\sim}{200}\;{\unicode{x00B5}{\rm m}}$ in the lateral dimension at the camera plane, due to the depth variations over an axial range of 10 µm [Fig. 1(c)]. The displacements exhibited a consistent linear dependence in each elemental image with respect to the axial positions, especially observable within a 4–5 µm range near the focal plane, which implies a steady axial or angular sensitivity of the system across a ${\sim}{5} \times$ enhanced DOF compared with corresponding wide-field microscopy, which DOF is typically 800 nm–1 µm. As measured, the experimental data of the radial displacement showed an overall slope at 21.82, in a good agreement with a slope of 22.52 predicted using the theoretical model [Fig. 1(c)].

D. Reconstruction and Hybrid PSF

Given the image formation, the reconstruction in HR-FLFM can be considered as an inverse process to retrieve the volume of the object through a wave-optics based Richardson–Lucy deconvolution of the elemental images and the 3D PSF [22,38,39] (Supplement 1). Here, our algorithm is able to complete a volumetric iteration within 0.18 s, which achieves reconstruction of the full volume in 4–9 s using 20 to 50 iterations (Supplement 1). The reconstruction process relies on iterative projections between the 3D object space and the 2D camera plane through convolutions with the 3D PSF. Therefore, a careful determination of the PSF plays an essential role in the reconstruction precision and final image quality.

In practice, both experimental and numerical PSFs have been previously employed for reconstruction [21,22]. However, the former strategy suffers from fluorescence fluctuations and a low SNR away from the focal plane that may cause reconstruction artifacts, while the latter strategy lacks the consideration of any experimental misalignment or aberrations that may affect the reconstruction precision. Both drawbacks may degrade the performance of the system and become especially detrimental when using a high-NA, oil-immersion objective lens. To solve the problem, here, we derived a hybrid PSF for the reconstruction, which intensity profiles were presented by the numerical PSF, while their spatial locations at each axial position were determined by the experimental results (Supplement 1). Here, both the experimental and numerical PSFs have been generated considering the same condition (e.g., the coverslip, oil immersion). This allows the reconstruction process to account for any deviations due to the optical system while maintaining a high SNR to avoid computational artifacts across the entire imaging depth.

3. RESULTS

A. System Characterization

To characterize the performance of HR-FLFM, we first imaged 200-nm dark-red fluorescent beads (T7280, ThermoFisher) distributed in agarose gel using the 647-nm laser. As seen, the three elemental images in the light-field acquisition captured distinct perspective views of the incident spatial and angular information [Fig. 2(a)]. This allows to reconstruct the sample using a single camera frame, recovering a volume of ${\sim}{70}\;{\unicode{x00B5}{\rm m}} \times {70}\;{\unicode{x00B5}{\rm m}} \times {4}\;{\unicode{x00B5}{\rm m}}$, consistent with the predicted FOV and DOF using the theoretical model [Fig. 2(b) and Supplement 1]. The focal images of the volume were synthesized by HR-FLFM in a good agreement with the axial stacks taken by scanning wide-field microscopy, exhibiting a ${5} \times$ extended DOF [Figs. 2(c)–2(e)]. Notably, HR-FLFM presents a high angular sensitivity with the MLA design, thereby showing an enhanced resolving power of the phantom profiles in the axial dimension [Figs. 2(d) and 2(e)]. As measured, the full width at half-maximum (FWHM) values of these reconstructed images fitted with a 3D Gaussian function exhibited 300–700 nm and 0.5–1.5 µm in the lateral and axial dimensions, respectively, across an axial range close to 4 µm, consistent with the theoretical prediction [Fig. 2(f) and Supplement 1].

 

Fig. 3. Imaging surface-stained fluorescent microsphere. (a)–(b) 3D reconstructed image (a) and the corresponding focal stacks in $x - y$ and cross-sectional image in $x - z$ (b) of a surface-stained, 4-µm fluorescent microsphere, showing the hollow structure. (c) Cross-sectional profiles across the center of the microsphere, exhibiting the FWHM values of the profiles (left and right) and their distances at 452 nm, 411 nm and 4.54 µm in $x$, 422 nm, 452 nm and 4.56 µm in $y$, and 1.50, 1.02, and 3.99 µm in $z$, respectively. Scale bar: 1 µm.

Download Full Size | PDF

 

Fig. 4. Imaging mitochondria in fixed mammalian cells using HR-FLFM. (a),(b) Raw light-field (a) and 3D reconstructed HR-FLFM (b) images of immune-stained mitochondria in COS-7 cells. (c) Wide-field image of the same cells as in (b), showing a limited DOF compared with HR-FLFM. (d) Zoomed-in image of the corresponding boxed region in (b). The top and bottom insets display the zoomed-in images of the red and blue boxed regions in (d), respectively, showing well-resolved submicrometer hollow structures. (e),and (f) Cross-sectional profiles along the (e) orange and (f) blue dashed lines in (d), resolving lateral structures as close as 400–500 nm. (g) Zoomed-in wide-field image (top left) of the corresponding boxed region in (c) and the focal stacks of the same region in (d) reconstructed by HR-FLFM. The arrows point to the regions captured by HR-FLFM while less observable in wide-field microscopy due to the limited DOF. (h) Zoomed-in image of the corresponding boxed region in (b). The top and bottom insets, respectively, display the $y - z$ and $x - z$ views of the boxed region in (h), showing well-resolved void-like mitochondrial structures. (i) Cross-sectional profile along the dashed line in (h), resolving axial structures separated by 577 nm. (j) Zoomed-in wide-field image (top left) of the corresponding boxed region in (c) and the focal stacks of the same region in (h) reconstructed by HR-FLFM. The arrows point to the regions captured entirely by HR-FLFM while only partially observable in wide-field microscopy due to the limited DOF. The depth information in (b), (d), and (h) is coded as in the color-scale bar in (b). Scale bars: (a)–(c) 10 µm; (d), (g), (h), and (j) 1 µm; (d) and (h) insets 500 nm.

Download Full Size | PDF

 

Fig. 5. Imaging peroxisomes and mitochondria in living cells using HR-FLFM. (a) 3D reconstructed HR-FLFM image of GFP-stained peroxisomes in a living COS-7 cell taken at a volume acquisition time of 5 ms at $t = {0}\;{\rm s}$. (b) Cross-sectional profiles of the peroxisome marked by the red arrow in (a), exhibiting FWHM values of 286, 261, and 350 nm in $x,y,z$, respectively. (c) 3D tracking of the peroxisome marked by the yellow arrow in (a) at the nanometer scale over the time course. The time points are color-coded according to the color scale bar. (d) Zoomed-in images of the blue boxed region in (a) at the time points $t = {0}$, 3.0, 6.0, 9.0, 12.0 and 15.0 s. The arrows indicate multiple moving peroxisomes over the time course. (e) Zoomed-in images of the red boxed region in (a) at the time points $t = {12.755}$, 12.760, 12.765, 12.770, 12.775 and 12.780 s. (f) Cross-sectional profiles in the lateral (left) and axial (right) dimensions with respect to the corresponding dashed lines in (e), exhibiting structural variations of nearby peroxisomes as close as ${\sim}{500}\;{\rm nm}$ at every 5 ms in all three dimensions. (g) Two-color stack projection of 3D reconstructed HR-FLFM images of GFP-stained peroxisomes (green) and MitoTracker-stained mitochondria (magenta) in living COS-7 cells taken at a volume acquisition time of 20 ms at $t = {0}\;{\rm s}$. (h) Zoomed-in $x - z$ projection of the boxed region in (g) at the time points $t = {0}$, 6.0, 12.0, 18.0, 24.0, and 30.0 s, showing dynamic interactions between the organelles over the time course. The depth information in (a), (d), and (e) is coded as in the color-scale bar in (a). Scale bars: (a),(g) 5 µm; (d),(h) 1 µm; and (e) 500 nm.

Download Full Size | PDF

Next, we imaged a surface-stained, 4-µm fluorescent microsphere (T7283, ThermoFisher) and reconstructed the 3D hollow structure using HR-FLFM [Fig. 3(a) and Supplement 1]. The focal stacks across the entire thickness (${\sim}{4}\;{\unicode{x00B5}{\rm m}}$) can be synthesized [Figs. 3(a) and 3(b)]. It is mentioned that the optical design of HR-FLFM gains the sectioning capability to effectively mitigate the cross-talk between the axial slices due to the overlapping 3D information [22]. The lateral and axial cross-sectional profiles exhibited the respective FWHM values of the stained surface at 400–500 nm and 1–1.5 µm [Fig. 3(c)], consistent with the experimental measurements at the corresponding depths in Fig. 2(f).

B. Imaging Immuno-Stained Mitochondria in Mammalian Cells

We next demonstrated HR-FLFM imaging of biological samples. We first imaged immuno-stained mitochondria in COS-7 cells using the 647-nm laser (Fig. 4). As seen, the system recorded the incident light field in three elemental images, allowing to reconstruct two dividing cells using a single camera frame at a volume acquisition time of 0.1 s [Figs. 4(a) and 4(b)]. In particular, HR-FLFM captured those mitochondria that were out-of-focus and poorly observable using wide-field microscopy due to its limited DOF [Figs. 4(b) and 4(c)]. The fine structural variations of mitochondria can be clearly detectable throughout the synthesized focal stacks over an axial range ${\gt}{4}\;{\unicode{x00B5}{\rm m}}$, consistent with the axial stacks taken by scanning wide-field microscopy [Figs. 4(d)–4(j)]. Notably, the densely packed organelles were reconstructed without noticeable artifacts or cross-talks between axial slices due to the Fourier configuration, exhibiting an improved volumetric imaging capability compared to our previously reported LFM using a conventional light-field scheme [17]. Furthermore, utilizing the hybrid PSF, the result has shown a substantially enhanced image quality, in comparison with reconstructed images using either the experimental or numerical PSFs (Supplement 1). The cross-sectional profiles exhibited that various mitochondrial void-like filaments separated by 400–500 nm were clearly resolved in all three dimensions in the HR-FLFM images [Figs. 4(e), 4(f), and 4(i)], presenting a near-diffraction-limited 3D resolution over a ${5} \times$ extended DOF, consistent with the measurements using the caliber samples in Figs. 2 and 3.

C. Imaging Peroxisomes and Mitochondria in Living Cells

Finally, we performed live-cell imaging of intracellular organelles using HR-FLFM (Fig. 5). Using the 488-nm laser, we first imaged GFP-labelled peroxisomes in living COS-7 cells. Peroxisomes are single membrane organelles responsible for a number of metabolic processes that underlie cellular homeostasis [40]. These compartments are small and remarkably fluid in the cytoplasm in response to versatile environmental cues, thus posing a challenge for capturing their spatiotemporal processes in a significant volume of cells using conventional imaging methods.

Without the need for scanning, HR-FLFM permits a low light exposure (${\lt}{0.25}\;{{\rm J}\cdot {\rm cm}^{- 2}}$ per volume) for time-lapse observation of peroxisomes at a volume acquisition time of 5 ms over thousands of time points without noticeable photodamage (Visualization 1) [41]. The reconstructed images of individual peroxisomes exhibited FWHM values at 200–400 nm in both lateral and axial dimensions, consistent with the known 0.1–1 µm diameter and typical spherical shape of the organelle [42] [Figs. 5(a) and 5(b)]. In the COS-7 cells, peroxisomes are prominently observable within an axial range of 1.5 µm, and their motions can be continuously tracked in three dimensions by localizing the reconstructed images at a nanometer-level precision using Gaussian fitting [43] [Fig. 5(c) and Visualization 2]. The high spatiotemporal resolution of HR-FLFM allows us to visualize the interacting peroxisomes separated below 500 nm in all three dimensions and migrating at a speed of ${\sim}{50}\; {\unicode{x00B5}{\rm m}} \cdot {\rm s}^{- 1}$ [Figs. 5(d)–5(f)]. Furthermore, we introduced an sCMOS denoising algorithm in the reconstruction pipeline, which can enhance the elemental images to facilitate live-cell imaging at a lower photon flux (thus a lower SNR) [44]. This algorithm effectively improved the reconstruction results to allow for a fast acquisition rate or a low illumination (thus reduced photo damage) for continuous observation (Supplement 1).

Last, we performed two-color HR-FLFM imaging of GFP-labelled peroxisomes and MitoTracker-labeled mitochondria in living COS-7 cells. Emerging studies on peroxisomes and mitochondria have recently identified the two organelles to function together to regulate a variety of essential cellular metabolism and signaling pathways, which dysfunction is critically relevant to many disease pathologies [45]. HR-FLFM acquired the optical signals of peroxisomes and mitochondria sequentially with the 488- and 647-nm lasers at a volume acquisition time of 20 ms over minutes (Supplement 1). The reconstructed two-color images revealed the structural relationship and the rapid interactions of peroxisomes with mitochondrial spheroids [Fig. 5(g)]. As seen, the high resolution and volumetric imaging capability allow us to visualize individual fast-moving peroxisomes as close as a few hundred nanometers in all three dimensions. Consistent with single-color imaging, the peroxisomes were located primarily in a thickness of 1–1.5 µm and engulfed within dynamic mitochondrial voids distributed throughout an axial range of 3–4 µm [Fig. 5(h) and Visualization 3]. These results demonstrated the capability of HR-FLFM for recording dynamic organelles in a volumetric context, showing no compromise in the spatial and temporal acquisition, which has been an essentially limiting step for live-cell imaging.

4. CONCLUSIONS

In summary, we have developed HR-FLFM for fast, volumetric multicolor live-cell imaging. The light-field methodology transforms conventional cell imaging relying on scanning-based or epi-fluorescence microscopy. The method allows probing the spatiotemporal contextual information in cells with a near-diffraction-limited 3D resolution, a five-fold extended DOF in several micrometers, and a volume acquisition time up to milliseconds. The system demonstrates instrumentation simplicity, low photo damage for time-lapse observation, and a high compatibility with various cell assays and imaging conditions. We anticipate HR-FLFM to offer a promising paradigm for interrogation of complex intracellular biomolecules, organelles, and microenvironments that underlie diverse spatiotemporal regulations of cellular processes and functions.