Laser-scanning confocal fluorescence microscopy is an indispensable tool for biomedical research by virtue of its high spatial resolution. Its temporal resolution is equally important, but is still inadequate for many applications. Here we present a confocal fluorescence microscope that, for the first time to our knowledge, surpasses the highest possible frame rate constrained only by the fluorescence lifetime of fluorophores (typically a few to several nanoseconds). This microscope is enabled by integrating a broadband, spatially distributed, dual-frequency comb or spatial dual-comb and quadrature amplitude modulation for optimizing spectral efficiency into frequency-division multiplexing with single-pixel photodetection for signal integration. Specifically, we demonstrate confocal fluorescence microscopy at a record high frame rate of 16,000 frames/s. To show its broad biomedical utility, we use the microscope to demonstrate 3D volumetric confocal fluorescence microscopy of cellular dynamics at 104 volumes/s and confocal fluorescence imaging flow cytometry of hematological and microalgal cells at up to 2 m/s.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Laser-scanning confocal fluorescence microscopy is a powerful method for biological and medical research by virtue of its high spatial resolution in 3D . Also, temporal resolution (i.e., frame rate, volume rate) is equally critical for many applications that require real-time observation of fast transient events (e.g., biochemical behaviors in living cells ), neural activity , and high-throughput characterization of tissues and cells (e.g., wide-field endomicroscopy [4,5], high-throughput imaging flow cytometry [6–8]). In such applications, low frame rates result in motion blur and failure to capture fast transient information. The ultimate speed limit of laser-scanning confocal fluorescence microscopy is given by the fluorescence lifetime of fluorophores (typically a few to several nanoseconds at each dwell pixel; see Supplement 1, Fig. S1) [1,9]. In fact, as the scanning speed is significantly increased, the pixel dwell time becomes shorter than the fluorescence lifetime, such that the obtained fluorescence image suffers from a significant decrease of signal-to-noise ratio (SNR) because the majority of fluorescence photons are emitted after the pixel dwell time and thus do not contribute to the image contrast. For example, if the fluorescence lifetime of fluorophores is 4 ns for 2D confocal fluorescence imaging with dwell pixels, the highest possible frame rate is given by (which has yet to be realized).
Over the last decade, various efforts have been made toward the development of laser-scanning confocal fluorescence microscopes that provide higher frame rates [1,10,11], but they face both technical and fundamental limitations that hamper reaching the fluorescence-lifetime-limited speed with a decent SNR. Traditionally, a pair of galvanometric mirrors has long been used to steer the laser beam over the sample within a field of view (FOV), but this setup can only reach at best due to the inertia of the mechanical scanners [1,12]. The use of acousto-optic deflectors (AODs) overcomes the inertial barrier and provides times higher frame rates than the galvanometric mirrors, yielding (getting closer to the fluorescence lifetime limit, although further improvement by a factor of is needed) [1,13,14]. However, this traditional approach of increasing the scan speed will not simply address the speed problem since it fundamentally sacrifices fluorescence signal integration time at every dwell pixel in trade for the higher speed; the faster the scan is, the shorter the dwell time is, resulting in SNR reduction. Increasing the power of the excitation beam to compensate for the signal loss does not improve SNR due to the saturated excitation of the fluorophores.
In this paper, we present a confocal fluorescence microscope that overcomes these limitations and, for the first time to our knowledge, surpasses the highest possible frame rate constrained only by the fluorescence lifetime of fluorophores. This microscope is made possible by integrating a broadband, spatially distributed, dual-frequency comb or spatial dual-comb (SDC) (analogous to the dual-frequency comb in the frequency domain [15–18]) and quadrature amplitude modulation (QAM) for optimizing spectral efficiency into frequency-division multiplexing (FDM) [19–24] with single-pixel photodetection for signal integration in order to leverage FDM’s utility to its fullest. Specifically, we demonstrate confocal fluorescence microscopy at a record high frame rate of 16,000 frames/s. To the best of our knowledge, this is the first confocal fluorescence microscope that surpasses the fluorescence lifetime limit of fluorophores. Furthermore, to show its broad utility for biological and medical applications, we use the microscope to demonstrate 3D volumetric confocal fluorescence microscopy of cellular dynamics at a record high volume rate of 104 volumes/s and confocal fluorescence imaging flow cytometry of hematological and microalgal cells at a record high speed of up to 2 m/s, which corresponds to a throughput of up to 20,000 cells/s assuming a 100-μm cell spacing. Our FDM confocal fluorescence microscope is as versatile as conventional confocal fluorescence microscopes and holds promise for diverse applications in molecular cell biology, microbiology, neuroscience, pathology, immunology, and medicine.
A. Schematic of FDM Confocal Fluorescence Microscopy
The FDM confocal fluorescence microscope is schematically shown in Fig. 1(a) in which the QAM-SDC beam (the details of which are shown below and Fig. S2) is used as an optical source for confocal microscopy. Here the beam is scanned by a resonant galvanometric scanner (Cambridge Technology CRS8) at 8 kHz (corresponding to 16,000 frames/s) and focused by an objective lens ( or ) onto the fluorescent sample. The focal spots of the QAM-SDC beam are aligned in a line and swept in the direction perpendicular to the line to obtain a 2D confocal fluorescence image. For 3D movie acquisition, the objective lens is scanned by a piezoelectric actuator in the axial direction at 52 Hz (corresponding to 104 volumes/s). The fluorescence from the sample is collected by the same objective lens in an epifluorescence detection configuration, de-scanned by the same galvanometric scanner, focused by a lens, and detected by a Si avalanche photodetector (APD) (Thorlabs APD430A2/M, analogue output). A slit aperture is placed at the focal plane of the lens in front of the APD to make the detection setup confocal as the focused fluorescence creates a line-shaped image. The use of the slit aperture results in lower spatial resolution than the case of using a pinhole aperture, which is commonly used in conventional confocal microscopy, particularly in the and directions, but the spatial resolution still surpasses that of wide-field microscopy and allows us to perform high-resolution 3D imaging (see Supplement 1 for more details) as a result of the confocality [25,26]. The output electrical signal from the APD is recorded and analyzed by a computer via a digitizer with a sampling rate of 1.25 GS/s (Spectrum M4i.2212-x8). 2D confocal fluorescence images and a 3D volumetric confocal fluorescence movie are reconstructed by using our homemade digital lock-in detection algorithms implemented on LabVIEW. Details of the signal processing are described in Supplement 1 and Fig. S4.
B. Spatial Dual-Comb Beam Generation with Quadrature Amplitude Modulation
The key component of the FDM confocal fluorescence microscope is the SDC beam used to illuminate the sample. The generation of the SDC beam is schematically shown in Fig. 1(b). First, a continuous-wave (CW) laser beam is split by the first half beam splitter (HBS) into two beams. The first beam is frequency-shifted by an acousto-optic deflector (AOD) driven by a multi-tone signal and diffracted at angles that correspond to the driving signal’s different modulation tones. The diffracted beam consists of multiple beams having different optical frequency shifts and different propagation directions that correspond to different frequency components of the multi-tone signal. The second beam is frequency-shifted by an acousto-optic frequency shifter (AOFS) and then frequency-shifted by another AOD driven by another multi-tone signal and diffracted just like the first beam. These two beams (comb 1, comb 2) are recombined at the second HBS in an interferometric configuration to produce an array of discrete beams with each having a different beat frequency or a spatially distributed dual frequency comb that consists of a series of equally spaced frequency elements.
The frequency-domain picture of the spatially distributed frequency comb is shown in Fig. 1(c), where the assignments of the multi-tone driving signal to the AODs and the driving signal to the AOFS are also depicted. Here, both AODs are set to have a modulation bandwidth of with a tone spacing of such that these parameters satisfy the relation , where is the number of tones, while the frequency shift is set to be such that the two frequency combs (comb 1, comb 2) are not overlapped in the frequency domain. In the spatial domain, these two frequency combs are overlapped at the second HBS and form an array of linearly aligned focal spots after passing through an objective lens in such a manner that beats or pairs are formed at each focal spot by the closest comb lines, the second closest comb lines, etc., as indicated in Fig. 1(c). As a result, when the spatially distributed dual-frequency comb beam is measured by a photodetector, the comb lines on the photodetector’s output electrical signal have frequencies of such that the comb beam has comb lines having a modulation bandwidth of , that is, twice the bandwidth of the AODs. The DC component of the output electrical signal from the photodetector is removed by a high-pass filter whose cutoff frequency lies between the DC frequency and lowest comb line frequency.
To fully utilize the frequency band obtained by the SDC, we also employ QAM—a technique of combining two amplitude-modulated signals into a single frequency channel to double the effective bandwidth [27–29]. By using the QAM, we can obtain the spectral efficiency of standard point-scanning confocal microscopy in the FDM-based system (see Supplement 1). In QAM, two beams are prepared for each of the tone frequencies, having the phase difference in the frequency beat. Modulation signals caused by the two beams are separated by digital signal processing and can thus be considered as independent signals. QAM is implemented by using the two beams’ polarization degree of freedom. The implementation of the QAM is shown in Fig. 1(d). The beams from the two AODs are set to be linearly polarized at 45 degrees and right circularly polarized, respectively. After their combination at the HBS, the horizontal (quadrature) and vertical (in-phase) components of the QAM-SDC beam are extracted by a polarization beam splitter (PBS), where the phase difference between the in-phase and quadrature components comes from the phase difference between the horizontal and vertical components of the right circularly polarized light. The resultant QAM-SDC beam is focused onto the target sample for illumination. Either the in-phase or quadrature component of the other output of the QAM-SDC beam from the PBS is simultaneously measured by a photodetector and digitized by the same digitizer to monitor the phase reference of each tone.
Our demonstration of the QAM-SDC beam generation is shown in Fig. 1(e). Here the wavelength of the laser (Coherent Genesis CX 488 STM) is 488 nm. The modulation frequency range of both AODs is 200–400 MHz (Brimrose TED-300-200-488). We synthesized driving signals to the AODs with an arbitrary waveform generator at 1.2 GS/s (Signatec PXDAC4800) and amplified it with a 30-dB radio-frequency (RF) amplifier to produce 100 comb lines with the comb spacing of () such that . We also used a tone generator (Rohde & Schwarz SMC100A) to produce the driving signal to the AOFS (Isomet 1250C) such that . A non-polarizing HBS was used to spatially combine the two frequency combs. The in-phase and quadrature beams were generated by combining the two frequency combs. To obtain the combs in Fig. 1(e), we used a 1-MHz high-pass filter to remove the unwanted DC component of the output electrical signal from a photodetector that measures the SDC beam. Furthermore, to reduce detrimental crosstalk between neighboring comb lines, we used a crosstalk cancellation technique (see Supplement 1) to suppress the power of the crosstalk components down to less than 2% of the power of the wanted neighboring comb lines. The total power of the SDC beam incident on the sample was , which corresponds to an intensity of . Note that since the bandwidth of the dual frequency comb is 400 MHz, which corresponds to an oscillation period of 2.5 ns, the FDM confocal fluorescence microscope achieves the fluorescence-lifetime-limited speed with most available fluorescent probes (see Supplement 1).
C. Imaging Speed of FDM Confocal Fluorescence Microscopy
The imaging speed of the FDM confocal fluorescence microscope is characterized by the modulation bandwidth of the QAM-SDC beam, which provides an explicit definition of the fluorescence-lifetime-limited speed. Assuming the available signal bandwidth of the QAM-SDC beam as and the number of pixels in the direction as , the frequency bandwidth of is assigned to each comb line (here the factor of 2 comes from the fact that the in-phase and quadrature components of the QAM-SDC beam share the frequency band). The highest frequency component that each comb line carries without inter-comb-line crosstalk is given by because the bandwidth has both positive and negative frequency components of the fluorescence signal. Thus, assuming Nyquist sampling in the direction, the line scanning rate is given by , resulting in the effective pixel readout rate of , which characterizes the imaging speed. On the other hand, the fluorescence lifetime limits the frequency bandwidth of the image data signal up to , which is characterized by the frequency response function given by30]. Therefore, we define the fluorescence-lifetime-limited imaging speed by the bandwidth as . As a result, the FDM confocal fluorescence microscope with QAM surpasses the fluorescence lifetime limit if the available signal bandwidth exceeds . The definition of the fluorescence-lifetime-limited imaging speed using the signal bandwidth is also employed in conventional laser-scanning confocal fluorescence microscopes (see Supplement 1). Since the FDM confocal fluorescence microscope and the conventional laser-scanning confocal fluorescence microscope assume a single-pixel photodetector (i.e., a single-channel data transmission device) for image data acquisition in common, their fluorescence-lifetime-limited imaging speeds are uniformly characterized in the frequency domain. In other words, in the context of communication engineering, the fluorescence lifetime limits the channel capacity of the image acquisition in these microscopes, which is not the case if a multi-pixel photodetection device such as a line sensor or an image sensor is employed for image acquisition (e.g., spinning-disk confocal fluorescence microscopes). Note that the FDM confocal fluorescence microscope can obtain fluorescence images within a period shorter than the fluorescence lifetime limit with a reasonable SNR since its pixel dwell time is longer than that of conventional confocal fluorescence microscopes and can hence compensate for the decreased fluorescence signal level caused by the fluorescence lifetime limit.
A. Characterization of FDM Confocal Fluorescence Microscopy
We characterized the FDM confocal fluorescence microscope at the maximum frame rate of 16,000 frames/s. Assuming that the resonant scanner’s motion was sinusoidal, we numerically corrected image distortion caused by its nonuniform scanning velocity. Since the intensity of each comb line was not uniform due to the frequency dependence of the transmission efficiency of the driving signal and the deflection efficiency of the AODs [Fig. 1(e)], we normalized the intensity map of the acquired image by the intensity profile of the SDC beam. We also acquired two-color fluorescence images by separately detecting two color components of the fluorescence light with a dichroic mirror and two APDs (see Supplement 1, Fig. S2) while acquiring bright-field images by detecting the transmitted SDC beam. Figure 2(a) shows an image of 6-μm fluorescent beads in the FOV obtained with a objective lens. The FOV, number of pixels, and spatial resolution of the microscope are , , and in the direction (beam array direction) and direction (galvanometric scan direction), respectively. As the fluorescent lifetime of the beads is about 1.5 ns, which corresponds to the response frequency of about 184 MHz (defined as the frequency at the modulation recovery of 50%) (see Supplement 1), the right part of the image was degraded due to the decreased SNR, indicating that the image was taken beyond the fluorescence-lifetime-limited speed. The fluorescence lifetime was estimated from the measured phase delay of the fluorescence from each spot as indicated in Fig. 2(a) (see Supplement 1). The data was fitted by a theoretical curve, yielding the aforementioned fluorescence lifetime value. A theoretical curve of the fluorescence intensity decay is also shown in the figure, which can be used for the image intensity correction. Figures 2(b) and 2(c) show a two-color image of MCF-7 breast cancer cells stained by Calcein-AM (a cell-permeant fluorescent dye used for labeling the cytoplasm of living cells) and a three-color image of microalgal cells (Euglena gracilis) stained by SYTO16 (a cell-permeant green fluorescent nucleic acid stain) taken with a objective lens. The FOV, number of pixels, and spatial resolution of the microscope are , , and in the and directions, respectively. The lifetime of Calcein-AM was estimated, by using the same procedure as above, to be 2.9 ns, which is in good agreement with a literature value . The phase delay of SYTO16 from the Euglena gracilis cells was reliably fitted by 82% of 4.1 ns of the fluorescence lifetime and 18% of 0.33 ns of the fluorescence lifetime. The former lifetime is consistent with a literature value .
B. Demonstration of 3D Volumetric FDM Confocal Fluorescence Microscopy
To show the broad utility of the FDM confocal fluorescence microscope, we used it to demonstrate 3D volumetric confocal fluorescence microscopy at a high volume rate of 104 volumes/s. Here a objective lens was scanned in the axial direction by the piezoelectric actuator driven by a 52-Hz square-wave signal. Since the actual motion of the objective lens was nearly sinusoidal due to the limited response frequency of the actuator such that the obtained 2D frames were unevenly spaced in the depth direction, we numerically corrected it and generated evenly spaced 2D frames using linear interpolation. In addition to the lateral spatial resolution of (), the axial spatial resolution is 5.8 μm (, axial direction). Figure 3 shows two-color 3D confocal fluorescence movies of freely moving motile Euglena gracilis cells (with image processing such as spatial low-pass filtering for noise reduction at the cost of spatial resolution, volume-by-volume correction of object position offsets, and correction of image intensity decay due to photobleaching). Euglena gracilis is a species of flagellated eukaryotes in the phylum Euglenida found in freshwater and has been used in biological research as a model organism for decades [33,34]. In general, it is difficult to perform 3D fluorescence imaging of motile Euglena gracilis cells because of their rapid motion with flagella. The green spots in Fig. 3 indicate the fluorescence of SYTO9, which was mainly emitted from the nucleus and also from the entire cell body by nonspecific binding of the dye, whereas the red spots indicate the autofluorescence of intracellular chlorophyll. The box size is . By virtue of the unprecedentedly high volume rate of 104 volumes/s, the 3D dynamical motion (swimming with flagella and flow) of the cells was captured without motion blurring. Visualization 1 and Visualization 2 show slow-motion movies of the cells.
C. Demonstration of FDM Confocal Fluorescence Imaging Flow Cytometry
Furthermore, we used the FDM confocal fluorescence microscope to perform high-throughput confocal fluorescence imaging flow cytometry as an important practical application . Specifically, we obtained confocal fluorescence images of blood cells flowing in a hydrodynamic-focusing microfluidic channel. The QAM-SDC beam was focused across the channel and was not scanned by the resonant scanner since the flow provided an equivalent scan in the direction orthogonal to the array beam. First, we imaged murine neutrophils and lymphocytes isolated from bone marrow with a objective lens with a numerical aperture of 0.95. The flow speed was set to 1 m/s (corresponding to a high throughput of 10,000 cells/s with an average spacing of 100 μm between consecutive cells), which is comparable to the flow speed of conventional flow cytometers. The cells were stained by SYTO16 for highlighting the nuclei of the cells. The bright-field images of the cells were also captured. Out-of-focus cell images were rejected from the subsequent image data analysis. Figure 4(a) shows the images of the murine neutrophils and lymphocytes, while Fig. 4(b) shows a scatter plot of the cells with three morphological features of the cells (cell area, nucleus area, and nucleus complexity) where a support vector machine (SVM) [36,37] was applied to classify the cells. The morphological features of each cell were extracted from a digitally segmented area in each cell image. Consequently, the two cell types were successfully classified with a high accuracy of 86.33%. Additionally, we performed SVM-based machine learning for morphological features obtained from each cell (see Table S1 in Supplement 1), which resulted in a further improved accuracy of 98.65%. These results verify the ability of our confocal fluorescence imaging flow cytometry to differentiate between murine neutrophils and lymphocytes via the identification and SVM of the morphological features of the nuclei and outer shape, which is not possible with conventional flow cytometry.
Next, we conducted high-throughput confocal fluorescence imaging flow cytometry of Euglena gracilis cells under different culture conditions (nitrogen-sufficient and nitrogen-deficient cultures) using a objective lens with a numerical aperture of 0.75 for imaging. Here, nitrogen deficiency is known to induce Euglena gracilis cells to accumulate lipids that can be refined into biofuels  (see Supplement 1). The flow speed was set to 2 m/s (corresponding to a high throughput of 20,000 cells/s with an average spacing of 100 μm between consecutive cells). Likewise, out-of-focus cell images were rejected from the subsequent image data analysis. Figure 4(c) shows the images of the Euglena gracilis cells in which their outer shape (from bright-field images), intracellular lipids (stained by BODIPY505/515), and chlorophyll (autofluorescent) are evident. Figure 4(d) shows a scatter plot of the cells with three morphological features of the cells (lipid amount per cell area, chlorophyll amount per cell area, and opacity) where a SVM was applied to classify the cells. The cells under the two different culture conditions were successfully classified with a high accuracy of 94.60%. Additionally, we performed SVM-based machine learning for morphological features obtained from each cell using CellProfiler (see Table S2 in Supplement 1) , which resulted in a further improved accuracy of 99.80%. These results firmly confirm the ability of our confocal fluorescence imaging flow cytometry to accurately recognize the differences between the cells in the two cultures with the single-cell resolution. This ability is particularly useful for real-time evaluation of various cultivation and genetic engineering methods for highly efficient microalgae-based metabolic engineering of food supplements, pharmaceutical drugs, and biofuels.
In addition to the significance of overcoming the fluorescence lifetime limit, the results in Fig. 2 suggest that our method can offer additional information about cells via the fluorescence lifetime, extending potential applications of the method, such as fluorescence lifetime imaging (FLIM) [40,41]. In fact, we can measure the phase delay of fluorescence signals at each pixel, which reflects the fluorescence lifetime of fluorophores within the pixel area. Specifically, the measurement of the phase delay can be done by taking the short-time Fourier-transform (STFT) approach to image reconstruction [see Supplement 1 and Fig. S4(c)]. Moreover, the intensity decay at each pixel due to the high temporal frequency modulation that was seen in Fig. 2 can potentially be leveraged to enhance the fluorescence lifetime information of each pixel. Note that requirements for the actual implementation of the fluorescence lifetime measurement (or imaging) highly depend on the type of the measurement. For example, if a rough estimation of the fluorescence lifetime without resolving its spatial distribution is of concern, a single-scan measurement may be sufficient, meaning that the measurement can be done at an ultrahigh frame rate such as 16,000 frames/s. On the other hand, if a spatially resolved high-precision measurement of the fluorescence lifetime is of interest, which is a common situation where FLIM is employed, multiple scans or a slower scan may be required to obtain a sufficient precision at the cost of the imaging speed. Moreover, if the fluorescence decay of target fluorophores is expressed by a multi-exponential function, multiple phase measurements with different modulation frequencies per pixel are necessary to estimate the multiple lifetime constants. In this case, multiple beam scans with position displacement in the direction is needed.
While our FDM confocal fluorescence microscope is somewhat similar to the microscope in Ref.  in excitation beam generation, there are significant differences between them in imaging speed and other specifications. First, and most significant, the imaging speed of our microscope is a factor of 8 higher than the microscope in Ref. , which comes from the doubled bandwidth of the AODs, the dual-AOD configuration of the SDC beam generation that provides the modulation bandwidth of the excitation beam a factor of 2 higher than that of the AODs, and the QAM that doubles the spectral efficiency. Second, the spatial resolution of our microscope in the direction is identical to that of conventional laser-scanning confocal fluorescence microscopy with a slit aperture, while the microscope in Ref.  is lower due to its FDM beam generation principles, namely, the interference between localized beam spots and a line focus beam (having a non-localized spatial distribution in the direction). Third, as discussed above, our microscope provides the phase delay information in addition to the intensity information, while the modulation bandwidth in Ref. , which is a factor of 4 lower than that of our microscope, is not sufficient to obtain the phase delay information for most fluorophores (see Supplement 1, Fig. S1).
The FDM confocal fluorescence microscope is not limited to the optical configurations shown in the above experiments and can work with an objective lens with any magnification and numerical aperture (including liquid immersion ones) as well as standard laser-scanning confocal microscope settings. An additional constraint characteristic of our method is that the FOV in the direction (i.e., along the beam spot array) is given by the magnification (or focal length) and numerical aperture of the objective lens. Assuming an identical spatial profile for the excitation beams incident on the objective lens, the FOV in the direction is inversely proportional to the magnification of the lens. Since the diameter of the excitation beams incident on the objective lens is adjustable by reconfiguring the beam expander so that it fits the pupil diameter of each objective lens, the FOV in the direction is inversely proportional to the numerical aperture of the objective lens.
As discussed above, the speed performance of the FDM confocal fluorescence microscope is characterized by the pixel readout rate. This indicates the existence of a trade-off relation between the number of pixels in the direction and the highest possible scanning speed in the direction (frame rate), which is explained by the comb-line structure shown in Fig. 1(c). Given that the available frequency bandwidth in the radio frequency domain is constant, for instance, a larger number of comb lines (i.e., an increased number of pixels in the direction) results in a smaller frequency spacing between adjacent comb lines, meaning that slower modulation (i.e., slower scanning speed) is allowed to avoid inter-pixel crosstalk. Therefore, higher scanning speed (i.e., higher pixel rate in the direction) is allowed if the number of pixels in the direction is compromised. A higher frame rate can also be achieved by compromising on the number of pixels in a single frame. Moreover, it is noteworthy that the present pixel readout rate (; the lowest possible sampling frequency that covers the temporal modulation frequency bandwidth of the excitation beams) exceeds that of the state-of-the art image sensors with sufficient sensitivity for fluorescence imaging, such as scientific CMOS cameras () and EM-CCD cameras (), manifesting that our microscope is essentially faster even compared with image-sensor-based fluorescence microscopes, including spinning disk confocal microscopes.
The SNR of fluorescence images provided by the FDM confocal fluorescence microscope can be further improved by making modifications to the microscope. For example, since the intensity of the excitation beam is less than the saturation level of typical fluorophores, using a laser with higher power improves the SNR. Similarly, reducing the number of frequency comb lines in the SDC beam also improves the SNR at the cost of the number of pixels in the direction. Also, in the weak signal regime where the detector noise is dominant, using a photomultiplier tube instead of the APD improves the SNR due to its higher photoelectron multiplication gain.
The FDM confocal fluorescence microscope has a few more advantages in 3D volumetric imaging than light-sheet microscopy (LSM). First, its volume rate surpasses that of LSM by virtue of its single-pixel photodetection for image acquisition. In fact, the limited frame rate of CCD/CMOS image sensors used in LSM greatly limits its volume rate. Second, our microscope has less structural complexity in the sample holder than LSM, which requires a custom-made sample holder and hence has higher degrees of freedom for practical applications.
As the FDM confocal fluorescence microscope requires higher excitation laser power than conventional laser-scanning confocal fluorescence microscopes, light-induced effects such as photobleaching, photodamage, and phototoxicity should be taken into account for practical use, although there are techniques to mitigate the effects. For example, time-lapse microscopy may be difficult, depending on the cell type. Meanwhile, in imaging flow cytometry, photobleaching and photodamage are less problematic because each cell is exposed by the excitation beam only once for a short period of time, indicating that imaging flow cytometry is a suitable application of the microscope.
In summary, we demonstrated, to the best of our knowledge, the fastest confocal fluorescence microscope that operates at the highest possible frame rate (16,000 frames/s in 2D, 104 volumes/s in 3D) and goes beyond the fluorescence lifetime limit of fluorophores. This technique is based on an integration of a broadband, spatially distributed, dual-frequency comb and QAM into FDM. To show its broad utility for biological and medical applications, we applied it to 3D volumetric confocal fluorescence microscopy of cellular dynamics at 104 volumes/s and confocal fluorescence imaging flow cytometry of hematological and microalgal cells at flow speeds of 1 m/s and 2 m/s, respectively. Our FDM confocal fluorescence microscopy is as versatile in settings as conventional confocal fluorescence microscopy and is expected to be a valuable tool for a diverse range of applications in molecular cell biology, microbiology, neuroscience, pathology, immunology, and medicine.
ImPACT Program of the Council for Science, Technology and Innovation (Cabinet Office, Government of Japan); Japan Society for the Promotion of Science (JSPS) (25702024, 25560190); JGC-S Scholarship Foundation; Mitsubishi Foundation; Konica Minolta Imaging Science Foundation Encouragement Award; Takeda Science Foundation; Noguchi Shitagau Research Grant; Epson International Scholarship Foundation; Ogino Prize; Advanced Photon Science Alliance of the Ministry of Education, Culture, Sports, Science and Technology (MEXT).
See Supplement 1 for supporting content.
1. J. B. Pawley, ed., Handbook of Biological Confocal Microscopy, 2nd ed. (Springer, 1995).
2. H. R. Petty, “Spatiotemporal chemical dynamics in living cells: from information trafficking to cell physiology,” Biosystems 83, 217–224 (2006). [CrossRef]
3. J. P. Nguyen, F. B. Shipley, A. N. Linder, G. S. Plummer, M. Liu, S. U. Setru, J. W. Shaevitz, and A. M. Leifer, “Whole-brain calcium imaging with cellular resolution in freely behaving Caenorhabditis elegans,” Proc. Natl. Acad. Sci. USA 113, E1074–E1081 (2015). [CrossRef]
4. A. L. Polglase, W. J. McLaren, S. A. Skinner, R. Kiesslich, M. F. Neurath, and P. M. Delaney, “A fluorescence confocal endomicroscope for in vivo microscopy of the upper- and the lower-GI tract,” Gastrointest. Endosc. 62, 686–695 (2005). [CrossRef]
5. D. C. Adler, Y. Chen, R. Huber, J. Schmitt, J. Connolly, and J. G. Fujimoto, “Three-dimensional endomicroscopy using optical coherence tomography,” Nat. Photonics 1, 709–716 (2007). [CrossRef]
6. P. K. Chattopadhyay, T. M. Gierahn, M. Roederer, and J. C. Love, “Single-cell technologies for monitoring immune systems,” Nat. Immunol. 15, 128–135 (2014). [CrossRef]
7. D. A. Basiji, W. E. Ortyn, L. Liang, V. Venkatachalam, and P. Morrissey, “Cellular image analysis and imaging by flow cytometry,” Clin. Chem. Lab. Med. 27, 653–670 (2007). [CrossRef]
8. K. Goda, A. Ayazi, D. R. Gossett, J. Sadasivam, C. K. Lonappan, E. Sollier, A. M. Fard, S. C. Hur, J. Adam, C. Murray, C. Wang, N. Brackbill, D. Di Carlo, and B. Jalali, “High-throughput single-microparticle imaging flow analyzer,” Proc. Natl. Acad. Sci. USA 109, 11630–11635 (2012). [CrossRef]
9. M. Y. Berezin and S. Achilefu, “Fluorescence lifetime measurements and biological imaging,” Chem. Rev. 110, 2641–2684 (2010). [CrossRef]
10. P. W. Winter and H. Shroff, “Faster fluorescence microscopy: advances in high speed biological imaging,” Curr. Opin. Chem. Biol. 20, 46–53 (2014). [CrossRef]
11. S. Choi, P. Kim, R. Boutilier, M. Y. Kim, Y. J. Lee, and H. Lee, “Development of a high speed laser scanning confocal microscope with an acquisition rate up to 200 frames per second,” Opt. Express 21, 23611–23618 (2013). [CrossRef]
12. A. Draaijer and P. M. Houpt, “A standard video-rate confocal laser-scanning reflection and fluorescence microscope,” Scanning 10, 139–145 (1988). [CrossRef]
13. “VTi—VT-Eye,” http://www.visitech.co.uk/vt-eye.html.
14. V. Bansal, S. Patel, and P. Saggau, “High-speed addressable confocal microscopy for functional imaging of cellular activity,” J. Biomed. Opt. 11, 34003 (2006). [CrossRef]
15. F. Keilmann, C. Gohle, and R. Holzwarth, “Time-domain mid-infrared frequency-comb spectrometer,” Opt. Lett. 29, 1542–1544 (2004). [CrossRef]
16. B. Bernhardt, A. Ozawa, P. Jacquet, M. Jacquey, Y. Kobayashi, T. Udem, R. Holzwarth, G. Guelachvili, T. W. Hänsch, and N. Picqué, “Cavity-enhanced dual-comb spectroscopy,” Nat. Photonics 4, 55–57 (2010). [CrossRef]
17. I. Coddington, N. Newbury, and W. Swann, “Dual-comb spectroscopy,” Optica 3, 414–426 (2016). [CrossRef]
18. S. Schiller, “Spectrometry with frequency combs,” Opt. Lett. 27, 766–768 (2002). [CrossRef]
19. E. D. Diebold, B. W. Buckley, D. R. Gossett, and B. Jalali, “Digitally synthesized beat frequency multiplexing for sub-millisecond fluorescence microscopy,” Nat. Photonics 7, 806–810 (2013). [CrossRef]
20. G. Futia, P. Schlup, D. G. Winters, and R. A. Bartels, “Spatially-chirped modulation imaging of absorption and fluorescent objects on single-element optical detector,” Opt. Express 19, 1626–1640 (2011). [CrossRef]
21. C.-S. Liao, P. Wang, P. Wang, J. Li, H. J. Lee, G. Eakins, and J.-X. Cheng, “Spectrometer-free vibrational imaging by retrieving stimulated Raman signal from highly scattered photons,” Sci. Adv. 1, e1500738 (2015). [CrossRef]
22. S. S. Howard, A. Straub, N. G. Horton, D. Kobat, and C. Xu, “Frequency-multiplexed in vivo multiphoton phosphorescence lifetime microscopy,” Nat. Photonics 7, 33–37 (2012). [CrossRef]
23. F. Wu, X. Zhang, J. Y. Cheung, K. Shi, Z. Liu, C. Luo, S. Yin, and P. Ruffin, “Frequency division multiplexed multichannel high-speed fluorescence confocal microscope,” Biophys. J. 91, 2290–2296 (2006). [CrossRef]
24. S. Weinstein and P. Ebert, “Data transmission by frequency-division multiplexing using the discrete Fourier transform,” IEEE Trans. Commun. Technol. 19, 628–634 (1971). [CrossRef]
25. C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35, 1169–1185 (1988). [CrossRef]
26. S. Kawata, R. Arimoto, and O. Nakamura, “Three-dimensional optical-transfer-function analysis for a laser-scan fluorescence microscope with an extended detector,” J. Opt. Soc. Am. A 8, 171–175 (1991). [CrossRef]
27. L. Hanzo, S. X. Ng, T. Keller, and W. Webb, Quadrature Amplitude Modulation: From Basics to Adaptive Trellis-Coded, Turbo-Equalised and Space-Time Coded OFDM, CDMA and MC-CDMA Systems, 2nd ed. (Wiley, 2004), p. 1136.
28. R. van Nee and R. Prasad, OFDM Wireless Multimedia Communications (Artech House, 2000).
29. W. Webb and L. Hanzo, Modern Quadrature Amplitude Modulation: Principles and Applications for Fixed and Wireless Communications (IEEE, 1994).
30. J. R. Lakowicz, Principles of Fluorescence Spectroscopy, 3rd ed. (Springer, 2006).
31. H. Choi, D. S. Tzeranis, J. W. Cha, P. Clémenceau, S. J. G. de Jong, L. K. van Geest, J. H. Moon, I. V. Yannas, and P. T. C. So, “3D-resolved fluorescence and phosphorescence lifetime imaging using temporal focusing wide-field two-photon excitation,” Opt. Express 20, 26219–26235 (2012). [CrossRef]
32. P. Herman, B. P. Maliwal, H.-J. Lin, and J. R. Lakowicz, “Frequency-domain fluorescence microscopy with the LED as a light source,” J. Microsc. 203, 176–181 (2001). [CrossRef]
33. M. Cramer and J. Myers, “Growth and photosynthetic characteristics of Euglena gracilis,” Arch. Mikrobiol. 17, 384–402 (1952). [CrossRef]
34. R. Schantz, M.-L. Schantz, and H. Duranton, “Changes in amino acid and peptide composition of Euglena gracilis cells during chloroplast development,” Plant Sci. Lett. 5, 313–324 (1975). [CrossRef]
35. J. Wu, A. H. L. Tang, A. T. Y. Mok, W. Yan, G. C. F. Chan, K. K. Y. Wong, and K. K. Tsia, “Multi-MHz laser-scanning single-cell fluorescence microscopy by spatiotemporally encoded virtual source array,” Biomed. Opt. Express 8, 4160–4171 (2017). [CrossRef]
36. L. Wang, ed., Support Vector Machines: Theory and Applications (Springer, 2005).
37. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines: And Other Kernel-Based Learning Methods (Cambridge University, 2000).
38. Y. Wakisaka, Y. Suzuki, O. Iwata, A. Nakashima, T. Ito, M. Hirose, R. Domon, M. Sugawara, N. Tsumura, H. Watarai, T. Shimobaba, K. Suzuki, K. Goda, and Y. Ozeki, “Probing the metabolic heterogeneity of live Euglena gracilis with stimulated Raman scattering microscopy,” Nat. Microbiol. 1, 16124 (2016). [CrossRef]
39. T. Blasi, H. Hennig, H. D. Summers, F. J. Theis, J. Cerveira, J. O. Patterson, D. Davies, A. Filby, A. E. Carpenter, and P. Rees, “Label-free cell cycle analysis for high-throughput imaging flow cytometry,” Nat. Commun. 7, 10256 (2016). [CrossRef]
40. C. Chang, D. Sud, and M. Mycek, “Fluorescence lifetime imaging microscopy,” Methods Cell Biol. 81, 495–524 (2007). [CrossRef]
41. P. Bastiaens, “Fluorescence lifetime imaging microscopy: spatial resolution of biochemical processes in the cell,” Trends Cell Biol. 9, 48–52 (1999). [CrossRef]