We present a theoretical investigation of an optical microscope design that achieves wide-field, multiphoton fluorescence microscopy with finer axial resolution than confocal microscopy. Our technique creates a thin plane of excitation light at the sample using height-staggered microlens arrays (HSMAs), wherein the height staggering of microlenses generate temporal focusing to suppress out-of-focus excitation, and the dense spacing of microlenses enables the implementation of structured illumination technique to eliminate residual out-of-focus signal. We use physical optics-based numerical simulations to demonstrate that our proposed technique can achieve diffraction-limited three-dimensional imaging through a simple optical design.
© 2013 OSA
Fluorescence is one of the most important optical contrasts in biological imaging, for it offers high molecular sensitivity and specificity. The general desire for obtaining three-dimensional (3D) information of dynamic biological systems has spurred the development of several fluorescence microscopies [1–4] with the ability to selectively image thin sections of an intact sample. This ability of optical sectioning can be categorized into two general regimes - intrinsic and extrinsic. Intrinsic optical sectioning utilizes an axially peaked excitation profile along the imaging path to obtain optical sectioning; examples include two-photon excitation microscopy (2PE)  and selective plane illumination microscopy (SPIM) . Extrinsic optical sectioning is performed post-excitation and primarily relies on a far-field imaging principle: the higher spatial-frequency components of the images decay more rapidly with defocusing . As a result, one can create a high-spatial-frequency excitation pattern and use appropriate spatial filters to selectively extract the in-focus signal. Confocal microscopy (CFM)  and structured illumination microscopy (SIM) are two examples that use physical and numerical spatial filters, respectively, to achieve equivalent optical sectioning .
Present imaging techniques, however, have important drawbacks. For CFM and 2PE, phototoxicity and photothermal damages are noted [1, 7], and are exacerbated if video-rate acquisition is required. To reduce these effects, spatially/temporally parallelized CFM/2PE systems, such as spinning disk confocal , multifocal multiphoton microscopy (MMM) , and time-multiplexed multifocal multiphoton microscopy (TM-MMM)  were developed. In these techniques, multiple foci are created as independent, parallel channels for excitation in and detection from the sample. To reduce crosstalk among parallel channels, however, the foci spacing has to be sufficiently sparse (typically <1 foci per 102-focus area, see Appendix A, B), which limits the degree of parallelization. SIM and SPIM do not suffer from parallelization issues; however both techniques have their own limitations. SIM can lead to significant photobleaching and low signal-to-noise ratio (SNR) in the processed images, because it excites fluorophores and receives fluorescence over a wide axial range at each acquisition. SPIM requires a compromise between axial resolution and the size of the field of view. Moreover, SPIM raises design issues and challenges in sample handling and manipulation for its separate excitation and imaging objectives in close proximity.
In principle, intrinsic and extrinsic optical sectioning can be combined to take the advantages of both. For example, it has been shown that a SPIM system can incorporate SIM to improve its optical sectioning when a reasonably large field of view is desired . Here, we propose a hybrid optical microscope design (Fig. 1) incorporating SIM and temporal focusing , a wide-field, intrinsic sectioning technique. Our design utilizes height-staggered microlens arrays (HS-MAs) having periodic patterns (Fig. 1(b)) to precisely control the time staggering for temporal focusing, and to provide an excitation pattern for SIM to further suppress residual out-of-focus signal. We herein refer to this approach as Microlens Array and Structured Illumination-based Plane-Projection Multiphoton Microscopy (MASI-PPMP).
2. MASI-PPMP: integrating (intrinsic) temporal focusing and (extrinsic) SIM
We have previously demonstrated a new technique, plane-projection multiphoton microscopy (PPMP) , in which temporal focusing with randomized excitation patterns is achieved through the randomly staggered light pulses (in the time domain) created by the surface roughness of a ground-glass optical diffuser. The randomized excitation patterns, as suggested by a previous report , could be used for SIM. However, mitigating the excitation patterns in the processed images requires considerable effort. For simplicity, periodic excitation patterns are commonly used , and chosen for the present study. Further, the design control afforded in the HSMA allows for optimization of the intrinsic sectioning.
The combination of the microlens arrays and temporal masks, as used in TM-MMM , can provide equivalent functions of the proposed HSMAs, although additional precision alignment and positioning are required. Notably, the step sizes of time staggering Δt in TM-MMM and MASI-PPMP are different. In TM-MMM, Δt is set to be much larger than the pulse duration τ0 to reduce temporal inter-foci interferences , by which individual foci can be considered as independent, parallel channels. In contrast, MASI-PPMP allows Δt < τ0 for practical reasons detailed below and in Supplementary Materials.
We begin the design of MASI-PPMP with consideration of the physical limitations of HS-MAs. The dimensions of HSMAs are limited by fabrication technologies as well as diffractive losses. Existing techniques cannot straightforwardly engineer micro-optics of depth variation >1 mm . In addition, when we consider the HSMA as an array of time-delay channels, a light pulse propagating in one channel can leak into its neighboring channels due to the nature of diffraction. A simple estimation of the appropriate dimensions of HSMAs is given in Appendix A. In short, to have negligible inter-channel light leakage, the spatial parameters of the HSMAs should satisfyEq. (1). These dimensions make the HSMA fabrication feasible through existing techniques, and can create up to ∼ 106 foci within a 2-inch aperture, thereby providing the periodic, high-spatial-frequency pattern for SIM. For a simple analysis, we assume that the total amount of time delay Δttot is separated equally into Nt distinct time-delay steps (with step size Δt, Eq. (2)). These time-delay steps are then arranged in a prescribed periodic pattern in the HSMA (Fig. 1(b)). Considering the propagation speed of light in a material, we have
Because of the limitation of Δhmax, the estimated Δt in Eq. (2) can be around or shorter than the pulse duration of conventional ultrafast oscillators or amplifiers above certain values of Nt. In such cases, we should take into account the temporal interferences among light pulses of different time delays. Notably, temporal masks with much larger Δttot have been proposed to avoid temporal interferences and to achieve scanningless TM-MMM . However, Eqs. (1) and (2) indicate that the required Δhmax falls far beyond the limits of existing fabrication techniques, and the aperture sizes of these temporal masks could be too large for standard biomedical microscopes (see Appendix B).
To have fabrication-feasible HSMAs and to improve the SNR of the SIM post-processed images, we turn to optimizing the parameters of HSMAs. In the following sections, we first construct a model that considers the temporal interferences (§3); through this model we investigate how intrinsic optical sectioning depends on the choice of Nt (and Δt) for a given Δttot (§4), and the arrangement of time delays in the HSMA (Appendix C). Then, we couple HSMAs with SIM to enhance optical sectioning (§5).
3. Construct a physical optics-based model taking into account temporal interferences
Given the temporal focusing effect in the proposed technique, and with the non-negligible temporal interferences among the light pulses, a time-independent model such as that previously used for TM-MMM  is no longer sufficient for analyzing the performances of our setup. Thus, we develop a new model taking into account the time-dependent optical phase of multiple spectral components in an ultrafast pulse. For simplicity, we use a Gaussian-pulse approach, i.e., in the excited area, the electric field E(r, z, t) at position (r, z) (z = 0 at the specimen plane) and time t is approximated as the Gaussian-weighted sum of a series of constant-interval (in k–space), in-phase light waves,12], we set k0 ≈ 7.85 × 104 cm−1 and a pulse duration τ0 of ≈ 30 fs (by using an appropriate σk). We then employ the amplitude point spread function (PSF) derived previously for high numerical-aperture (NA) lenses [14, 16], Ekj, as
Through Eqs. (3) and (4) we can numerically evaluate the time-dependent amplitude PSF of an ultrafast pulse focused by a well-corrected objective lens, EPSF (r,z,t). Having solved EPSF numerically, we estimate the electric field near the specimen plane, ESP(r,z,t), as the linear superposition of the EPSF from the individual microlenses,Media 1, which also reveals the dynamic process of temporal focusing (see Appendix D). Notably, to fulfill the wide-field illumination condition and to simplify the simulations, our model assumes that a ’unit’ microlens array is infinitely replicated in the transverse coordinates, as shown in Fig. 1(b). Under such a periodic condition, the physical optics properties in the projected region of one unit microlens array is sufficient to represent the entire system (the validity and details of this model are given in Appendix E).
Through Eq. (5), the excitation intensity profiles ISP(r, z) can be derived by integrating the excitation intensity over time, as
4. Optimize intrinsic optical sectioning through tuning Nt and Δt
To quantify the intrinsic optical sectioning created by a particular design of HSMA, we calculate the total fluorescence signal at a depth z, S(z), through integration of ISP(r, z) in Eq. (6) over the transverse coordinates, as
To determine the choice of (Nt, Δt) that produces the most efficient optical sectioning, we evaluate S(z) and the ratio of out-of-focus and in-focus signal, Sout / Sin, for various sets of Nt and Δt (constrained by Eq. (2)) with two inter-foci spacings, dfoci(≡ d/M, where M is the magnification of the microscopy system) ≈ 0.4 and 0.8 λ0, corresponding to ∼ 56 and ∼ 14 foci per 102-focus area, respectively (Fig. 2). Because of the square geometry of our HSMAs, we examine Nt = 22, 32, 42, ... and 92. Analyzing Sout /Sin reveals that the decay of the out-of-focus excitation significantly slows down between Nt = 25 and 64, corresponding to Δt ≈ 20-8 fs (i.e., 2/3τ0-1/4τ0) (Fig. 2(c)). We observe similar trends of Sout / Sin for various geometrical arrangements of the distinct time-delay steps (see Appendix C). This result suggests that the intrinsic optical sectioning for a fixed Δttot is optimized when Δt is slightly smaller than τ0. Further reducing Δt (equivalent to increasing Nt (Eq. (2))) can complicate the fabrication of the HSMAs without major improvement of the intrinsic optical sectioning. In addition, increasing dfoci leads to weaker out-of-focus excitation and a less complex axial excitation profile S(z) (Fig. 2(b)).
5. Obtain extrinsic optical sectioning via implementing SIM
Next, we implement SIM to extrinsically remove the residual out-of-focus signal of the optimized HSMA. SIM relies on the post processing of multiple images, each noted as IIM. Because the emitted fluorescence (wavelength assumed to be ∼ 0.56λ0) from the specimen is generally incoherent, we can estimate IIM from a convolution of the excitation intensity profile ISP and the intensity PSF of the microscopy system, ISYS, asEq. (8) then becomes 6]. In our case, the periodicity of ISP is 2-dimensional. We thus use 3-by-3 shifts (IIM1, IIM2, ..., IIM9 denote the obtained images) and apply SIM post-processing to extract the optically sectioned images, Eq. (7) with ISIM to evaluate the overall strength of optical sectioning (Fig. 3).
The results show that MASI-PPMP can achieve a better optical sectioning than CFM. Such results are reasonable because the fundamental mechanisms of intrinsic and extrinsic optical sectioning are independent from each other and can be combined. We further simulate the images of a virtual 3D object obtained by the different imaging techniques (Fig. 4) to illustrate the optical sectioning of MASI-PPMP. Compared with conventional epifluorescence microscopy (Fig. 4(b)), the images obtained before SIM post-processing (denoted as MA-PPMP) shows the ability of intrinsic optical sectioning. In the log-scale intensity plot (Fig. 4(c)), MASI-PPMP has the highest signal contrast between fluorescent and non-fluorescent areas, consistent with the results in Fig. 3. Moreover, the reconstructed 3D views show that MASI-PPMP successfully reproduces the details of the object (Fig. 4(d)).
We should note that the SIM process introduced here, i.e., the 9-frame imaging procedure and Eq. (10), is not the only way to remove out-of-focus signal from the obtained images IIM. Other methods using a high-spatial-frequency illumination pattern to distinguish in-focus and out-of-focus signals, such as HiLo microscopy [17, 18], may also be applied to the proposed optical setup. These methods use different imaging procedures and post-processing algorithms, which may lead to different imaging properties such as speed and spatial resolutions, such that different methods may be most suitable for different imaging applications. For example, the HiLo method, whose algorithms are more complicated than Eq. 10, requires only 2 frames to retrieve in-focus signal, one frame each with uniform and non-uniform illumination; the reduced number of frames may therefore shorten the imaging time. However, such an advantage exists only when the frame rate of the camera is much lower than the repetition rate of the ultrafast pulse train. If these two rates fall in the same order of magnitude, one will need to increase the exposure time to shift the structural illumination pattern around the sample so as to mimic the effect of uniform illumination, and the overall imaging time will eventually be similar to SIM.
In conclusion, our numerical modeling shows that MASI-PPMP can efficiently provide intrinsic optical sectioning via temporal focusing effect generated by the HSMAs, and achieve an axial resolution finer than CFM via the implementation of extrinsic optical-sectioning technique. Our analysis provides the design guide of HSMAs with small time-delay increments; the dimensions of the proposed HSMAs are compatible with standard biomedical microscopes, and feasible for existing fabrication methods. In contrast to conventional SIM, the intrinsic sectioning of our method can reduce photobleaching and increase the SNR of the processed images.
Compared with parallelized CFM and 2PE, MASI-PPMP has a denser spacing of foci (> 10 foci per 102-focus area), and thus has greater potential in high-speed imaging. In addition to the finer axial resolution than CFM, the lateral resolution can be enhanced via utilization of the periodic excitation structures and Fourier analysis of the obtained images . Moreover, MASI-PPMP can use the pulse train generated by ultrafast amplifiers, thereby yielding a sufficient signal level and moderate impact on the specimen at high acquisition rates, as demonstrated in our previous PPMP setup . Taken together, we have shown that MASI-PPMP has the potential to achieve (better-than-conventional) diffraction-limited, high-frame-rate 3D imaging with a simple, wide-field optical design.
Appendix A: Determining the dimensions of the time-delay channels
In this section we discuss the appropriate dimensions of the height-staggered microlens arrays (HSMAs) utilized in our technique. As mentioned in the main article, the light waves propagating along one particular time-delay channel can leak into the neighboring channels due to diffraction. In the microscope, this leakage appears as stray light, which induces multiphoton excitation at all depths and thus increases out-of-focus excitation.
To quantify such an effect, we use a simple model consisting of two adjacent microlenses with height staggering Δh, as shown in Fig. 5. When the light of wavelength λ0 exits from channel 1 (Ch1) and propagates to distance z, diffraction theory predicts the beam size, dB, as a function of λ0 and z. With this regard, we consider the inter-channel leakage to be non-negligible if dB before propagating to z = Δh is larger than the diameter of the microlens, d. In other words, the inter-channel leakage is negligible ifFig. 5, is to set the focus of Ch1 at z = Δh, so that the dB decreases upon exiting Ch1 and reaches its minimum, df, at z = Δh. Based on the diffraction-limited spot size of an ideal thin lens, we can estimate df as 20]. Combining Eqs. (11), (12) and the geometry, we have Eq. (14), and thus derive
The argument here also applies to the design of the combination of temporal masks and microlens arrays used in TM-MMM . In such a case, the ends of the time-delay channels are flat surfaces, and one can use the single-slit diffraction to approximate the beam size with propagation. Such an approach yields a similar criterion.
Appendix B: Size of the microlens arrays to achieve scanningless imaging using conventional time-multiplexing approach
In this section we estimate the size of the HSMAs for scanningless TM-MMM, using the criterion derived in the previous section. To achieve a complete, gap-less plane of illumination for a truly scanningless wide-field imaging requires the placement of adjacent foci no more than one focal spot diameter apart. However, to preserve the intrinsic optical sectioning of conventional multiphoton microscopy, the foci (of the same timing) have to be spatially separated by a certain distance, d0, which is commonly much larger than the size of the focal spot of the objective lens, dfs. As a result, there will be substantial un-illuminated area between the foci. For example, take an oil-immersion, NA 1.42 objective lens: the resultant d0 and dfs is approximately 5 λ and 0.3 λ, which makes the fraction of un-illuminated area approximately , equivalent to ∼0.4 foci per 102-focus area. To cover the un-illuminated area, Egner et al.  proposed to use foci that are largely separated in time, so that the interferences among these foci is negligible even though they partially overlap in space. Consequently, the number of distinct time-delay steps required to cover the un-illuminated area, Nt, can be estimated as
Given Nt, and using the propagation speed of light in the HSMA material, we can estimate the difference of height between the longest and shortest time channels, Δhmax, as:14], are equal to or larger than twice the pulse duration of the light pulse, τ0, i.e., Δt ≥ 2τ0 (where τ0 was set as ≈ 100 fs for conventional ultrafast oscillators) . Accordingly, we find that Δh ≈ 33 mm and d ≈ 0.15 mm using Eqs. (15)–(17), wherein λ and n are assumed to be ∼ 800 nm and 1.5, respectively. As a result, if one needs 1000-by-1000 foci in the field of view (FOV), the typical aperture of the entire HSMA will be as large as 1000×d ≈ 150 mm, which is considerably larger than the optical elements of a standard biomedical microscope. Furthermore, when an ultrafast pulse propagates in a long, narrow glass cylinder, both group velocity dispersion and modal dispersion broaden the pulse width, and thus reduce the axial resolution as well as excitation efficiency. From Eqs. (15), (17) and the requirement of Δt ≥ 2τ0, we can derive that Δh ∝ τ0 and , indicating that Δh and d can be scaled down via using small τ0. However, commonly available ultrafast systems typically achieve τ0 ≥ 10 fs, such that the resultant Δh and d remain considerably large. Moreover, the ultrafast pulses of smaller τ0 (and thus greater spectral bandwidth) will suffer more from group velocity dispersion and modal dispersion while propagating in the HSMA materials.
Appendix C: Various geometrical arrangements of distinct time-delay steps show a similar trend of Sout / Sin
Figure 6 shows the simulation results of the ratio between out-of-focus and in-focus signal, Sout / Sin, using various geometrical arrangements of distinct time-delay steps. As concluded in the main article, these Sout / Sin curves show that the decays of the out-of-focus excitation slow down significantly when Nt = 25–64, (Δt ≈ 2/3 τ0-1/4 τ0), although the curves from different patterns of time delays show slight differences in the values of Sout / Sin.
Appendix D: Simulation of the dynamic ESP in space reveals temporal focusing
In Media 1 (a static frame shown in Fig. 7), we show an example of the numerically simulated ESP. In particular, we plot the distributions of the real part of the electric field (Re(ESP), upper panel) and intensity (|ESP|2, lower panel). The geometrical arrangement of the distinct time-delay steps used for this simulation is a spiral pattern (Fig. 1(b) of main article) with Nt = 49 and dfoci ≈ 0.4 λ0. Media 1 shows that at most of the area, except for the in-focus region (z ≈ 0), the intensity is moderate during nearly the entire time course (100–700 fs). On the other hand, at in-focus region the intensity is high in a relatively short period of time (450–540 fs). Such a phenomenon is commonly recognized as temporal focusing .
Appendix E: Validity of the assumption of infinitely replicated unit HSMA in a wide-field microscope
In this section, we validate the model of infinitely replicated unit HSMAs for conventional wide-field biomedical microscopes. As described in the main article, the analysis of the properties of the proposed system can become simpler if we assume that the whole HSMA consists of infinitely, laterally repeated unit HSMAs. This is because the system is laterally periodic under such an assumption (referred to as the inf-HSMA model herein) and thus the physical properties (ex. the distribution of electric field) of one unit region can be used to describe the entire system. Although the infinite replication of unit HSMAs is not realistic for either experiments or numerical simulations, the inf-HSMA model is made tractable by assuming that, for the multiphoton excitation intensity ISP at arbitrary positions, the contributions from very distant foci are negligible. In other words, one can approximate the results of using the infinitely replicated unit HSMAs via consideration of only the contributions from the foci within a certain range. To justify such an approach, we need to examine if the computed ISP becomes stationary (i.e., converging to a certain value) as the range of foci considered to contribute, defined by rtot (Fig. 8(a)), is expanded.
To proceed, we compute time-integrated intensity I using Eq. (6) of the main article, with only the contributions to ESP from the foci within a distance rtot considered (Fig. 8(a)). Here we use Iend, the intensity at a given position as derived with a relatively large rtot (≈ 375 λ0, where λ0 is the central wavelength of the light pulse), as a reference value, and plot I/Iend as rtot increases from 0 to 125 λ0. Fig. 8(b) shows that values of I at all of the 100 randomly chosen positions converge to their corresponding Iend as rtot increases. In particular, we note that when rtot ≥ 62.5 λ0 (equivalent to 50 μm in physical dimensions), the error of I, defined as |I − Iend |/Iend, is less than 0.1%. Such results suggest that, for a given error tolerance in numerical simulations, we can use the inf-HSMA model by considering only the contributions of foci within a certain distance rinf. Here, we have rinf ≈ 50 μm for a 0.1% error tolerance. In the physical microscopy system, we can also use rinf to determine the region wherein the inf-HSMA assumption is valid, as shown in Fig. 8(c). For conventional biomedical microscopes using M = 60X objective lenses, the diameter of the full FOV is typically lager than 300 μm. Thus, the inf-HSMA model is valid in the central region of diameter larger than 200 μm. At the image plane, this region corresponds to a disk of diameter ∼ 12 mm (200 μm×M) or larger, which is able to cover most conventional imaging devices.
We gratefully acknowledge Dr. Alexander Egner of Laser-Laboratorium Gttingen e.V. for his kind advices on algorithm development. We are indebted to Mr. Yu-Hang Chen of National Taiwan University (NTU) for his assistance with the simulations, and we thank Dr. Chien-Cheng Chen of NTU for helpful comments. The support from Ellison Medical Foundation is acknowledged.
References and links
1. J. B. Pawley, Handbook of Biological Confocal Microscopy, 3rd ed. (Springer, 2006). [CrossRef]
3. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science 305, 1007–1009 (2004). [CrossRef]
5. T. Wilson, Confocal Microscopy (Academic Press, 1990).
6. M. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22, 1905–1907 (1997). [CrossRef]
7. B. Masters, P. So, C. Buehler, N. Barry, J. Sutin, W. Mantulin, and E. Gratton, “Mitigating thermal mechanical damage potential during two-photon dermal imaging,” J. Biomed. Opt. 9, 1265–1270 (2004). [CrossRef]
8. I. Akira, T. Takeo, I. Katsumi, S. Yumiko, K. Yasuhito, M. Kenta, A. Michio, and U. Isao, “High-speed confocal fluorescence microscopy using a nipkow scanner with microlenses for 3-d imaging of single fluorescent molecule in real time,” Bioimaging 4, 57–62 (1996-06).
9. J. Bewersdorf, R. Pick, and S. Hell, “Multifocal multiphoton microscopy,” Opt. Lett. 23, 655–657 (1998). [CrossRef]
10. V. Andresen, A. Egner, and S. Hell, “Time-multiplexed multifocal multiphoton microscope,” Opt. Lett. 26, 75–77 (2001). [CrossRef]
11. P. J. Keller, A. D. Schmidt, A. Santella, K. Khairy, Z. Bao, J. Wittbrodt, and E. H. K. Stelzer, “Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy,” Nat. Methods 7, 637–U55 (2010). [CrossRef]
12. J.-Y. Yu, C.-H. Kuo, D. B. Holland, Y. Chen, M. Ouyang, G. A. Blake, R. Zadoyan, and C.-L. Guo, “Wide-field optical sectioning for live-tissue imaging by plane-projection multiphoton microscopy,” J. Biomed. Opt. 16, 116009 (2011). [CrossRef]
13. C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumination,” Opt. Lett. 30, 3350–3352 (2005). [CrossRef]
14. A. Egner and S. Hell, “Time multiplexing and parallelization in multifocal multiphoton microscopy,” J. Opt. Soc. Am. A 17, 1192–1201 (2000). [CrossRef]
15. J. Jahns and K.-H. Brenner, Microoptics: from Technology to Applications, vol. v. 97 (Springer, 2004).
16. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light, 6th ed. (Cambridge University Press, 1997).
20. E. Hecht, Optics, 4th ed. (Addison-Wesley, 2002).