Analysis of super-resolution via 3D structured illumination intensity correlation microscopy

Anton Classen; Joachim von Zanthier; Girish S. Agarwal

doi:10.1364/OE.26.027492

1. Introduction

Since it was first shown that the resolution limit, posed by diffraction, can be overcome [1], a variety of superresolution microscopy methods have been developed. Yet, each technique comes with certain requirements and limitations, thus justifying an ongoing pursuit of novel methods. One group of methods relies on stimulated ground or excited state depletion and a non-linear fluorophore response to deterministically engineer the effective excitation point spread function (PSF) [1–3]. Other methods stochastically localize single photoswitchable molecules via centroid fitting of the PSF [4–7].

Another branch of methods makes use of intensity correlations that are evaluated from an image series [8–10]. For these intensity correlation microscopy (ICM) techniques, statistically blinking fluorophores [8] or quantum emitters that exhibit anti-bunching [9] can be used to enhance the resolution, both in widefield [9] or confocal microscopy [10], by shrinking the effective PSF by the factor $\sqrt{m}$ (with correlation order m), and thus leading to a resolution improvement of up to factor m when including deconvolution. Especially the first approach, known as superresolution optical fluctuation imaging (SOFI), is widely applied due to its combination of resolution improvement with low complexity of use [11–13]. Yet, in practice high correlation orders are not evaluated due to strong brightness skewing in the final image and long measurement times to obtain a reliable evaluation [11]. Together with the moderate scaling of factor m, ICM currently does not provide resolutions far below the diffraction limit.

In parallel, structured illumination microscopy (SIM) was developed, where by the use of spatial frequency mixing the resolution is doubled within the linear wave optics regime [14,15]. The non-linear derivative saturated SIM leads to an in principle unlimited resolution, though at the cost of necessitating high intensities [16, 17]. Other derivatives combine SIM with the third-order process of CARS [18,19], or with surface plasmons [20–22] to access higher spatial frequency information. 3D SIM doubles both, the lateral and axial resolution [23], while standing wave fluorescence microscopy techniques [24, 25] highly enhance the axial resolution via a dense axially structured illumination, but not the lateral one. Double-objective illumination and detection techniques [26, 27] with 3D-SIM attain the axial resolution of standing wave fluorescence microscopy and the lateral one of 2D SIM [28]. Today, the SIM toolbox is considered to be one of the most powerful and versatile superresolution techniques, due to its combination of resolution improvement with good acquisition speed and flexibility of use [29].

Recently, we showed that SIM and ICM based on antibunching can be combined to enhance the lateral resolution [30]. For correlation order m the enhancement scales favorably as m + m = 2m (when including deconvolution), which is a large improvement over the moderate factor m scaling of antibunching microscopy itself. A similar result for 2D SIM combined with SOFI was later derived by Zhao et al. [31], resulting in structured illumination SOFI. While the two approaches make use of different physical processes (antibunching or statistical fluctuations) their final signals take the same form such that we identify them as structured illumination intensity correlation microscopy (SI-ICM).

Here, we propose to use 3D structured illumination [23] in combination with ICM to equally enhance the axial resolution by the factor 2m. This is a crucial step resulting in full 3D superresolution capability of SI-ICM. We present the theory and illustrate the basic flow chart of the technique. We point out that ICM and SIM operate within the linear regime and are established techniques in the field of superresolution microscopy. Thus, SI-ICM bears the potential for full 3D deep-subwavelength resolution at low illumination levels.

2. Theory

Without loss of generality we assume R ≡ r for the coordinates in the object and image plane, respectively, i.e. a magnification of one. Let h(r) be the 3D PSF of a given widefield microscope, with r = (x, y, z). H(k) ≡ FT {h(r)} denotes the corresponding 3D optical transfer function (OTF) obtained by Fourier transform (FT) of h(r), where k = (k_x, k_y, k_z) denotes the spatial frequency in reciprocal space. The lateral and axial widths of the PSF determine the resolution power a microscope provides to discern individual close-by emitters. The lateral width is usually smaller than the axial one. Moreover the axial resolution can not properly be defined in widefield microscopy, which is due to the missing z-cone in Fourier space [see Fig. 1(a)] [32]. Optical sectioning capability in z-direction can however be retrieved by the measurement of z-stacks and deconvolution, using a pinhole as in confocal microscopy, or a variety of other means. Note that in ICM, the missing z-cone is intrinsically removed and thus true optical sectioning capability is provided already via the correlation analysis [9,33].

Fig. 1 Illustrations of total OTF supports of 3D-SIM (left column) and second-order 3D SI-ICM (right column). Image (a) depicts the OTF of a widefield microscope and (b) the 3D Gaussian H(k) as approximation. (c) shows the Fourier transform of the structured illumination of Eq. (10) with the center positions k_j = (k_ρ, k_z)_j (j = 1, . . ., 7) given in the main text (see the central blue and outer green dots). Combining (a) and (c) yields the images in (d), where the OTF support of widefield microscopy, 2D SIM for one single orientation α, 3D-SIM for one α and 3D-SIM for three orientations α = 0, $\frac{π}{3}$ , $\frac{2 π}{3}$ are shown. The final image of (d) provides a two-fold enlarged support along all axes. Image (e) shows the OTF H₂(k) of second-order ICM, which is enlarged by the factor 2 along all axes. Image (f) depicts the Fourier transform of the squared structured illumination, where the outer (red) dots represent the contributions from the first higher harmonics. Again, combining images (e) and (f) yields the OTF supports displayed in g), i.e., of second-order ICM, second-order ICM with 2D-SIM for a single α, second-order ICM with 3D SIM for a single α and second-order ICM with 3D-SIM for four orientations α = 0, $\frac{π}{4}$ , $\frac{2 π}{4}$ , $\frac{3 π}{4}$ . The total support for this case is already enhanced by the factor 4 along all axes.

Download Full Size | PDF

Here, as an approximation to the real widefield microscopy PSF, we consider a 3D Gaussian PSF of the form [8]

h (r) = exp [- \frac{x^{2} + y^{2}}{w_{ρ}^{2}} - \frac{z^{2}}{w_{z}^{2}}],

where w_ρ and w_z denote the lateral and axial width, and ρ = (x² + y²)^1/2. The OTF is then also a 3D Gaussian in reciprocal space [see Fig. 1(b)]. We point out that this is a useful assumption, since below the effective PSF will be the original PSF taken to the mth power h_m(r) ≡ h(r)^m and thus approaches a 3D Gaussian. The same is valid for the OTF H_m(k) ≡ FT {h_m(r)} = FT{h(r)} *· · · * FT{h(r)} = H(k) *· · · * H(k) which is the m-fold convolution of itself (denoted by ‘*’) and equally approaches a 3D Gaussian in Fourier space. We consider the ratio w_z/w_ρ = 3.0 to mimic a typical widefield microscopy PSF, where the resolvable distances along ρ and z between two close-by emitters are given by [34]

Δ ρ_{min} = 0.61 \frac{λ}{𝒜} Δ z_{min} = 2 \frac{n λ}{𝒜^{2}} .

In Eq. (2), 𝒜 is the numerical aperture, n the refractive index and λ the wavelength of the emitted fluorescence light. To simplify the illustration, we set the stokes shift to zero, resulting in equal wavelengths for excitation and emission, i.e. λ_ex = λ_em ≡ λ. Note though that λ_ex ≠ λ_em can easily be incorporated in the analysis.

The fluorophores are considered to be driven (far) below saturation resulting in a linear response to the (monochromatic) illumination intensity I_str(r). In widefield microscopy and ICM a plane-wave illumination leads to the flat excitation intensity I_str(r) = I₀. The model system to be imaged $n (r) \propto \sum_{i = 1}^{N} δ (r - r_{i})$ can be described by an ensemble of approximately point-like emitters at positions r_i. Considering the convolution by the PSF the (time-averaged) image of this ensemble, taken in the image plane, reads

〈 I (r, t) 〉 \equiv I (r) = h (r) * n (r) = I_{0} \sum_{i = 1}^{N} h (r - r_{i}) .

I₀ denotes the average emitter intensity, which here is assumed to be equal for each emitter. Note that the intensity I(r) ≡ G⁽¹⁾(r) can also be recognized as Glauber’s first-order equal-time intensity correlation function G⁽¹⁾(r, t; r, t) = 〈 Ê⁽⁻⁾(r, t) Ê⁽⁺⁾(r, t)〉 assuming an ergodic system [35]. Ê⁽⁺⁾ and Ê⁽⁻⁾ are the positive and negative frequency parts of the electric field operator [30].

Intensity correlation analysis can enhance the resolution, given that a certain process enables to discern and localize individual emitters within a sub-diffraction area or volume. For quantum emitters it is the intrinsic antibunching property that allows for an enhanced resolution [9], while for SOFI it is the independent and statistical blinking of fluorophores [8]. The resulting final signals are however of the same form with the PSF being taken to the mth power. Since the detailed derivations of ICM can be found elsewhere [8,9] we only provide a brief sketch here. First, consider the squared (measured) intensity of Eq. (3)

{[G^{(1)} (r)]}^{2} = I_{0}^{2} \sum_{i, j = 1}^{N} h (r - r_{i}) h (r - r_{j}) \equiv I_{0}^{2} \sum_{i = 1}^{N} h^{2} (r - r_{i}) + I_{0}^{2} \sum_{i \neq j}^{N} h (r - r_{i}) h (r - r_{j}) .

Simply squaring the intensity does not provide superresolution, but in the first sum of Eq. (4) the squared PSF arises. It is only the second sum with the (detrimental) cross terms which prevents the entire signal to be superresolving. To isolate the terms with the squared PSF (without application of any a priori knowledge) one can make use of the second-order intensity correlation function G⁽²⁾(r) ≡ G⁽²⁾(r, r) = 〈Ê⁽⁻⁾(r, t) Ê⁽⁻⁾(r, t)Ê⁽⁺⁾(r, t)Ê⁽⁺⁾(r, t)〉 [35]. The correlation functions for the two different approaches read [8,9,30]

\begin{array}{l} Antibunching : & G^{(2)} (r) \propto I_{0}^{2} \sum_{i \neq j}^{N} h (r - r_{i}) h (r - r_{j}) \\ SOFI : & G^{2} (r) \propto I_{0}^{2} \sum_{i, j = 1}^{N} h (r - r_{i}) h (r - r_{j}) + \sum_{i = 1}^{N} h^{2} (r - r_{i}) 〈 Δ I_{i} {(t)}^{2} 〉 \end{array}

For antibunching, each individual source (e.g fluorophores, quantum dots, etc) can emit at most one photon per excitation cycle such that only cross terms survive. In SOFI, the blinking fluorophores I_i(t) = I₀ + ΔI_i(t), with zero-mean fluctuations ΔI_i(t), lead to the excess countrates h²(r − r_i) 〈ΔI_i(t)²〉 compared to [G⁽¹⁾(r)]². A convenient subtraction of the terms in Eqs. (4) and (5) results in the sought-after (second-order) ICM signals [8,9,30]

\begin{array}{l} Antibunching : & {ICM}_{2} (r) = {[G^{(1)} (r)]}^{2} - G^{(2)} (r) = I_{0}^{2} \sum_{i = 1}^{N} h^{2} (r - r_{i}), \\ SOFI : & {ICM}_{2} (r) = G^{2} (r) - {[G^{(1)} (r)]}^{2} = \bar{{(Δ I)}^{2}} \sum_{i = 1}^{N} h^{2} (r - r_{i}) . \end{array}

where we considered

\bar{{(Δ I)}^{2}} = 〈 Δ I_{i} {(t)}^{2} 〉

for each emitter. The squared PSF h²(r) directly leads to a resolution enhancement of factor

\sqrt{2}

. Though, the OTF H₂(k) = H(k) * H(k) is effectively twice as large. Hence, by rescaling the strongly suppressed Fourier amplitudes in the outer rims of the support the resolution can be doubled [depending on the signal-to-noise ratio (SNR)]. This process, known as deconvolution, is achieved trough application of a Wiener filter [see Eq. (12) below]. Higher order ICM_m(r) signals are evaluated in a similar manner as a combination of all correlation orders up to m and read [8,9,30]

{ICM}_{m} (r) = \sum_{i = 1}^{N} h_{m} (r - r_{i}),

where we set

I_{0}^{m} \equiv 1

and

\bar{{(Δ I)}^{m}} \equiv 1

for all orders m to simplify the illustration. Note that the required combinations for SOFI are equivalent to so-called cumulants [8], and are different from the ones for antibunching microscopy which can be found in [9]. Even though high correlation orders can in principle be evaluated, the moderate scaling with m prevents resolutions far below the diffraction limit. In addition different molecular brightnesses and/or blinking ratios 〈ΔI_i(t)〉 become more pronounced with rising order m, what skews the final image [11]. While this problem is mitigated by use of balanced cumulants [36], in practice still only low correlation orders are utilized.

A different and independent approach to enhance the resolution of optical microscopy is SIM. Only recently it was realized that SIM and ICM can fruitfully be combined to enhance the lateral resolution [30,31]. The goal of this manuscript is to show that the same holds true for the axial resolution and thus full 3D superresolution is possible via 3D SI-ICM. Towards this, we review 2D [14,15] and especially 3D SIM [23] in detail, since this knowledge will be particularly helpful to understand the combination of SIM and ICM in Eq. (13) below.

In 2D SIM two coherent plane waves are superposed at an angle which e.g. stem from the ±1 diffraction orders of a grating. The electric field distribution reads E = e^{i(k_xx+k_yy)+ik_zz} + e^{−i(k_xx+k_yy)+ik_zz} resulting in the intensity pattern I_str(r) = |E|² = 1 + cos[2(k_xx + k_yy)]. We denote the orientation of the pattern by α = tan⁻¹(k_y/k_x). Moving the grating laterally (along α) acts as opposite lateral phase shifts ±φ_r on the two beams and thus leads to the pattern $I_{str} (r) = I_{0} [\frac{1}{2} + \frac{1}{2} cos (k_{0} r + 2 φ_{r})]$ , with $| k_{0} | \equiv 2 {(k_{x}^{2} + k_{y}^{2})}^{1 / 2}$ . Due to the illumination the individual emitter intensities are scaled by I_str(r_i), what results in measured images of the form [15]

SIM (r) = h (r) * [n (r) \times I_{str} (r, α, φ_{r})] = \sum_{i = 1}^{N} h (r - r_{i}) \times I_{str} (r_{i}, α, φ_{r}) .

The multiplication n(r) × I_str(r, α, φ) in real space corresponds to a mixing of the object’s spatial frequencies k with the spatial frequency k₀ in Fourier space. Hence, initially unobservable spatial frequencies k > k_max are encoded in the microscope’s OTF support. Taking a set of linearly independent images and applying computational post-processing allows for the retrieval of this information. The required procedure will be outlined below for 3D-SIM. The resolution enhancement reads (k₀ + k_max)/k_max and reaches 2 for the diffraction limit k₀ = k_max.

In 3D SIM the 0^th-order beam e^ikz of the diffraction grating is added [23]. In addition to the lateral phase shifts ±φ_r, we consider an axial phase shift φ_z on the central beam, introduced e.g. by an optical element placed on the optical axis. The electric field thus reads

E (x, y, z) = e^{i (k_{x} x + k_{y} y - φ_{r}) + i k_{z} z} + e^{i (k z + φ_{z})} + e^{- i (k_{x} x + k_{y} y - φ_{r}) + i k_{z} z},

and the 3D intensity pattern I_str(r) = |E(x, y, z)|² calculates to

I_{str} (r) = 3 + 2 cos [2 (k_{x} x + k_{y} y) + 2 φ_{r}] + 4 cos [(k_{x} x + k_{y} y) + φ_{r}] cos [(k - k_{z}) z + φ_{z}],

This pattern contains seven spatial frequency components. Taking the Fourier transform

FT {I (r)} = I_{0} \sum_{j = 1}^{7} e^{i φ_{j}} δ (k - k_{j})

yields seven delta peaks in Fourier space with phases φ_j (which are combinations of φ_r and φ_z). For ±1 diffraction orders propagating at the angles ±60° the positions read k_j = (k_ρ, k_z)_j = (0, 0)k, (

\sqrt{3} / 2

, 1/2)k, (

\sqrt{3} / 2

, −1/2)k, (

- \sqrt{3} / 2

, 1/2)k, (

- \sqrt{3} / 2

, −1/2)k, (

\sqrt{3}

, 0)k, (

- \sqrt{3}

, 0)k, with

k_{ρ} = {(k_{x}^{2} + k_{y}^{2})}^{1 / 2}

. These are shown in Fig. 1(c), where the axes are normalized to the wavenumber k = 2π/λ. Using this 3D illumination, taking the Fourier transform of Eq. (8) and utilizing convolution theorems yields [23]

FT {SIM (r)} = H (k) \times \sum_{j = 1}^{7} c_{j} e^{i φ_{j}} \tilde{n} (k - k_{j}),

where the c_j represent weights [see prefactors of cosines in Eq. (10)]. Note that ñ(k) = FT{n(r)} contains the sought-after spatial frequency information of the unknown object under investigation. In a single image all seven components are superposed. To disentangle them at least seven independent 3D images (by measurement of z-stacks) are required. This is established by varying the phases φ_r and φ_z and creates the linear system An = G, where the elements of the 7×7 matrix A are given by the phase terms e^iφ_j of Eq. (11) and n denotes a vector with entries ñ(k − k_j). The vector G possesses Eq. (11) as entries and the system is solved by n = A⁻¹G.

In practice a convenient choice of exactly seven independent images is not readily achieved [23]. As a result the components in Fig. 1(c) are first disentangled along the lateral direction by taking five linearly independent images with phases φ_r = 0, $\frac{2 π}{5}$ , $\frac{4 π}{5}$ , $\frac{6 π}{5}$ , $\frac{8 π}{5}$ for a fixed φ_z = 0. The application of a 5× 5 matrix $A_{r}^{- 1}$ yields the intermediate components:

ñ₁(k) = ñ(k − [k, k]), ñ₄(k) = ñ(k − [ $- \sqrt{3} k$ , 0]), ñ₅(k) = ñ(k − [ $\sqrt{3} k$ , 0])
ñ₂(k) = ñ(k − [ $- \frac{\sqrt{3} k}{2}$ , $- \frac{k}{2}$ ]) + ñ(k − [ $\frac{\sqrt{3} k}{2}$ , $- \frac{k}{2}$ ])
ñ₃(k) = ñ(k − [ $- \frac{\sqrt{3} k}{2}$ , $\frac{k}{2}$ ]) + ñ(k − [ $\frac{\sqrt{3} k}{2}$ , $\frac{k}{2}$ ]),

where ñ₁(k), ñ₄(k) and ñ₅(k) are already fully isolated, and ñ₂(k) and ñ₃(k) contain two components each. To disentangle them an additional measurement series (with 5 different φ_r) for

φ_{z} = \frac{2 π}{3}

, and application of a 2 × 2 matrix

A_{z}^{- 1}

, is required. In summary the flow chart requires 2 × 5 = 10 images. Note that, alternatively to varying the axial phase φ_z one can regard the axial modulation to act on the PSF and keep it fixed with respect to the objective coordinates axes [23]. Correspondingly the axial OTF support is enhanced two-fold and requires a two-fold finer sampling of focal planes to maintain the Nyquist sampling rate. The required amount of images is hence the same for both approaches.

To sufficiently cover the enlarged OTF support the procedure needs to be repeated for three orientations $α = {tan}^{- 1} (k_{y} / k_{x}) = 0, \frac{1 π}{3}, \frac{2 π}{3}$ [see the final image in Fig. 1(d)]. In the next step the disentangled components need to be shifted back to their true positions in Fourier space, post-processed appropriately and merged into a large homogenous support. This is achieved trough application of the formula [23]

{\tilde{n}}_{new} (k) = \frac{\sum_{j} {\tilde{n}}_{j} (k + k_{j})}{[\sum_{j} H (k + k_{j})] + γ} A (k),

where γ is a constant that prevents division by zero and should be chosen noise-dependently. A(k) is a triangular apodization function in 3D and serves the purpose of reducing ringing in the final image, which is obtained via inverse Fourier transformation of Eq. (12). The deconvolution in Eq. (12) also enhances the suppressed spatial frequency information from the outer rims of the OTF support and thus leads to the highest possible resolution for the given data. The straightforward assembly in Fourier space requires near-integer numbers k_j = (k_x, k_y, k_z)_j [15,23] such that the assembly is typically conducted in real space by first applying the inverse Fourier transform to each component and then multiplying by the complex wave e^ik_jr. Note that while in 3D-SIM the resolution is doubled along all axes, it is still limited by diffraction.

SI-ICM fruitfully combines SIM and ICM. That is, the structured illumination encodes information from outside the original OTF support via spatial frequency mixing and and the correlation analysis effectively raises all signals to the m th power. The schematic setup and the experimental flowchart are shown in Fig. 2. A laser illuminates a diffraction grating which produces the diffraction orders −1, 0, +1. After collimation by the lens L the three beams are coupled into the back focal of a microscope objective (MO) and form the three dimensional structured illumination (3D-SI). Rotation and translation of the grating varies the orientation α and the lateral phase φ_r, respectively. An additional optical element on the optical axis varies the axial phase φ_z. The fluorescence emission from the fluorophores is captured by the same MO and guided toward an CCD camera, which captures an image series for each I_str(r_i, α, φ_j). The basic flow chart in the experiment would be: i) set a specific value set (φ_r, φ_z, α) for the 3D-SI, ii) take a 2D image series and evaluate it according to ICM algorithms [cf. Eq. (6)], iii) repeat the second step for varying φ_r, iv) repeat steps two and three for different focal planes to obtain a z-stack, also with a sufficient number of values φ_z per focal plane, v) repeat steps two to four for the next pattern orientation α, and vi) apply a SIM reconstruction algorithm to the set of 3D ICM images with different illumination pattern values (φ_r, φ_z, α).

Fig. 2 Schematic setup of an SI-ICM experiment (left side) and the corresponding flowchart to obtain the sought-after superresolving images (right side). For details see text.

Download Full Size | PDF

In mathematical terms the outlined procedure corresponds to a combination of Eqs. (7) and (8) which results in

{SI-ICM}_{m} (r) = \sum_{i = 1}^{N} h_{m} (r - r_{i}) \times I_{str} {(r_{i}, α, φ_{j})}^{m} .

Now, higher harmonics up to cos (mk₀r) arise and the individual OTF H_m(k) is enlarged by the factor m (when including deconvolution). For 2D SIM combined with a correlation analysis the lateral resolution was shown to be enhanced by up to m + m = 2m [30,31].

To illustrate the outcome of Eq. (13) we consider the 3D structured illumination of Eq. (10) and second-order SI-ICM (m = 2). Taking the Fourier transform of Eq. (13) thus yields

FT {{SI-ICM}_{2} (r)} = H_{2} (k) \times \sum_{j = 1}^{19} c_{j} e^{i φ_{j}} \tilde{n} (k - k_{j}),

where 19 spatial frequency components arise, which can readily be calculated by executing I_str(r)² = (|E(x, y, z)|²)². Higher harmonics now arise along the lateral and axial direction, as illustrated in Fig. 1(f) by the added most outer (red) dots. Moreover, the OTF is enlarged by the factor 2 [see Fig. 1(e)]. The total improvement reaches up to 2 + 2 = 4 along all axes [see the 3D volume in Fig. 1(g)]. The larger OTF H₂(k) also reduces the need for many orientations α as compared to saturated SIM [17] so four α = 0,

\frac{1 π}{4}

,

\frac{2 π}{4}

,

\frac{3 π}{4}

are sufficient [30].

In principle 19 independent 3D images would suffice, yet a convenient choice is not readily achieved. Hence, as above, in the first step nine images with different lateral phases $φ_{r} = \frac{2 π}{9} j$ (j = 0, . . ., 8) need to be acquired to disentangle the components along the lateral direction. By this, only the lateral components with center positions ( $2 \sqrt{3}$ , 0)k, ( $- 2 \sqrt{3}$ , 0)k [see Fig. 1(f)] are isolated. The remaining contributions are still composed of two or three individual axial components. Hence the first step needs to be repeated for three axial phases φ_z = 0, $\frac{2 π}{3}$ , $\frac{4 π}{3}$ , resulting in 3 × 9 = 27 measurements per orientation α. Including the four orientations α a total of 108 images is obtained.

3. Simulation

To illustrate the mathematical description of section 2, we performed a basic simulation of the formulas in Eqs. (3), (7), (8) and (13), where we utilized the 3D PSF of Eq. (1) [see Fig. 3(b)]. In case of Eq. (7) we consider the direct result with a resolution enhancement of $\sqrt{m}$ and after deconvolution via a Wiener filter and triangular apodization, resulting in an enhancement of up to factor m [see e.g. Figs. 3(d) and 3(e) for second-order correlation and Fig. 3(h) for fourth-order correlation]. Since we are interested in the resolution power of ICM, SIM and SI-ICM relative to widefield microscopy we use dimensionless units and merely normalize r by the Rayleigh limit Δρ_min of Eq. (2).

Fig. 3 The figure shows (a) an object consisting of three emitters at positions r₁ = (−0.16, 0.16, 0.05), r₂ = (0.26, −0.26, 0.57) and r₃ = (0.26, −0.26, 0.68) (in units of Δρ_min) and (b) the 3D PSF of Eq. (1) utilized in the simulation. The images (c) – (i) are obtained by the methods (c) widefield microscopy, (d) second-order ICM, (e) second-order ICM + Deconvolution, (f) 3D-SIM, (g) 16^th-order ICM, (h) fourth-order ICM + Deconvolution, and (i) second-order 3D-SI-ICM. For details on the simulation see text.

Download Full Size | PDF

We chose a setup with three close-by emitters distributed in the 3D object space within a sub-diffraction limited volume [see Fig. 3(a)]. The coordinates r₁ = (−0.16, 0.16, 0.05), r₂ = (0.26, −0.26, 0.57) and r₃ = (0.26, −0.26, −0.68) with pair-wise lateral or axial separations were chosen to demonstrate the full 3D resolution capabilities of SI-ICM. For illustration purposes we used theoretical data without noise. A discussion of possible practical limitations and requirements is given below. The 3D data was calculated with respect to the 3D PSF of Eq. (1). In the experiment it would be obtained through the flow chart outlined in section 2. For the deconvolution post-processing we always chose γ = 0.001 for the methods ICM, SIM and SI-ICM. As outlined above, for SIM five lateral phases φ_r, two axial phases φ_z and three orientations α are required, resulting in a total of 30 images.For SI-ICM₂ the required phases are nine φ_r, three φ_z and for four α, yielding a total of 108 images.

Figure 3 shows the simulations results for seven different signals. These are (c) widefield microscopy, (d) second-order ICM, (e) second-order ICM + Deconvolution, (f) 3D-SIM, (g) 16^th-order ICM, (h) fourth-order ICM + Deconvolution, and (i) second-order 3D-SI-ICM. The image in (g) was chosen for comparison purposes, since it provides a four-fold resolution enhancement over regular widefield microscopy due to the PSF being taken to the 16th power. The simulation results are in good agreement with the theory and it can be seen that 3D SI-ICM equally enhances the lateral and axial resolution and thus provides full 3D superresolution. Further it can be seen that second-order SI-ICM achieves the same resolution enhancement as fourth-order ICM + Deconvolution, what can be a major advantage since the evaluation of high correlation orders is challenging.

For widefield SI-ICM in 2D and 3D the question is whether the approach will show superior performance in experiments compared to the already existing techniques. The final answer can of course only be given by future experiments, but an estimation of the expected requirements might guide the way. Toward this we compare the required amount of frames for SI-ICM₂ and ICM₄, for the SOFI variant with blinking fluorophores. First, for 3D SI-ICM₂ we showed that 108 images series are required per focal plane. For ICM₄ a single image series (however with more frames) is sufficient for a single transverse plane. To account for the increased axial resolution the sampling of focal planes is required to be four times as dense to fulfill the Nyquist rate. For imaging a 3D volume the difference in image series results in the factor 27.

Yet, the required amount of frames per series differs significantly with rising correlation order [37,38]. Further it strongly depends on the imaging scenario, that is the blinking dynamics and the respective on-time to off-time ratio. The emitter density within a diffraction limited volume also strongly affects the convergence of the (higher-order) cumulants toward their theoretical values [38]. Generally the second-order cumulant converges within 100–500 frames (depending on the specific scenario). The fourth-order cumulant requires around ∼ 15 times more frames in the best case to converge, but this value quickly diverges, for example, when the emitter density increases. The second-order cumulant, by contrast, quickly approaches an asymptotic limit and can be used with very high density samples [38]. An additional major advantage is the smaller brightness skewing of the second-order cumulant compared to higher orders.

Once a 3D ICM image stack with different illumination patterns I_str(r_i, α, φ_j) is obtained with a certain signal to noise ratio a SIM reconstruction algorithm needs to reconstruct the final 3D SI-ICM image. Hereby the question arises how the performance and resolution of this algorithm scales with a given signal to noise ratio. Since this question has been answered elsewhere, we refer the reader to [30,31] or other more detailed studies [39].

Finally, we point out that a first proof-of-principle experiment that relies on the SI-ICM principle, while in a confocal microscopy setting, was recently demonstrated by Tenne et al. [40]. In the paper the authors combine image scanning microscopy (which can be regarded as a confocal SIM variant [29]) with the evaluation of quantum correlations by making use of antibunching of individual quantum dots. While their setting is different from the widefield microscopy setup discussed here it delivers a first cornerstone towards real applications.

Regarding the generally high amount of frames, we point out that very fast cameras exist, which possess frame rates as high as a few kHz. Even modern EM-CCD cameras such as the Andor iXon 897 achieve these frame rates using the cropped mode. Considering a total amount of frames 200 × 108 × 10 ≈ 2 × 10⁵, the measurement time would equal a few tens of seconds for a full 3D superresolution image with 10 focal planes. With improving detector technology this value can be foreseen to be significantly smaller in the future.

4. Conclusion

In this paper we proposed to enhance the resolution power of ICM through the addition of 3D structured illumination. We presented a mathematical treatment that predicts a resolution enhancement of m + m = 2m through SI-ICM with correlation order m, both along the lateral and axial direction. Compared to the enhancement factor m of ICM alone this is a major boost that allows to reach deep-subwavelength resolution already with much lower correlation orders. Further, we outlined the flow chart for an experiment and illustrated the results via basic simulations that matched the theoretical predictions. Moreover we pointed out that SIM in combination with second-order ICM requires an comparable amount of images as fourth-order ICM itself and and would outperforms each method on its own. We note that since SI-ICM fully operates within the linear regime it bears the potential to increase the resolution in particular for imaging biological specimen at low illumination levels, i.e., especially in cases where other methods can not be utilized.

Further enhancements of the axial resolution can be achieved by combining SI-ICM with the double-objective 3D-SIM technique known as I⁵S [28]. The three added coherent beams from the second objective lead to very fast modulations along the axial direction. Additionally, the OTF is enlarged along the axial direction as in 4Pi- and I⁵-microscopy [26,27]. Again, SI-ICM would square the PSF and the excitation pattern.

Another promising future route may be to combine SI-ICM with plasmonic SIM techniques [20–22]. Even though these techniques are limited to 2D, they allow for spatial frequencies k₀ > k_max of the standing wave pattern. In linear plasmonic SIM k₀ = 2k_max should not be exceeded to prevent gaps in the OTF support coverage. The resolution enhancement is thus limited to the factor 3 [21,22]. SI-ICM, however, would highly benefit from spatial frequencies k₀ > 2k_max since the enlarged OTF H_m(k) prevents an early formation of gaps and the higher harmonics cos (mk₀r) reach out to very high spatial frequencies.

Funding

A.C. and J.v.Z: Erlangen Graduate School in Advanced Optical Technologies (SAOT) ; G.S.A.: Welch Foundation (Award number: A-1943-20180324).

Acknowledgments

A.C. gratefully acknowledges the hospitality of Texas A&M University where parts of this work were done.

References

1. S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19, 780–782 (1994). [CrossRef] [PubMed]

2. S. W. Hell and M. Kroug, “Ground-state-depletion fluorscence microscopy: A concept for breaking the diffraction resolution limit,” Appl. Phys. B 60, 495–497 (1995). [CrossRef]

3. M. Hofmann, C. Eggeling, S. Jakobs, and S. W. Hell, “Breaking the diffraction barrier in fluorescence microscopy at low light intensities by using reversibly photoswitchable proteins,” Proc. Natl. Acad. Sci. USA 102, 17565–17569 (2005). [CrossRef] [PubMed]

4. R. M. Dickson, A. B. Cubitt, R. Y. Tsien, and W. E. Moerner, “On/off blinking and switching behaviour of single molecules of green fluorescent protein,” Nature 388, 355–358 (1997). [CrossRef] [PubMed]

5. S. T. Hess, T. P. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation,” Biophys. J. 91, 4258–4272 (2006). [CrossRef] [PubMed]

6. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Meth. 3, 793–796 (2006). [CrossRef]

7. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313, 1642–1645 (2006). [CrossRef] [PubMed]

8. T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. USA 106, 22287–22292 (2009). [CrossRef] [PubMed]

9. O. Schwartz, J. M. Levitt, R. Tenne, S. Itzhakov, Z. Deutsch, and D. Oron, “Superresolution microscopy with quantum emitters,” Nano Lett. 13, 5832–5836 (2013). [CrossRef] [PubMed]

10. D. Gatto Monticone, K. Katamadze, P. Traina, E. Moreva, J. Forneris, I. Ruo-Berchera, P. Olivero, I. P. Degiovanni, G. Brida, and M. Genovese, “Beating the abbe diffraction limit in confocal microscopy via nonclassical photon statistics,” Phys. Rev. Lett. 113, 143602 (2014). [CrossRef] [PubMed]

11. S. C. Stein, A. Huss, I. Gregor, and J. Enderlein, Super-Resolution Imaging in Biomedicine (CRC Press, 2016).

12. X. Zhang, X. Chen, Z. Zeng, M. Zhang, Y. Sun, P. Xi, J. Peng, and P. Xu, “Development of a reversibly switchable fluorescent protein for super-resolution optical fluctuation imaging (SOFI),” ACS Nano 9, 2659–2667 (2015). [CrossRef] [PubMed]

13. P. Dedecker, G. C. H. Mo, T. Dertinger, and J. Zhang, “Widely accessible method for superresolution fluorescence imaging of living systems,” Proc. Natl. Acad. Sci USA 109, 10909–10914 (2012). [CrossRef] [PubMed]

14. R. Heintzmann and C. G. Cremer, “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating,” Proc. SPIE 3568, 185–196 (1999). [CrossRef]

15. M. G. L. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198, 82–87 (2000). [CrossRef] [PubMed]

16. R. Heintzmann, T. M. Jovin, and C. Cremer, “Saturated patterned excitation microscopy—a concept for optical resolution improvement,” J. Opt. Soc. Am. A 19, 1599–1609 (2002). [CrossRef]

17. M. G. L. Gustafsson, “Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution,” Proc. Natl. Acad. Sci. USA 102, 13081–13086 (2005). [CrossRef] [PubMed]

18. K. M. Hajek, B. Littleton, D. Turk, T. J. McIntyre, and H. Rubinsztein-Dunlop, “A method for achieving super-resolved widefield CARS microscopy,” Opt. Express 18, 19263–19272 (2010). [CrossRef] [PubMed]

19. J. H. Park, S.-W. Lee, E. S. Lee, and J. Y. Lee, “A method for super-resolved CARS microscopy with structured illumination in two dimensions,” Opt. Express 22, 9854–9870 (2014). [CrossRef] [PubMed]

20. X. Zeng, M. Al-Amri, and M. S. Zubairy, “Nanometer-scale microscopy via graphene plasmons,” Phys. Rev. B 90, 235418 (2014). [CrossRef]

21. F. Wei, D. Lu, H. Shen, W. Wan, J. L. Ponsetto, E. Huang, and Z. Liu, “Wide field super-resolution surface imaging through plasmonic structured illumination microscopy,” Nano Lett. 14, 4634–4639 (2014). [CrossRef] [PubMed]

22. J. L. Ponsetto, A. Bezryadina, F. Wei, K. Onishi, H. Shen, E. Huang, L. Ferrari, Q. Ma, Y. Zou, and Z. Liu, “Experimental demonstration of localized plasmonic structured illumination microscopy,” ACS Nano 11, 5344–5350 (2017). [CrossRef] [PubMed]

23. M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J. 94, 4957–4970 (2008). [CrossRef] [PubMed]

24. B. Bailey, D. L. Farkas, D. L. Taylor, and F. Lanni, “Enhancement of axial resolution in fluorescence microscopy by standing-wave excitation,” Nature 366, 44–48 (1993). [CrossRef] [PubMed]

25. R. Freimann, S. Pentz, and H. Hörler, “Development of a standing-wave fluorescence microscope with high nodal plane flatness,” J. Microsc. 187, 193–200 (1997). [CrossRef] [PubMed]

26. S. W. Hell, E. H. K. Stelzer, S. Lindek, and C. Cremer, “Confocal microscopy with an increased detection aperture: type-b 4pi confocal microscopy,” Opt. Lett. 19, 222–224 (1994). [CrossRef] [PubMed]

27. M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “I5M: 3D widefield light microscopy with better than 100 nm axial resolution,” J. Microsc.195, 10–16. [PubMed]

28. L. Shao, B. Isaac, S. Uzawa, D. A. Agard, J. W. Sedat, and M. G. L. Gustafsson, “I5S: Wide-field light microscopy with 100-nm-scale resolution in three dimensions,” Biophys. J. 94, 4971–4983 (2008). [CrossRef] [PubMed]

29. F. Ströhl and C. F. Kaminski, “Frontiers in structured illumination microscopy,” Optica 3, 667–677 (2016). [CrossRef]

30. A. Classen, J. von Zanthier, M. O. Scully, and G. S. Agarwal, “Superresolution via structured illumination quantum correlation microscopy,” Optica 4, 580–587 (2017). [CrossRef]

31. G. Zhao, C. Zheng, C. Kuang, and X. Liu, “Resolution-enhanced SOFI via structured illumination,” Opt. Lett. 42, 3956–3959 (2017). [CrossRef] [PubMed]

32. M. Born and E. Wolf, Principles of Optics (Cambridge University, 1999). [CrossRef]

33. T. Dertinger, J. Xu, O. F. Naini, R. Vogel, and S. Weiss, “SOFI-based 3D superresolution sectioning with a widefield microscope,” Opt. Nanoscopy 1, 2 (2012). [CrossRef]

34. L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge University, 2006). [CrossRef]

35. R. J. Glauber, “The quantum theory of optical coherence,” Phys. Rev. 130, 2529–2539 (1963). [CrossRef]

36. S. Geissbuehler, N. L. Bocchio, C. Dellagiacoma, C. Berclaz, M. Leutenegger, and T. Lasser, “Mapping molecular statistics with balanced super-resolution optical fluctuation imaging (bSOFI),” Opt. Nanoscopy 1, 4 (2012). [CrossRef]

37. S. Geissbuehler, C. Dellagiacoma, and T. Lasser, “Comparison between SOFI and STORM,” Biomed. Opt. Express 2, 408–420 (2011). [CrossRef] [PubMed]

38. S. C. Stein, “Advanced data processing in super-resolution microscopy,” Ph.D. thesis, Georg-August-UniversitätGöttingen (2017).

39. E. A. Ingerman, R. A. London, R. Heintzmann, and M. G. L. Gustafsson, “Signal, noise and resolution in linear and nonlinear structured-illumination microscopy,” J. Microsc. 0, 1–23 (2018).

40. R. Tenne, U. Rossman, B. Rephael, Y. Israel, A. Krupinski-Ptaszek, R. Lapkiewicz, Y. Silberberg, and D. Oron, “Super-resolution enhancement by quantum image scanning microscopy,” arXiv:1806.07661 (2018).

Analysis of super-resolution via 3D structured illumination intensity correlation microscopy

Abstract

1. Introduction

2. Theory

3. Simulation

4. Conclusion

Funding

Acknowledgments

References

Cited By

Figures (3)

Equations (14)

Optics Express