Point spread function (PSF) engineering is used in single emitter localization to measure the emitter position in 3D and possibly other parameters such as the emission color or dipole orientation as well. Advanced PSF models such as spline fits to experimental PSFs or the vectorial PSF model can be used in the corresponding localization algorithms in order to model the intricate spot shape and deformations correctly. The complexity of the optical architecture and fit model makes PSF engineering approaches particularly sensitive to optical aberrations. Here, we present a calibration and alignment protocol for fluorescence microscopes equipped with a spatial light modulator (SLM) with the goal of establishing a wavefront error well below the diffraction limit for optimum application of complex engineered PSFs. We achieve high-precision wavefront control, to a level below 20 mλ wavefront aberration over a 30 minute time window after the calibration procedure, using a separate light path for calibrating the pixel-to-pixel variations of the SLM, and alignment of the SLM with respect to the optical axis and Fourier plane within 3 μm (x/y) and 100 μm (z) error. Aberrations are retrieved from a fit of the vectorial PSF model to a bead z-stack and compensated with a residual wavefront error comparable to the error of the SLM calibration step. This well-calibrated and corrected setup makes it possible to create complex ‘3D+λ’ PSFs that fit very well to the vectorial PSF model. Proof-of-principle bead experiments show precisions below 10 nm in x, y, and λ, and below 20 nm in z over an axial range of 1 μm with 2000 signal photons and 12 background photons.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
The diffraction limit to resolution is overcome in Single Molecule Localization Microscopy (SMLM) by estimating the location of individual molecules from sparsely distributed emission spots across the field of view of the camera [1–4]. The achievable resolution is limited by the localization precision and the density of fluorescent labels and can be on the order of several tens of nanometers in practice . Several groups have extended this technique to 3D localization. Estimation of the axial position of the molecules is made possible by adding a cylindrical lens to the optical path [6–8] or by using bi-plane or multi-focus imaging [9–11]. The elongation of the spot (astigmatism) or the difference in spot size (bi-plane imaging) encodes for the axial position. The addition of a Spatial Light Modulator (SLM) incorporated with a 4F relay system to the imaging light path enables the design of a broader class of Point Spread Functions (PSFs), opening up the possibility to optimize the localization performance of the microscope . Notable proposals in the literature are single and double helix PSFs derived from Gauss-Laguerre modes [13–16], or made with annular zones with increasing helical charge [17,18], saddle point or tetrapod designs , which use higher orders of astigmatism in addition to primary astigmatism, phase-ramps  and self-bending beams . Interferometric imaging can also be used to establish high-performance axial localization, but requires a complex 4π optical setup .
Other properties of the fluorescent molecules could be of interest next to the 3D-position, such as the emitter dipole orientation  and the wavelength of the emitted light. The latter is relevant for imaging multiple protein species in a specimen that are labeled with fluorophores with different (excitation and) emission spectra. Encoding the emission color into the PSF shape enables multi-color imaging with a single imaging light path and a single camera, operating at the full field of view. This has been proposed by our group in  for 2D-localization and generalized in  for 3D-localization. An alternative approach to ‘3D+λ’ localization has been described in  by Shechtman et al.. The key idea of [24,25] is to have the SLM function as a Diffractive Optical Element (DOE), which splits the emission spot into two or three sub-spots. These sub-spots correspond to the diffraction orders generated by the DOE, and the distance between the spots and their relative intensity provide information on the emission wavelength of the fluorophore. The shape of the repetitive zones of the DOE can be designed for making the shape of the sub-spots change with the axial position of the fluorophore, thereby enabling 3D-localization.
A common characteristic of all engineered PSFs is their complexity compared to the simple 2D focused spots, which must be represented in the PSF model that is used in the parameter fitting algorithm for estimating the 3D-position (and possibly the emission color or molecular orientation). Simplified PSF models such as the Gaussian model , the scalar diffraction based Airy model, the Gibson-Lanni model , or effective models based on Hermite functions  cannot meet this requirement. A solution is the use of an experimental reference PSF, or a spline fit of such a PSF as model PSF [30–33], or the use of one or multiple Look Up Tables (LUTs) to estimate the z-position . We have shown previously that a vectorial PSF model can also be used for complex 3D and 3D+λ engineered PSFs . It is known that the vectorial PSF model is the physically correct model for image formation in high-NA fluorescence imaging systems. Another common characteristic of complex engineered PSFs is the sensitivity to aberrations that perturb the designed PSF shape and in this way negatively affect precision and accuracy. In order to achieve precisions down to the Cramér-Rao Lower Bound (CRLB), the best possible precision for an unbiased estimator, the aberration level of the optical system should be controlled to well within the diffraction limit (0.072λ root mean square wavefront aberration) , a condition which is often not met in practice. Correction of aberrations using a deformable mirror or with the SLM that is present anyway for producing the engineered PSF is therefore required. The control parameters of the adaptive optics component can be set using image based metrics [35–37] or via measurement of the to-be corrected aberrations. The latter may be done via phase retrieval algorithms based on the introduction of phase diversity, often in the form of a through-focus bead scan. This has been implemented in high numerical aperture microscope system , in localization microscopy  and used to improve the quality of a STED laser focus . These algorithms rely on smoothing and apply constraints to suppress noise effects. The noise statistics however can be incorporated into a Maximum Likelihood Estimation (MLE) based phase retrieval algorithm, which makes it possible to assess the optimality of the retrieval process by means of a CRLB analysis .
Here, we present a calibration and alignment protocol for fluorescence microscopes equipped with an SLM with the goal of establishing a wavefront error well below the diffraction limit, which is needed for an optimum application of complex engineered PSFs in single emitter localization. This high-precision wavefront control is combined with the use of the vectorial PSF model, both in the MLE-based phase retrieval algorithm for estimating the aberrations, and in the localization algorithm for estimating the emitter position in 3D, signal photon count, and background photon level (and possibly emission color). We report on proof-of-principle experiments on fluorescent beads for different 3D+λ engineered PSFs for testing whether the proposed calibration protocol and fitting with a vectorial PSF model give rise to precisions matching the CRLB.
The outline of this paper is as follows. In section 2 the vectorial fitting routine and aberration retrieval is described, section 3 describes the experimental setup and the calibration and alignment techniques, section 4 shows the experimental results on the aberration retrieval and correction as well as on different 3D+λ engineered PSFs, and section 5 summarizes the main conclusions of the paper.
2.1. Vectorial PSF model for an emitter with non-zero size
The vectorial PSF model [25,42] is further refined by taking into account the non-zero size of the fluorescent beads we use in the experiments (175 nm and 200 nm), which induces a blurring on the scale of 1–2 pixels. We describe the PSF of such a bead as a PSF of a freely rotating single dipole emitter convoluted with a sphere ◯(r⃗):43], and W (ρ⃗) the aberration function, which is the sum of a contribution from the microscope system and a contribution from the SLM. The wavevector k⃗ (ρ⃗) is a function of the pupil coordinates: 25] but now involve a convolution:
2.2. Aberration retrieval from a through focus PSF scan
Aberration retrieval is done by adapting the vectorial fitting routine to estimate the aberration coefficients from a through-focus image stack of a fluorescent bead. The aberration function is expressed as a linear sum of root mean square (rms) normalized Zernike polynomials :
3. Experimental setup and methods
3.1. Experimental setup
The setup, shown in Fig. 1, consists of a Nikon Ti-E microscope with a TIRF APO objective lens (NA = 1.49, M = 100), a 200 mm tube lens, a relay system with an SLM is built on one of the exit ports of the microscope. The relay system consists of two achromatic lenses (f1 = 100 mm, Thorlabs AC254-100-A and f2 = 200 mm, Thorlabs AC508-200-A), a nematic Liquid Crystal On Silicon (LCOS) SLM (Meadowlark, XY-series, 512×512 pixels, pixel size =15 μm, design wavelength = 532 nm) and a polarizing beam splitter to filter the x-polarized light which is not modulated by the SLM. The first achromatic lens relays the light on the SLM in a beam with a diameter of 3 mm and the second relay lens ensures Nyquist sampling of the fluorescent objects at the EMCCD (Andor iXon Ultra - X987, 512×512 pixels, pixel size = 16 μm, backprojected to object space 80 nm). The microscope is equipped with a set of lasers with wavelengths 405 nm (Coherent Cube), 488 nm (Coherent Sapphire), 561 nm (Coherent Sapphire), and 642 nm (MPB Communications). Either a dichroic filter set for the green (Ex: Semrock FF01-460/60-25, Di: Semrock Di02-R532-25X36, Em: Semrock FF01-545/55-25) or a quadband dichroic filter set (Chroma - TRF89902) is used.
This standard setup is augmented with a novel, second light path for calibration of the SLM. This SLM calibration light path is designed for measuring the retardation difference between the x and y-polarized light incident on the SLM and consists of a laser (Thorlabs - CPS532, wavelength 532 nm, 0.9 mW) which illuminates the SLM via a beam expander, a polarizing beam splitter and a λ/2 wave-plate. The reflected and polarization-modulated light passes the λ/2 wave-plate and polarizing beam splitter again and is imaged onto a CMOS camera (Thorlabs - DCC1545M, 1280×1024 pixels, pixel size = 5.2 μm). A rotating diffuser is added to reduce speckle and two linear polarizing filters, which are aligned with the polarization axes of the polarizing beamsplitter, are added to reduce internal reflections. The intensity image captured by the CMOS camera is mapped to the intensity pertaining to specific SLM pixels Ipxl. The polarization transfer through the λ/2 wave-plate and the SLM for the calibration light path is described by the Jones-matrix:
3.2. SLM calibration
In order to measure the modulation of a certain SLM pixel the mapping from the SLM onto the camera of the calibration path is needed. This mapping is obtained by applying a checkerboard pattern with increasing voltages to the SLM. The difference between the average captured image and the image when no voltage is applied is used as input for a corner detection algorithm (findcheckerboard from Matlab - Mathworks) to find the corner points. An affine transformation is fitted to these points and used to find the CMOS pixels corresponding to each SLM pixel.
The calibration procedure for an SLM pixel is graphically explained in Fig. 2. First, the intensity response as function of applied voltage is measured in 256 steps, giving rise to a sequence of minima and maxima, which correspond to a retardation of π or 2π. All pixels inside the illuminated SLM plane appear to have three maxima, implying a total phase modulation of 4π or 1094 nm. The voltages for which these extrema occur are found by fitting parabolae to the three points near the extrema, which increases the precision and fully utilizes the 16 bit control of the SLM. The intensity is then divided into four segments which are scaled to [0 1] and converted to phase using the inverse of Eq. (11) over these segments. The phase response is used to construct an individual Look Up Table (LUT) for each SLM pixel, compensating the non-uniformity of the SLM. The LUT-parameters vary smoothly over the SLM and correspond roughly with the Fabry-Perot fringes visible by eye, indicating that the differences in phase response are due to variations of the thickness of the liquid crystal layer. Additional pixel-to-pixel variations may arise from pixel-to-pixel variations in the underlying silicon switching circuitry. The complete calibration takes about 5 minutes (3 minutes scanning and 2 minutes computing time on a quadcore 3.3 GHz i7 processor), but can in principle be optimized to run faster.
The calibration is verified by applying a uniform retardation profile over the complete calibration range to measure the root mean square error of the reflected wavefront and compare the average intensity to the ideal intensity response. This verification is performed immediately after calibration and also after 45 minutes, and subsequently compared to the calibration provided by the manufacturer, which is only for 2π modulation, see Fig. 2(F). The rms error of the wavefront with the manufacturer’s LUT is around 100 mλ and within specification, but too high for the stated goal of our research: to achieve wavefront control well below the diffraction limit in order to optimally apply complex engineered PSFs in single emitter localization. The individual pixel calibration method reduces the wavefront error by an order of magnitude to around 10 mλ immediately after the calibration and to 20 mλ after 45 minutes. The deterioration of performance over time is attributed to temperature fluctuations and the associated mechanical drift of the different optical components. We conclude that there is about a half hour window of opportunity to conduct experiments with a precision of the wavefront control within 20 mλ rms wavefront aberration. All subsequent experiments have been done within this time frame. We mention that dedicated temperature control of the setup can extend the total time with high-precision wavefront control.
3.3. Axial and lateral alignment of the SLM
The position of the Optical Axis (OA) of the emission path on the SLM is needed in order to project the phase profile at the correct position on the SLM. Furthermore the SLM should be aligned with the plane conjugate to the pupil plane of the objective lens, the Fourier plane, to ensure that every point inside the Field Of View (FOV) has the same phase modulation. A procedure is used to directly estimate this alignment by imaging a set of beads in the FOV and applying a defocus modulation, without the need for additional optical components. For a bead in the center of the FOV the chief ray of emitted light beam aligns with the OA and intersects the SLM plane at a position (,) with respect to the coordinate frame that has its origin at the center of the SLM area designated for use in the aberration correction and PSF engineering. Here and are dimensionless pupil coordinates (real space coordinates normalized by the pupil radius). If the SLM introduces defocus then this parabolic aberration profile will be decentered w.r.t. the OA:Eq. (12) to the k⃗ (ρ⃗) · r⃗ term in Eq. (2). Therefore, the lateral alignment of the SLM can be estimated from the shift of a bead in the center of the FOV and the axial alignment can be estimated from the shift of the beads at the rim of the FOV. The center of these beams will intersect the SLM at a different lateral position if the SLM is not aligned with the Fourier plane as illustrated in Figs. 3(A)–3(D). The images of Fig. 3(C) and 3(D) where taken by replacing the SLM with a camera (Thorlabs - DCC1545M) and the camera alignment served as an estimate for the axial alignment of the SLM. The position of the beads is estimated from bright bead images (high signal-to-background) using a centroid weighted fit and the shift is measured for multiple defocus coefficients and then fitted with a linear line in order to reduce the effects of manual over and under focusing as shown in Figs. 3(E)–3(F). The procedure results in an initial lateral misalignment of the OA with (8,10)±0.3 SLM pixels, and after correction of the SLM aberration center with only (0.2,0.1)±0.3 SLM pixels (about 3 μm error), indicating that the phase profile could be aligned with the optical axis in a single iteration. The differences in shifts of the beads at the rim of the FOV of less than about 0.5 SLM pixels indicates that the SLM is aligned with the Fourier plane within approximately 100 μm.
3.4. Correction of oblique angle of incidence at SLM
The beam is reflected by the SLM under an oblique angle of θ = 20 degrees. The aberration function W (x, y) added by the SLM is related to the aberration function Wslm (x′, y′) under normal incidence, where (x, y) are the pupil coordinates normal to the beam axis and (x′, y′) are the coordinates in the SLM plane, by:Equation (14) describes two effects. The first is the scaling of the phase depth due to the oblique incidence, the second is the projection of the circular pupil onto the SLM plane, giving an anisotropic stretch and hence an elliptical cross-section (aspect ratio cos (20 deg) = 0.94). Implementing this transformation ensures that the phase profile contributed by the SLM corresponds to the aberration function for normal incidence used in the localization algorithm.
3.5. Sample and fluorescence emission spectra
All proof-of-principle experiments are performed on fluorescent beads emitting in the green (Ex/Em peaks: 505/515 nm, size = 175 nm, ThermoFisher - PS-Speck) with 488 nm laser excitation and the red (Ex/Em: 660/680 nm, size = 200 nm, ThermoFisher - TetraSpeck) with 642 nm laser excitation. The weighted average emission wavelength of the green emitting bead, weighted with the product of the specified emission spectrum and the filter spectra, is 536 nm (Chroma quadband filter) or 552 nm (Semrock green filter), the weighted average emission wavelength of the red emitting bead is 692 nm. The beads are put on a cover slip and immersed in oil (n = 1.51) to guarantee the best possible refractive index matching. The emission spectra of the fluorophores are measured by introducing a blazed grating profile at the SLM (pitch = 100 μm, maximum path length modulation pd = 500 nm), see Fig. 4(A) for the results. The relative large pitch is sufficient to make a rough estimate of the emission spectra, but is not comparable to the spectral resolving power of a spectrometer. The spectral broadening is mainly due to the diffraction limited spot size on the camera, leading to an apparent non-zero emission in the bandstop regions of the dichroic. The spectral resolution is estimated to be on the order of 40 nm (estimate obtained from the ratio of the 0th order spot size to the distance between the 0th and 1st order times the peak emission wavelength). Blazed grating profiles are applied in the x and y-direction (the direction of incidence is tilted in the x-direction at the SLM) giving rise to substantially the same emission spectrum, thereby confirming that the anisotropic stretch of the SLM aberration function described by the obliquity correction of Eq. (14) is correct. The emission peaks are found at around 520 nm and 680 nm, and the weighted average emission wavelengths are 536 nm and 693 nm, for the green and red beads, respectively, matching the bead specifications. The path length modulation of the SLM pd is calibrated by applying a blazed grating with increasing phase depth. This results in diffraction orders m = 0, 1, . . . with amplitudes:Fig. 4(C). This results in a phase depth modulation which is 3.8% higher for the green bead and can be regarded as a small correction of the 1/cos(θ′) term in the oblique angle correction. The measured phase depth in the deep red is 74% of the phase depth expected from the SLM calibration experiments. This may be due to fringe field effects between the pixels [44–46], possibly in relation to the dielectric mirror and coatings that are optimized for green light, to inherent chromatic dispersion of the liquid crystal and/or to spurious reflections in the system contributing to the 0th order. Both differences in modulation are incorporated into the MLE-based localization/wavelength fitting routine by correcting the phase modulation with a factor 1.038 and 0.74 for the green and red bead, respectively. The observed variation of effective phase depth with wavelength is not incorporated into the calibration of the SLM for pixel-to-pixel variations with the additional calibration light path.
4. Experimental results
4.1. Aberration retrieval
The aberration retrieval and subsequent correction is tested experimentally for beads emitting in the green, using the dichroic filter set for green emission. In particular, the dominant Zernike aberration modes, the fit precision, and the goodness of fit is assessed. To this end through-focus PSF stacks of 21 slices in a 2 μm range are recorded, converted to photon counts with a gain calibration procedure, and fitted on a 31×31 pixel Region Of Interest (ROI), where the 3D-position of the bead, the number of signal photons, the number of background photons per pixel, and a set of Zernike aberration coefficients are used as fit parameters in the MLE fitting routine. Measurements are repeated 5 times for determining the precision. The fitted signal photon count per focal slice is around 2.4×104, the fitted background photon count around 19 photons/pixel. Figure 5(A) shows the retrieved Zernike aberration coefficients for fitted modes (n, m) satisfying n + |m| ≤ 2 (j + 1) with j = 1, 2, 3, 4, 5 the fit order (j = 1 includes primary astigmatism, coma, and spherical aberration, j = 2 includes secondary astigmatism, coma, and spherical aberration, and primary trefoil, etc.), Fig. 5(B) shows the corresponding retrieved wavefronts. The Zernike coefficients found at fit order j match well with the ones found at a lower fit order j − 1, indicating that there is little cross-talk between Zernike modes in the MLE fitting routine, and that there is no overfitting with large numbers of fitted aberration coefficients. Even for the case j = 5 for which 45 modes are retrieved, there seem to be no problems with convergence (typically only about 6 iterations are needed), and reproducibility of the fit. The dominant Zernike modes appear to be primary astigmatism (), secondary coma (), and two higher orders spherical aberration (, ). The correction collar of the objective lens is used to reduce the primary spherical aberration () as much as possible, but this does not seem to compensate for the higher orders of spherical aberration.
The experimental fit precision for all Zernike modes is below 1.5 mλ and typically around 0.6 mλ, somewhat higher than the CRLB, which is around 0.3 mλ [Fig. 5(C)]. The impact of photon count on the fit precision of the aberration retrieval is further assessed with a simulation study. To this end through-focus PSF stacks of 21 slices in a 2 μm range are simulated where the total rms aberration level is kept constant at 10, 45 and 80 mλ. We use Ncfg = 100 random instances per aberration level. The set of aberrations used consists of all Zernikes modes of radial order n ≤ 4 (except piston, tip, tilt, defocus). Next, shot noise is added corresponding to a range of signal photon counts Nph and a background of b = 10 photons per camera pixel, and these sets of noisy through-focus PSFs are used as input for the MLE aberration fitter. The fit precision is quantified by the average rms error of the wavefront:Fig. 5(D) shows the result, and indicates that the precision scales as , in agreement with expectations. The average residual wavefront error of the fit appears to be drop below 1 mλ for a through-focus stack with more than 104 signal photons, corresponding to the experimental conditions in the aberration retrieval tests.
The goodness of fit is estimated with a chi-square test. The chi-square statistic is defined as:Figure 5(E) shows the measured χ2-values in relation to the expected value K = 21 × 312 = 2.0 × 104. It appears that the χ2-value converges to a level about 20% higher than the expected value, significantly more than the expected standard deviation 2×103. This shows that there are still some model errors left, probably effects of photobleaching and illumination intensity variations (it is assumed in the fit that the same level of signal and background photon count applies to each focal slice). Other effects which possibly contribute to the discrepancy are the non-zero spectral bandwidth of the collected fluorescence emission, residual scattering at the SLM, and amplitude aberrations or apodization due to e.g. variations in transmission through the objective lens and the dichroics with pupil position .
4.2. Aberration correction
The overall rms wavefront error, as determined from the fitted Zernike coefficients converges to a value around 65 mλ, just below the diffraction limit, with increasing number of Zernike modes [Fig. 5(F)], proving that aberration correction is needed for successful application of complex engineered PSFs. There is a trade-off between the number and type of Zernike modes that can be corrected and the peak-valley value of the wavefront that is needed for the desired engineered PSFs, in view of the limited phase dynamic range of the SLM. In the experiments on engineered PSFs we take all Zernike modes with radial order n ≤ 4 into account, similar to previous studies in the literature [18, 41]. In order to evaluate the best possible performance of the aberration correction we also include second order coma ( and ), second order spherical aberration (), and third order spherical aberration (). Figure 6 and Visualization 1 and Visualization 2 show the experimental and fitted through-focus PSF without and with aberration correction. The agreement between measured and fitted PSFs is quite well, especially in the 1 μm range around focus. It appears that the experimental through-focus PSF is more rotationally symmetric after correction, proving that asymmetry inducing aberrations are reduced. In addition, the asymmetry between spot shapes above and below focus is greatly reduced after correction, indicating that spherical aberration is largely eliminated. The wavefront error estimated with the aberration retrieval algorithm reduced from 59 ± 1 mλ to 13.4 ± 0.4 mλ [Fig. 6(G)]. This residual error is close to the calibration precision of the SLM, indicating that the level of aberration correction is limited by the precision of the SLM control.
A final test of the aberration retrieval and correction procedure is performed by deliberately adding single Zernike modes () on top of the corrected wavefront and subsequently feeding the aberrated through-focus stack to the aberration fitting routine, see Fig. 7 for the results. The fitting routine correctly retrieves each aberration with an estimated value of 67 ± 4 mλ, averaged over the 15 displayed aberrations, somewhat higher than the expected 60 mλ. All other retrieved aberrations remain at the level of 20 mλ or less, pointing to the specificity of the aberration retrieval and correction procedure. In particular, there is little crosstalk from added aberrations of radial order n to retrieved aberrations of order n − 1, which confirms the correct lateral alignment of the SLM phase profile with respect to the optical axis.
4.3. Analysis of engineered PSFs
Proof-of-principle experiments of engineered PSFs have been performed with green and red emitting beads, using the quadband dichroic filter set. Aberrations are corrected only up to radial order n ≤ 4 in order to save phase dynamical range on the SLM for the engineered PSFs. Primary spherical aberration is compensated by the correction collar of the objective lens. Three different designs have been tested. The first is a binary grating splitting the spot into two ±1st diffraction orders, where the grating zones are curved to induce astigmatism to the two orders (grating zone shape described by Zernike coefficients and , as in ). The nominal wavelength λ0 is equal to 520 nm for the green emitting beads and 690 nm for the red emitting beads. The second engineered PSF is a blazed grating for splitting the spot into a 0th and +1st order (grating zones curved to induce astigmatism, the shape is described by Zernike coefficients and ) and an overall continuous astigmatic aberration profile with Zernike coefficient . The third engineered PSF is a double helix configuration (annular design [17,18] with four rings and exponent α = 1/2, and phase depth = λ0). These design parameters are set by balancing the achievable precision with the axial range and with the footprint of the spot on the detector, but can in theory be improved by e.g. incorporating higher order astigmatism in case of the astigmatic profiles (mimicking the saddle-point or tetrapod PSF ). Through-focus image stacks of these engineered PSFs are recorded for different signal photon counts while keeping the background constant using the camera frame-time and the trans-illumination unit for conventional brightfield microscopy for providing extra background photons at shorter frame times as tuning parameters, and subsequently fitted using MLE and the vectorial PSF model.
Figure 8 shows the phase profile for the three engineered PSF designs and examples of measured spots and the corresponding fits for two estimated signal photon counts (5· 103 and 14 · 104). The overall experimental PSF, obtained by adding all recorded spots after first upsampling 3× and subsequently shifting with the fitted xy-position of the bead, is shown as well, along with the prediction of the vectorial PSF model. In this way the experimental PSF is built up from the cumulative signal of ∼ 108 photons. The agreement between experiment and the theoretical vectorial PSF is generally excellent, even the fringe structures at the largest defocus values match very well. The remaining discrepancies, mainly a slight broadening of the spots, is attributed to the non-zero spectral width of the light incident on the camera, due to the width of the emission spectrum and the width of the bandpass regions of the quadband dichroic. There is also a small asymmetry in the fringe structure, which is probably caused by the residual higher order spherical aberrations in the optical system.
The achieved precision in x/y/z/λ for the three engineered PSFs in focus as a function of signal photon count are shown in Figs. 9(A)–9(D), and for one signal level as a function of the axial position in Figs. 9(E)–9(H). The localization precision is obtained by fitting 25 acquisitions of a single bead at each z-position. The precisions follow the CRLB as a function of signal photon count and axial position, with the exception of the z-precision, which is somewhat worse than the CRLB. This is attributed to the higher orders of spherical aberration, which are not corrected for these datasets. The localization precision values appear to level off at values around x/y/z = 2/2/5 nm for high photon counts, probably due to effects of drift. The overall performance of the three engineered PSFs concerning precision appears to be quite similar. Figure 9(F) shows the average fitted z-position as a function of stage z-position. Both astigmatic engineered PSFs have a linear response with a fitted slope of 1.03±0.02 (blazed) and 1.01±0.02 (binary) in the green, but the double helix PSF and the blazed astigmatic PSF in the red underestimate the z-position slightly with a slope of 0.94±0.02 and 0.93±0.03, respectively. This bias may also be due to uncorrected higher order spherical aberration. The estimated wavelengths shown in Figs. 9(J)–9(K) are close to the measured weighted average emission wavelengths and resolve the green and red bead excellently. There seems to be a small overestimation of the wavelength for the green bead, which is constant over the entire axial range for the binary astigmatic PSF and the double helix PSF, but not for the blazed astigmatic PSF. The accuracy of the z-position is most likely affected by these small over and underestimations of the wavelength, as these parameters are closely coupled in the fitting routine.
In summary, we have shown how a dedicated calibration protocol for high-NA fluorescence microscopes equipped with an adaptive optical element such as an SLM can reduce the aberrations to around 20 mλ rms wavefront aberration. This high level of wavefront control enables single emitter localization with complex engineered PSFs, where the full vectorial PSF model is used in the fitting routine. A key ingredient in the calibration protocol is a separate SLM calibration light path for suppressing the effect of the non-uniformity of the SLM. In these well-controlled experimental circumstances the experimental PSFs conform very well to the predictions of the vectorial PSF model. The method is further tested with proof-of-principle experiments for fitting the 3D-position and the emission wavelength of single emitters. The precision in x/y/z/λ is similar to the CRLB in all four fit parameters, and a high accuracy in z and λ is obtained, without an experimental reference PSF or LUT.
An open issue is the residual model mismatch, as revealed by the χ2-test. A next step could be the expansion of the fit model with an intensity scaling that varies with the focal slice in order to take into account effects of photobleaching and illumination intensity fluctuations. Another inroad is to include apodization or amplitude aberrations, which model the dependence of the transmission of the different optical components (objective lens, dichroics) on pupil position. The optimum number of fit parameters can be assessed with statistical methods for model selection, in particular methods based on optimizing the Akaike information criterion, basically a weighted sum of the number of fit parameters and the maximum log-likelihood obtained by the fitting routine .
Another next step in the technical developments described in this paper would be to fit only the 3D-position keeping the emission wavelength fixed to the known weighted average fluorescence emission wavelength. This could possibly further improve the z-accuracy and precision. The non-zero spectral width could be taken into account by averaging the single-wavelength PSF over the spectrum weighted by the fluorescence emission and dichroic bandpass filter. Such an approach would enable identification of fluorescent labels with different emission spectra using a Generalized Likelihood Ratio Test (GRLT)  by comparing the likelihood of the fit for the known peak wavelengths of the different fluorescent labels.
An improvement of the current experimental setup for photon starved applications would be the replacement of the polarization sensitive LCOS-SLM by a polarization insensitive adaptive optical element such as a Deformable Mirror (DM). A similar calibration and alignment protocol as described here could be used, including the use of a calibration branch to suppress effects of thermal drift in the DM. Another direction of future research is to use the SLM or DM to mitigate the effects of sample induced aberrations. This can be done most effectively when the aberrations primarily originate from a sufficiently thin layer in the sample volume (on the order of the focal depth) by placing the adaptive optical element at a plane conjugate to the aberration inducing layer in the sample .
Data and software for aberration retrieval and ‘3D+λ’ PSF fitting is available at .
European Research Council (grant no. 648580); Merton College, Oxford, United Kingdom (Junior Research Fellowship C.S.S.).
We thank Aurèle Adam and Bernd Rieger for useful research advice.
References and links
1. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313, 1643–1645 (2006). [CrossRef]
5. R. P. J. Nieuwenhuizen, K. A. Lidke, M. Bates, D. Leyton Puig, D. Grünwald, S. Stallinga, and B. Rieger, “Measuring image resolution in optical nanoscopy,” Nat. Methods 10, 557–562 (2013). [CrossRef] [PubMed]
6. H. P. Kao and A. S. Verkman, “Tracking of single fluorescent particles in three dimensions: use of cylindrical optics to encode particle position,” Biophys. J. 67, 1291–1300 (1994). [CrossRef] [PubMed]
7. L. Holtzer, T. Meckel, and T. Schmidt, “Nanometric three-dimensional tracking of individual quantum dots in cells,” Appl. Phys. Lett. 90, 053902 (2007). [CrossRef]
10. S. Ram, P. Prabhat, J. Chao, E. S. Ward, and R. J. Ober, “High accuracy 3D quantum dot tracking with multifocal plane microscopy for the study of fast intracellular dynamics in live cells,” Biophys. J. 95, 6025–6043 (2008). [CrossRef] [PubMed]
11. M. F. Juette, T. J. D Gould, M. D. Lessard, M. J. Mlodzianoski, B. S. Nagpure, B. T. Bennett, S. T. Hess, and J. Bewersdorf, “Three-dimensional sub–100 nm resolution fluorescence microscopy of thick samples,” Nat. Methods 5, 527–529 (2008). [CrossRef] [PubMed]
13. M. D. Lew, S. F. Lee, M. Badieirostami, and W. E. Moerner, “Corkscrew point spread function for far-field three-dimensional nanoscale localization of pointlike objects,” Opt. Lett. 36, 202–204 (2011). [CrossRef] [PubMed]
14. S. R. P. Pavani and R. Piestun, “High-efficiency rotating point spread functions,” Opt. Express 16, 3484–3489 (2009). [CrossRef]
15. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. U.S.A. 116, 2995–2999 (2009). [CrossRef]
18. C. Roider, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Axial super-localisation using rotating point spread functions shaped by polarisation-dependent phase modulation,” Opt. Express 22, 4029–4037 (2014). [CrossRef] [PubMed]
19. Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise 3D scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15, 4194–4199 (2015). [CrossRef] [PubMed]
20. D. Baddeley, M. B. Cannell, and C. Soeller, “Three-dimensional sub-100 nm super-resolution imaging of biological samples using a phase ramp in the objective pupil,” Nano Research 4, 589–598 (2011). [CrossRef]
21. S. Jia, J. C. Vaughan, and X. Zhuang, “Isotropic three-dimensional super-resolution imaging with a self-bending point spread function,” Nat. Photonics 8, 302–306, (2014). [CrossRef]
22. G. Shtengel, J. A. Galbraith, C. G. Galbraith, J. Lippincott-Schwartz, J. M. Gillette, S. Manley, R. Sougrat, C. M. Waterman, P. Kanchanawong, M. W. Davidson, R. D. Fetter, and H.F. Hess, “Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure,” Proc. Natl. Acad. Sci. U.S.A. 106, 3125–3130 (2009). [CrossRef] [PubMed]
23. M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, and W. E. Moerner, “The role of molecular dipole orientation in single-molecule fluorescence microscopy and implications for super-resolution imaging,” Chem. Phys. Chem. 15, 587–599 (2014). [CrossRef] [PubMed]
24. J. Broeken, B. Rieger, and S. Stallinga, “Simultaneous measurement of position and color of single fluorescent emitters using diffractive optics,” Opt. Lett. 39, 3352–3355 (2014). [CrossRef] [PubMed]
25. C. Smith, M. Huisman, M. Siemons, D. Grünwald, and S. Stallinga, “Simultaneous measurement of emission color and 3D position of single molecules,” Opt. Express 24, 4996–5013 (2016). [CrossRef] [PubMed]
26. Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10, 590–594 (2016). [CrossRef] [PubMed]
28. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 9, 154–166 (1992). [CrossRef] [PubMed]
29. S. Stallinga and B. Rieger, “Position and orientation estimation of fixed dipole emitters using an effective Hermite point spread function model,” Opt. Express 20, 5896–5921 (2012). [CrossRef] [PubMed]
30. H. Kirshner, C. Vonesch, and M. Unser, “Can localization microscopy benefit from approximation theory?” 10th International Symposium on Biomedical Imaging, 588–591 (2013).
31. A. Tahmasbi, E. S. Ward, and R. J. Ober, “Determination of localization accuracy based on experimentally acquired image sets: applications to single molecule microscopy,” Opt. Express 23, 7630–7652 (2015). [CrossRef] [PubMed]
33. Y. Li, M. Mund, P. Hoess, U. Matti, B. Nijmeijer, V. J. Sabinina, J. Ellenberg, I. Schoen, and J. Ries, “Fast, robust and precise 3D localization for arbitrary point spread functions,” bioRxiv 172643; doi: https://doi.org/10.1101/172643.
34. A. Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2, 985–993 (2015). [CrossRef]
37. D. Burke, B. Patton, F. Huang, J. Bewersdorf, and M. J. Booth, “Adaptive optics correction of specimen-induced aberrations in single-molecule switching microscopy,” Optica 2, 177–185 (2015). [CrossRef]
39. S. Liu, E. B. Kromann, W. D. Krueger, J. Bewersdorf, and K. A. Lidke, “Three dimensional single molecule localization using a phase retrieved pupil function,” Opt. Express 21, 29462–29487 (2013). [CrossRef]
40. E. B. Kromann, T. J. Gould, M. F. Juette, J. E. Wilhjelm, and J. Bewersdorf, “Quantitative pupil analysis in stimulated emission depletion microscopy using phase retrieval,” Opt. Lett. 37, 1805–1807 (2012). [CrossRef] [PubMed]
42. K. I. Mortensen, L. S. Churchman, J. A. Spudich, and H. Flyvbjerg, “Optimized localization analysis for single-molecule tracking and super-resolution microscopy,” Nat. Methods , 7, 377–381 (2010). [CrossRef] [PubMed]
43. S. Stallinga, “Effect of rotational diffusion in an orientational potential well on the point spread function of electric dipole emitters,” J. Opt. Soc. Am. A 32, 213–223 (2015). [CrossRef]
44. E. Ronzitti, M. Guillon, V. de Sars, and V. Emiliani, “LCoS nematic SLM characterization and modeling for diffraction efficiency optimization, zero and ghost orders suppression,” Opt. Express 20, 17843–17855 (2012). [CrossRef] [PubMed]
47. H. Akaike, “A new look at the statistical model identification,” IEEE Trans. Automatic Control 19, 716–723 (1974). [CrossRef]
48. C. S. Smith, S. Stallinga, K. A. Lidke, B. Rieger, and D. Grünwald, “Probability-based particle detection that enables threshold-free and robust in vivo single molecule tracking,” Mol. Biol. Cell 26, 4057–4062 (2015). [CrossRef]
50. S. Stallinga, ftp://qiftp.tudelft.nl/stallinga/wavefrontcontrolPSFengineeringSMLM.zip.