Abstract

In microscopy, proper modeling of the image formation has a substantial effect on the precision and accuracy in localization experiments and facilitates the correction of aberrations in adaptive optics experiments. The observed images are subject to polarization effects, refractive index variations, and system specific constraints. Previously reported techniques have addressed these challenges by using complicated calibration samples, computationally heavy numerical algorithms, and various mathematical simplifications. In this work, we present a phase retrieval approach based on an analytical derivation of the vectorial diffraction model. Our method produces an accurate estimate of the system’s phase information, without any prior knowledge about the aberrations, in under a minute.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Attaining phase information in diffraction experiments is of great practical interest; however, intensity measurements, i.e. images, do not explicitly capture complex values. This hidden information can be estimated by phase retrieval (PR) [1] – namely, recovery of a complex signal from intensity measurements of its Fourier transform, which has therefore been employed in a wide variety of applications, e.g. X-ray crystallography [2], astronomy [3], optical ptychography [4,5], adaptive optics [6,7], and the subject of this work: localization microscopy [8,9].

In a typical 2D localization microscopy experiment, the localization step is performed by fitting images of well-separated emitters with a known point-spread function (PSF), e.g. an Airy disc. The reliance on precise knowledge of the PSF is highlighted in single-molecule localization microscopy (SMLM), where the signal-to-noise ratio (SNR) is limited. Acquiring additional information can be achieved by PSF engineering, where the PSF shape is intentionally perturbed to encode the desired information. This approach has been used for 3D measurements, e.g. astigmatism [10], Tetrapod [11] and double helix (DH) [12] PSFs; encoding emitter wavelength [1315] and molecular orientation [16]. A closely related application is adaptive optics, which employs additional optical elements to counteract aberrations present in the imaging system that distort the PSF. In all mentioned cases, PR can be used to correct the theoretically simulated PSFs or to design the phase mask which creates them [17].

In PR, the complex signal is numerically estimated from the magnitude of its Fourier transform. Any employed algorithm requires the nonobvious selection of a penalty function, and must overcome degeneracies in the solution, unstable derivatives, non-convex optimization, and more [18]. So far, PR methods have made use of the iterative Gerchberg-Saxton (GS) algorithm [19] and its variations [20], estimation over the Zernike basis of aberrations [2123], and pixel-wise numerical gradient calculations [24]. The first consideration of these algorithms is the tradeoff between computational efficiency and model accuracy; namely, a limited number of parameters must be chosen to describe the phase mask and various model approximations are employed. To reduce computational burden, the mentioned discrete approaches employ coarse graining in the physical space, while the methods that use a polynomial basis, e.g. Zernike polynomials, reduce the number of parameters by modeling only the most common optical aberrations.

A second consideration is the selection of an optical model. Thus far, most methods employ the scalar diffraction model [21,25]. The scalar model is more computationally efficient to simulate than the vectorial model; however, this approximation deteriorates for emitters near the coverslip, precisely where most calibration measurements are performed. While in some cases, PR can be performed within a biological sample (away from the coverslip), which would make the scalar model appropriate [26], it is often impractical.

In this work, we present a vectorial implementation of phase retrieval (VIPR): a PR technique based on an analytical derivative of the vectorial diffraction model that converges to a robust and accurate solution in a pixel-wise manner. Our implementation is very fast thanks to the GPU-optimized implementation using fast Fourier transforms (FFT). We provide a flexible software, suitable for both freely rotating emitters and fixed dipoles [27]. Furthermore, our method requires no priors on the pupil plane phase pattern (making it especially suitable for adaptive optics) and, intriguingly, also corrects for experimentally-observed effects, e.g. axial-position dependent lateral shift (also known as wobble), removing the need to calibrate and correct data in post-processing [28]. Our approach is designed to retrieve the phase information from a simple calibration measurement: images of a single bright emitter over a range of focal positions. The retrieved phase can then be applied to volumetric samples, e.g. 3D imaging of mitochondria and microtubule blinking data in cell imaging.

2. Methods

2.1 Optical models

An illustration of the optical setup is shown in Fig. 1(a). Briefly, the imaging system consists of a Nikon Eclipse-Ti inverted fluorescence microscope with a Plan Apo 100X/1.45 NA Nikon objective. The emission path was extended with two achromatic doublet lenses (focal length 15 cm) forming a 4-f system. A liquid crystal spatial light modulator (LC-SLM) is placed in the conjugate back focal plane (Meadowlark 1920X1152 liquid crystal on silicon) with an EMCCD camera (iXon 897 Life, Andor) at the image plane. The calibration sample consists of a water-covered glass coverslip (Fisher Scientific) with 40 nm fluorescent beads adhered to the surface (FluoSpheres (580/605), ThermoFisher), which is illuminated with $50\frac{W}{{c{m^2}}}$, 560 nm light (iChrome MLE, Toptica).

 figure: Fig. 1.

Fig. 1. (a) Standard PSF engineering optical setup. The emission from a fluorescent molecule is collected by a high NA oil-immersed objective. A 4-f Fourier processing setup extends the optical path and enables the introduction of a phase mask, implemented via LC-SLM which modulates only the polarization transmitted by the Linear Polarizer (LP). (b) (Experimentally measured) A fluorescent bead scanned over 3 µm, imaged with a designed phase mask (using our method) which encodes a four letter PSF at different focal plane positions - demonstrates the generality of pixel-wise PSF engineering. (c) Schematic of our simulation model described in Section 2.1 where the position of the molecule and the phase mask define the PSF.

Download Full Size | PPT Slide | PDF

Modeling the 3D PSF of the microscope can be done in various methods. A frequently used model is known as the scalar approximation. This model presupposes that the emission pattern of each point source is spherical. This is a good approximation when the objective NA is low, the emitter is rotationally mobile [29], and particularly when imaging emitters far from optical interfaces, i.e. beyond near-field effects such as super-critical angle fluorescence (SAF). Another crucial difference between the vectorial and scalar models is that volumetric imaging far from the coverslip (beyond ∼1 wavelength), has an effective NA that is limited by the sample’s refractive index rather than the objective NA (details in Appendix A). The main advantage of the scalar model is the reduced computational complexity in numerical simulations. Our method implements the more complex vectorial model (in addition to the scalar model) to account for the mentioned effects.

Designing arbitrary PSFs (Figs. 1(a)–1(b)) has two additional challenges: (1) the 3D PSF is smooth, due to the diffraction limit; as seen in Fig. 1(b), the designed letters had to be spaced by $1{\; }\mu m$. (2) All the optical aberrations (system and sample specific) are unaccounted for and adaptive optics should then be deployed to improve the result.

In this section, we discuss freely-rotating emitters for both the scalar and vectorial models. Here, the intensity pattern $I(x,y;\alpha ,\vec{q},P,{\sigma _B})$ of a quasi-monochromatic source at the image plane is given by [3032] :

$$I(x,y;\alpha ,\vec{q},P,{\sigma _B}) = \sum\limits_w^{} {|{g_{img,w}}(x,y;\alpha ,\vec{q},P){|^2}}\otimes B,$$
where $\otimes $ denotes the 2D convolution operator, P is the phase aberration (phase mask) from the ideal (clear aperture) system and $\alpha $ encompasses the optical parameters required to model the imaging system, including refractive indices, objective parameters, emission wavelength and more. $\vec{q}$ describes the coordinates relative to the coverslip:$\vec{q} = ({x_0},{y_0},{z_0}, - NFP)$;$({x_0}{y_0},{z_0})$ are the emitter’s position coordinates and Nominal Focal Plane ($NFP$) is the objective displacement from focusing on the coverslip. $w$ are the electrical field components in the model (Appendix A): $w = 1$ for the scalar model, $w = 1,2\ldots 6$ for the full vectorial description and $w = 1,2,3$ or $w = 4,5,6$ for the vectorial model with a linear polarizer, relevant for polarization dependent LC-SLMs. $B$ denotes the optical blurring (estimated by a Gaussian kernel with a spatial standard deviation of ${\sigma _B}$). In some models [30,33], an additional filter is added which describes the 3D distribution of emitters for the case of large fluorescent particles. For beads smaller than the wavelength, we approximate these filters by a Gaussian blur to reduce computational complexity.

The electrical field components ${g_{img,w}}(x,y;\alpha ,\vec{q},P)$ in Eq. (1) are generated by:

$${g_{img,w}}(x,y;\alpha ,\vec{q},P) = {{\cal F}_{2D}}[{E_{bfp,w}}(\widehat x,\widehat y;\alpha ,\vec{q},P)],$$
where ${{\cal F}_{2D}}$ denotes the 2D Fourier transform operator, $(\widehat x,\widehat y)$ are the Cartesian coordinates at the back focal plane and ${E_{bfp,w}}$ represents the electrical field components at the back focal plane given by:
$${E_{bfp,w}}(\widehat x,\widehat y;\alpha ,\vec{q},P) = {g_{bfp,w}}(\widehat x,\widehat y;\alpha ,\vec{q}) \cdot S(\widehat x,\widehat y;\alpha ,\vec{q}) \cdot \exp [j \cdot \psi (\widehat x,\widehat y;\alpha ,\vec{q},P)],$$
${g_{bfp,w}}$ and $S$ are the Green’s tensor components and back focal plane support (aperture), respectively (both are defined in Appendix A). $\psi$ is the phase term defined by:
$$\psi (\widehat x,\widehat y;\alpha ,\vec{q},P) = P(\widehat x,\widehat y) + \sum\limits_{l = 1}^4 {{\phi _l}(\widehat x,\widehat y;\alpha ){q^{(l)}}} ,$$
where ${q^{(l)}}$ are the elements of $\vec{q}$. We assume that the phase mask is a single matrix for all the electric field components, i.e. there are no birefringent effects in the back focal plane and we neglect any axial aberrations (which cannot be described by a single phase mask). ${\phi _l}(\widehat x,\widehat y;\alpha )$ are known phase terms that are associated with the $\vec{q}$ coordinates (Appendix A).

In the next sections, we assume that the optical system is fixed and that we measure a pixelated version of the intensity $I(x,y;\alpha ,\vec{q},P,{\sigma _B})$. Thus we drop the dependence on the optical system parameters $\alpha$, and keep only the phase mask $P{\; }$ and the blur kernel std ${\sigma _B}$ as free parameters, e.g. $I(n;P,\vec{q},{\sigma _B})$, where $n = 1,2\ldots N$ is the pixel number.

2.2 Cost functions

The next step in optimization is introducing the observed z-stack. To do so, a cost function which penalizes the difference between the simulation and measurements is required. Such cost functions can serve in both the PR step and subsequent localization steps. All of the considered cost functions are of the form $C = \frac{1}{{NQ}}\sum\limits_{i = 1}^Q {\sum\limits_{n = 1}^N {f[I(n;P,{{\vec{q}}_i},{\sigma _B}),{Z_i}(n),{\sigma _i}]} }$ where ${\sigma _i}$ is a noise parameter and ${Z_i}(n)$ is the observed image associated with positions ${\vec{q}_i}, i = 1,2\ldots Q$, e.g. in the typical case of PR from a z-stack of a bead on the coverslip surface, these correspond to different values of $NFP$.

Common examples of cost functions include the L1 [34] and L2 [24] norms. Other cost functions employ statistical estimation, e.g. are derived assuming Poisson distribution and thus define a negative log-likelihood cost [11,25,35,36]:

$${C_{P,NLL}}[I(n;P,{\vec{q}_i},{\sigma _B}),{Z_i}(n)] ={-} \frac{1}{{NQ}}\sum\limits_{i = 1}^Q {\sum\limits_{n = 1}^N {{Z_i}(n) \cdot log[I(n;P,{{\vec{q}}_i},{\sigma _B})] - I(n;P,{{\vec{q}}_i},{\sigma _B})} } .$$
We assume that the intensity distribution $\widetilde I(n;P,{\vec{q}_i},{\sigma _B})$ is governed by Poisson statistics with a large mean value, thus a Gaussian approximation is valid (by the law of large numbers) in good calibration samples. In many cases, an additive background term is introduced to the model PSF to better match experimental data [25]. This background noise $bg(n)$ can be approximated to be white noise, independent from the emitter, with variance of ${\sigma _i}^2$:
$$\widetilde I(n;P,{\vec{q}_i},{\sigma _B}) + bg(n)\sim {\cal N}(I(n;P,{\vec{q}_i},{\sigma _B}),I(n;P,{\vec{q}_i},{\sigma _B}) + {\sigma _i}^2).$$
This distribution defines a negative log likelihood cost function:
$$\begin{aligned}{C_{G,NLL}}[I(n;P,{\vec{q}_i},{\sigma _B}),{Z_i}(n),{\sigma _i}] & = \frac{1}{{2NQ}}\sum\limits_{i = 1}^Q \sum\limits_{n = 1}^N {log[2\pi (I(n;P,{{\vec{q}}_i},{\sigma _B}) + {\sigma _i}^2)]}\\ & \quad + \frac{{{{[{Z_i}(n) - I(n;P,{{\vec{q}}_i},{\sigma _B})]}^2}}}{{I(n;P,{{\vec{q}}_i},{\sigma _B}) + {\sigma _i}^2}}.\end{aligned}$$
As can be seen in Eq. (7), the resulting cost function is a weighted L2 norm (could be viewed as a quadratic approximation of the Poisson log-likelihood cost) added to a logarithmic term. Unlike ${C_{P,NLL}}$, we found the cost ${C_{G,NLL}}$ to be is empirically more stable for PR, mainly due to uncertainty in signal estimation (Eq. (20)). A comparison between the two latter costs is given in Appendix B in terms of their applicability and performance in localization.

2.3 Analytical gradient for phase retrieval

Phase retrieval can be viewed as a nonconvex optimization problem [37]. In general, such optimization problems pose a challenge as they can have spurious local minimizers, saddle points, and significant sensitivity to good initialization. However, it has been shown that PR can achieve convergence using stochastic gradient descent (SGD) [18].

To facilitate a fast gradient computation, we analytically derive the gradient of the cost function with respect to the pixelated phase mask $P(m),m = 1,2\ldots ,M$, i.e. $\frac{{\partial C}}{{\partial P}} \in {{\mathbb R}^M}$. In this section we describe the general derivation for dipoles with full rotational mobility (the fixed dipole case is derived in Appendix E). For simplicity, we derive the gradient for a single position ${\vec{q}_i}$($Q = 1$) and denote $I_i(n) = I(n;P,{\vec{q}_i},{\sigma _B})$. The first derivative in the chain rule (for the form of cost functions defined in Section 2.2) gives:

$$\frac{{\partial C}}{{\partial I_i(n)}} = \frac{1}{N}\frac{{\partial f[I_i(n),{Z_i}(n),{\sigma _i}]}}{{\partial I_i(n)}},$$
where $\frac{{\partial C}}{{\partial I}} \in {{\mathbb R}^N}$. The second step is the derivative of Eq. (1) which relies on Wirtinger coordinates [18,38] :
$$\frac{{\partial I_i(n)}}{{\partial P(m)}} = 2{\mathop{\rm Re}\nolimits} \{ \sum\limits_w^{} {[\frac{{\partial {g_{img,w}}(n;P,{{\vec{q}}_i})}}{{\partial P(m)}}\overline {{g_{img,w}}} (n;P,{{\vec{q}}_i})]} \otimes B\} ,$$
where $\frac{{\partial {g_{img,w}}}}{{\partial P}} \in {{\mathbb C}^{NxM}},\frac{{\partial I}}{{\partial P}} \in {{\mathbb R}^{NxM}}$ and $\overline {{g_{img,w}}}$ denotes the complex conjugate of ${g_{img,w}}$. The matrix element $\frac{{\partial {g_{img,w}}(n;P,{{\vec{q}}_i})}}{{\partial P(m)}}$ is derived according to Eqs. (2)–(4):
$$\begin{array}{l} \frac{{\partial {g_{img,w}}(n;P,{{\vec{q}}_i})}}{{\partial P(m)}} = \frac{\partial }{{\partial P(m)}}{{\cal F}_{\widetilde m}}{[{E_{bfp,w}}(\widetilde m;P,{{\vec{q}}_i})]_{}} = \\ = \frac{\partial }{{\partial P(m)}}\sum\limits_{\widetilde m = 1}^M {\exp (\frac{{ - j2\pi n\widetilde m}}{M})[{E_{bfp,w}}(\widetilde m;P,{{\vec{q}}_i})]} = j\exp (\frac{{ - j2\pi n\widetilde m}}{M}){E_{bfp,w}}(\widetilde m;P,{{\vec{q}}_i}){\delta _{m,\widetilde m}} \end{array},$$
where ${\delta _{m,\widetilde m}}$ denotes Kronecker’s delta and ${{\cal F}_a}$ is the discrete Fourier transform over index a. The final step is the matrix multiplication of Eqs. (8)–(9):
$$\frac{{\partial C}}{{\partial P(m)}} = \frac{2}{N}\sum\limits_w^{} {{\mathop{\rm Re}\nolimits} (\frac{{\partial {g_{img,w}}(n;P,{{\vec{q}}_i})}}{{\partial P(m)}}\overline {\{ {g_{img,w}}} (n;P,{{\vec{q}}_i})\otimes B\frac{{\partial f[I_i(n),{Z_i}(n),{\sigma _i}]}}{{\partial I_i(n)}}\} )} .$$
By zero padding Fourier space such that $M = N$, the matrix multiplication in Eq. (11) gives a Fourier transform over n (using Eq. (10)):
$$\frac{{\partial C}}{{\partial P(m)}} = \frac{2}{N}\sum\limits_w^{} {{\mathop{\rm Re}\nolimits} ({E_{bfp,w}}(m;P,{{\vec{q}}_i}){{\cal F}_n}_{}[j\overline {{g_{img,w}}} (n;P,{{\vec{q}}_i})\frac{{\partial f(I_i(n),{Z_i}(n),{\sigma _i})}}{{\partial I_i(n)}}\otimes B])} .$$
Using Eq. (12), we calculate a single iteration using a few Fourier transforms: the full vectorial model ($w = 1,2,\ldots ,6$) requires 12 FFT calculations of size $M\; $ per z position: six to generate the image and six to generate the gradient; critically, a pixel-wise numerical derivative would require $6({M + 1} )\; $ FFT calculations of size $\; M\; $ per z position (six FFT calculation for a single direction derivative at any pixel and another six to generate the initial image).

2.4 Phase mask optimization workflow

Phase retrieval can be used either for recovering the pupil plane from measured PSFs, or for designing a new phase mask which will produce desired PSFs. Both of these applications amount to very similar procedures. In this section we describe the optimization flow which generates an estimated phase mask from the measured 3D PSF using the previously derived gradient. For mask design, the difference is that no denoising or experimental blur correction are required. The first step in the optimization procedure requires the input of a z-stack ${Z_i}(n)$ of length $Q$ (typically 40-50 images) with associated coordinates ${\vec{q}_i}$. Note that the PR result is the estimation of the phase in the back focal plane, assuming known coordinates ${\vec{q}_i}.$ Any mismatch between experiment and assumed coordinates will affect the result. For example, an inaccurate axial position input is compensated by an added defocus correction in the phase mask. Optimally, as many images would be used as possible within the axial range of interest, considering practical limitations, e.g. stage resolution and photobleaching. Next, the background is estimated from the corners of the cropped images (close to the PSF) to subtract the mean and estimate the noise standard deviation ($\widehat {{\sigma _i}}$), this background subtraction method works well with wide-field illumination in a small FOV, and it can be replaced by experiment-dependent customized methods.

We initialize the phase mask with an analytical gradient similar to [18]. This improves the convergence rate, because at this point, the simulated images (standard PSF) and measured images are not necessarily correlated. Subsequent ${K_1}$ iterations calculate the gradient for a batch of images (chosen by stochastic sampling) via Adam [39] with a step size of $\eta = 3 \cdot {10^{ - 2}}.$ The estimated blur kernel $\widehat {{\sigma _B}}$ and the noise $\widehat {{\sigma _i}}$ are drawn from a normal distribution in each iteration to reduce the dependence of the solution on the exact estimates. The full algorithm is described in Algorithm 1.

Two optional steps have been included in the software which may improve results in some cases at the cost of additional computation time. The first is fine-tuning the blur kernel (default $\widehat {{\sigma _B}} = 0.75$) after $\; {K_1}/3\; $ iterations. The blur kernel estimation is done using the MATLAB routine fmincon. The second option is performing ${K_2}$ iterations with a variant of directed sampling [40,41] where the side information is the correlation of the current model with the measured image stack, i.e. compute the gradient at positions where the correlation between the simulation and data is minimal. This can improve results when the used phase mask converges poorly at some focal positions; in this work, we found this step necessary only for the misaligned Tetrapod (Figs. 4 and 8). Future work can include optimization of this final step based on recent advances in adaptive sampling [42], which could further improve convergence.

Importantly, this method is very fast, thanks to the FFT implementation (Eq. (12)): Table 1. Shows the average time required to calculate a single gradient as a function of number of pixels (binning) in the phase mask. The algorithm is orders of magnitudes faster than numerical pixel-wise gradient calculation [24], and much faster than fitting Zernike polynomials implemented by the MATLAB function fminunc [34]. A single gradient calculation for a Zernike coefficient is on the same time scale as the presented VIPR gradient, thus the theoretical acceleration factor is exactly the amount of Zernike polynomials considered. The results were timed on a laptop with INTEL i7-8750H CPU and NVIDIA GeForce RTX 2070 GPU. Typical pupil plane recovery (vectorial model with a linear polarizer) from a fluorescent bead z-stack requires 300 iterations over 44000 pixels, using a batch size of 3 z positions (per iteration), and takes 31.5 seconds on a GPU (Table 1) compared to ∼30 minutes for Zernike polynomial fitting with a global gradient computation. An equivalent numeric pixel-wise gradient based calculation takes ∼9 days (∼25,000 times slower). Note that the significance of the improvement in run-time over Zernike polynomial fitting inflates quickly when the PR becomes a step in a larger iterative procedure, such as designing new optimal PSFs [17], live adaptive optics implementations and spatial aberration corrections [43].

Tables Icon

Table 1. Average run time for a gradient calculation using the analytical gradient vs. a numerical gradient

oe-28-7-10179-i001

3. Results and discussion

3.1 Comparison with existing methods

For benchmarking, we compare our vectorial PR (VIPR) to two alternatives: (1) analytically optimizing (with our method) the scalar diffraction model and (2) adding Zernike polynomials (first 60 polynomials without tip and tilt) to the initially designed phase mask using the vectorial model [21,34]. Note that using Zernike polynomials requires an additional step to address observed wobble, by fitting the z-stack with the initial guessed mask to center it. Using the described methods, we retrieved phase masks from a z-stacks using a $4\; \mu m\; $ Tetrapod phase mask [11,44], with one image per NFP at a step size of $100\; nm\; $ (measured stack at Fig. 2(b) second row with an estimated image sum (signal) of $3.5\cdot {10^5}ADU$ and background ${\sim} {\cal N}(870,8100)$). To quantify the performance, we analyzed z-stacks with ten images per focal position (Figs. 2(b)–2(e)). Robustness of our approach to image noise was tested by computationally degrading the images with a Gaussian-distributed noise (std equal to a multiplier scalar times the estimated std of the noise).

While VIPR deteriorates in precision with added noise, the other methods, interestingly, initially improve after the addition of some noise; this is likely due to VIPR capturing the subtle details of measured intensity patterns and thus avoids bad localizations at specific axial positions (Appendix C). To test the axial localization bias, i.e. the z accuracy, and the wobble behavior we evaluated the performance of the methods on a short z-stack (one image per focal position). The total error is a combination of method-related PR error, measurement error (stage and readout accuracy) and SNR, in the two sets of data (the calibration set and the testing set). We found that our method incorporates the wobble better and has minimal axial bias (Appendix C and Visualization 1).

VIPR is based on sensor-less phase retrieval, i.e. we do not measure the wavefront directly at the back focal plane with a wavefront sensor. Additionally, the main sources of aberrations in PSF engineering setup (especially with the introduction of large phase modulations) is the LC-SLM device itself [45] and the objective lens. These aberrations might introduce deviations between the designed phase mask (Fig. 2(b) first row) and the PR mask; however, the essential result is creating an accurate PSF generation model (Visualization 1 and Visualization 2) for localization and estimation of the systematic and sample specific effective phase modulation that the measured PSF had.

 figure: Fig. 2.

Fig. 2. (a) Optimization evolution of the PR mask. (b) First row is the theoretical simulation of the designed phase mask (left) and PSFs (right). Second row shows the calibration measurements, which were used for PR, acquired by imaging 40 nm fluorescent beads. The next three rows show the simulated model using the reconstructed phase masks (left) and PSFs from the tested methods, e.g. VIPR (blue), Scalar PR (using our method with a scalar model-red) without the SAF light modulation (hence the clear annulus around the PR mask) and adding Zernike aberrations to the vectorial model (green), using the prior knowledge of the designed phase mask. The most dominant aberrations where spherical, of the second and fourth order. (c) Precision error on the full axial range as a function of added noise and (d) precision error in each z position at a single noise multiplier (24). (e) Full axial range reconstruction with ten images per focal position (510 images) using each of the retrieved phase masks. Zoom-ins shows that the Scalar PR (red) suffers from a bias in the axial range scaling. The orange outline shows a single PSF and MLE fits using each of the phase masks.

Download Full Size | PPT Slide | PDF

3.2 Generality to phase patterns

The benefit of pixel-wise methods is their broad applicability to model any aberration, e.g. non-smooth ones, which require many Zernike modes to describe. The Tetrapod phase mask presented in Section 3.1 is the best-case-scenario for Zernike polynomials as the designed phase mask is directly constructed by a sum of low order Zernike polynomials. To test the generality of our method, we imaged z-stacks with 4 different phase masks using the LC-SLM. The first mask consists of a randomly generated combination of low amplitude Zernike polynomials to test the applicability of the method for adaptive optics (Fig. 3(a)). The resulting PSF simulations from VIPR produce similar PSFs to the measured data, which is the crucial step in PR and the desired result. However, the deviations between the designed phase mask and the result (highlighted in Fig. 3(a) but present in all the obtained phase masks) show that the strongest optical aberrations occur on the edges of the back focal plane. We attribute these deviations to objective specific aberrations, especially at the SAF frequencies, and to the validity of the aplanatic approximation at the edges of the NA.

The second case shows a machine-learned phase mask which was recently demonstrated to optimally measure dense fields of emitters in 3D (Fig. 3(b)) [46]. The third test (Fig. 8) deals with severe misalignments in the Tetrapod phase mask, which has a large area of unmodulated pixels. The results for the double-helix phase mask, which comes from optimization of Gauss-Laguerre basis functions and is commonly used in PSF engineering [12] are also shown in Fig. 9 and Visualization 2.

 figure: Fig. 3.

Fig. 3. (a) Top row – theoretical phase mask, created from low amplitude combination of random Zernike polynomials and acquired images used to reconstruct the second row using VIPR. (b) Top row – theoretical machine learned phase mask and acquired images used to reconstruct the second row using VIPR.

Download Full Size | PPT Slide | PDF

4. Experimental demonstrations of STORM data

Direct stochastic optical reconstruction microscopy (dSTORM) is a technique in which a super-resolution image is reconstructed from localization of sparse fluorescent molecules that are switched on and off (blinking) using high power laser [47,48]. Both experiments (Section 4 and Appendix F) were performed using a dielectric $4{\; }\mu m$ Tetrapod phase mask, which exhibits better photon efficiency than the LC-SLM polarization dependent implementation. PR was performed in a similar way to the LC-SLM data (by removing the polarizer). Appendix F shows a reconstruction of fluorescently labeled mitochondria in COS7 cells, using the deep-learning network which trains on simulations only, adapted from [46]. This demonstrates the viability of our in-vitro method in predicting in-vivo PSFs.

To show the robustness of our method to suboptimal situations, we used a misaligned $4{\; }\mu m$ dielectric Tetrapod phase mask to image microtubule blinking data. We show in Fig. 4 the applicability of our method in recovering unusual aberrations, which are otherwise challenging to model. The use of image-based interpolation methods can also address these issues; However, such methods need a good volumetric calibration [49] and are challenging to deploy on a GPU as it requires much more parameters to model the PSF [22]. The PR predictions (using the vectorial model) were used to localize 10 K frame of cell data (Fig. 4(a)). Using the localization and an axial range of ${\pm} 25{\; }nm$ around the desired positions, we created mean experimental PSFs (Fig. 4(c)) to show their matching with our model.

 figure: Fig. 4.

Fig. 4. (a) Two example experimental frames taken from a microtubule blinking experiment covering a range of ∼$3.5\; \mu m$. Distinct PSFs (A-F) from various axial positions were picked by a localization with deep-learning net that was trained on: (d) the phase retrieved mask (by VIPR) from a coverslip z-stack with FluoSpheres(625/645). (b) Simulated PSFs at the axial position matching to emitters A-F using the PR mask with VIPR. (c) Average PSFs from 10 K frames in a range of ${\pm} 25\; nm$ around the stated axial position. (e) Measured PSF in the top of the range with the matching simulation and a spline fit performed on the coverslip data. The difference in shapes shows the importance of incorporating the refractive index mismatch in the optical model as PSFs far from the coverslip experience large deviations from the z-stack data.

Download Full Size | PPT Slide | PDF

5. Conclusion

In this work, we have developed and demonstrated an accurate and fast optimization method for PR and PSF design. Our method estimates the pupil-plane phase distribution in a microscope experiment based on Fourier optics. The optimization uses only basic knowledge of the optical system and a target image set (simulated or measured), with no priors on the phase mask. The derivation is general and can be adopted for many microscopy applications, sample variation (fixed or freely rotating dipoles) and optical processing methodologies (e.g. a polarized LC-SLM, a dielectric phase mask or a deformable mirror can be directly modeled with their real pixel size and optical constraints). The performance of VIPR is mainly limited by SNR, the used approximations (e.g. numerical pixelation, aplanatic performance, axially invariant NA, etc.), number of measurements, and accuracy in the knowledge of the optical parameters. Slight improvements could be made by retrieving an amplitude mask and by measuring the optical parameters, e.g. NA, magnification, etc., as part of the calibration process at the cost of computational complexity. The pixel-wise derivation can be adopted to many applications by simply changing the cost function and optimization constraints. Furthermore, this approach can be applied to adaptive optics due to the versatility and speed of our implementation.

Appendix A

In this section we describe the analytical (far-field) electrical field derivation at the back focal plane which is required in Eq. (3) to calculate -${E_{bfp,w}}(\widehat x,\widehat y;\alpha ,\vec{q}),w = 1,2\ldots 6$. For this section, we assume that there is no phase mask in the Fourier plane (a unit aperture). The source is an emitter located at ${r_0} = ({x_0},{y_0},{z_0})$ with a dipole vector $\overrightarrow \mu = ({\mu _1},{\mu _2},{\mu _3})$. Emitters are immersed in a sample with refractive index ${n_m}$, placed above an oil immersion objective with refractive index ${n_{imm}}$. The emission from the source can be described by the far-field Green tensor ${G_{ff}}({r_0},r)$ [31,32,50], which links between the three polarizations of the emitter to the electrical field generated at the far field position $r = (\hat{x},\hat{y},z)$:

$${G_{ff}}({r_0},r) = A \cdot {e^{j\psi }} \cdot \left( {\begin{array}{@{}ccc@{}} {\frac{{{{\cos }^2}(\varphi ){z^2}}}{{||r|{|^2}}}{\Phi ^{(2)}} + {{\sin }^2}(\varphi ){\Phi ^{(3)}}}&{\sin (\varphi )\cos (\varphi )(\frac{{{z^2}}}{{||r|{|^2}}}{\Phi ^{(2)}} - {\Phi ^{(3)}})}&{ - \frac{{\rho \cos (\varphi )z}}{{||r|{|^2}}}{\Phi ^{(1)}}}\\ {\sin (\varphi )\cos (\varphi )(\frac{{{z^2}}}{{||r|{|^2}}}{\Phi ^{(2)}} - {\Phi ^{(3)}})}&{\frac{{{{\sin }^2}(\varphi ){z^2}}}{{||r|{|^2}}}{\Phi ^{(2)}} + {{\cos }^2}(\varphi ){\Phi ^{(3)}}}&{ - \frac{{\rho \sin (\varphi )z}}{{||r|{|^2}}}{\Phi ^{(1)}}}\\ { - \frac{{\rho \cos (\varphi )z}}{{||r|{|^2}}}{\Phi ^{(2)}}}&{ - \frac{{\rho \sin (\varphi )z}}{{||r|{|^2}}}{\Phi ^{(2)}}}&{\left( {1 - \frac{{{z^2}}}{{||r|{|^2}}}} \right){\Phi ^{(1)}}} \end{array}} \right),$$
where $(\varphi ,\rho )$ denote the angular coordinates $\hat{x} = \rho \cos (\varphi ),\hat{y} = \rho \sin (\varphi )$. The propagation path is defined by the refractive indices and the propagation angles in the immersion oil ${\theta _{imm}}$ $\left(\rho=\left\| r\|\sin \left(\theta_{i m m}\right), z=\right\| r \| \cos \left(\theta_{i m m}\right)\right)$ and sample medium ${\theta _m}$ (connected by Snell’s law). The coefficients ${\Phi ^{(v)}},v = 1,2,3$ describe the transmission of the different polarizations at the coverslip interface,
$${\Phi ^{(1)}} = \frac{{{{\left( {\frac{{{n_{imm}}}}{{{n_m}}}} \right)}^2}\cos ({\theta _{imm}})}}{{\cos ({\theta _m})}}{t_p}({\theta _m},{n_{imm}},{n_m}),$$
$${\Phi ^{(2)}} = \frac{{{n_{imm}}}}{{{n_m}}}{t_p}({\theta _m},{n_{imm}},{n_m}),$$
$${\Phi ^{(3)}} = \frac{{\left( {\frac{{{n_{imm}}}}{{{n_m}}}} \right)\cos ({\theta _{imm}})}}{{\cos ({\theta _m})}}{t_s}({\theta _m},{n_{imm}},{n_m}).$$
$A\; $ is an amplitude coefficient that can be ignored as the intensity is normalized according to a desired signal intensity. ${t_p}$ and ${t_s}$ are the Fresnel transmission coefficients for polarizations p and s, respectively. The terms ${\phi _l}(\widehat x,\widehat y;\alpha )$ in $\psi (\widehat x,\widehat y;\alpha ,\vec{q}) = \sum\limits_{l = 1}^4 {{\phi _l}(\widehat x,\widehat y;\alpha ){q^{(l)}}}$ (Eq. (4)) are defined by axial and lateral phases. The lateral phases ($l = 1,2$) are more convenient to describe in the back focal plane using Fourier optics [51] assuming a linear shift invariant system:
$$\begin{array}{l} {\phi _1}(\widehat x,\widehat y) = k\frac{M}{f}\widehat x,\\ {\phi _2}(\widehat x,\widehat y) = k\frac{M}{f}\widehat y, \end{array}$$
where $\; M\; $ is the system magnification, $\; f\; $ is the focal length of the 4-f lenses and $k$ is the free-space wavenumber. The phase induced by the propagation along the optical axis in the different media ($l = 3,4$) under the assumption that this defocus is small compared to the depth of field:
$$\begin{array}{l} {\phi _3}(\widehat x,\widehat y) = k{n_m}\cos ({\theta _m}),\\ {\phi _4}(\widehat x,\widehat y) = k{n_{imm}}\cos ({\theta _{imm}}). \end{array}$$

The support of the Fourier plane, $S(\widehat x,\widehat y)$ (Eq. (3)), in such systems is usually limited by the objective’s aperture given by Abbe’s sine rule. However, a subtle yet important phenomena can be observed here, usually referred to as super-critical angle fluorescence (SAF) [31,32]. When the objective’s NA is higher than the refractive index of the sample media, a range of ${\theta _{imm}}$ propagation angles is available between the critical angle and the NA where $\cos ({\theta _m})$ becomes imaginary (creating evanescent waves which effectively limit the NA of the system). In PR techniques such as this, where the emitters are bound to the coverslip, the SAF is a significant part of the measured intensity [52].

The last component needed to describe the electrical field at the back focal plane, is the ray rotation matrix which is induced by the objective, assuming a perfect objective lens (full transmission):

$$R = \sqrt {\frac{{{n_{imm}}}}{{\cos ({\theta _{imm}})}}} \left( {\begin{array}{@{}ccc@{}} {{{\cos }^2}(\varphi )\cos ({\theta_{imm}}) + {{\sin }^2}(\varphi )}&{\frac{{\sin (2\varphi )}}{2}(\cos ({\theta_{imm}}) - 1)}&{ - \cos (\varphi )\sin ({\theta_{imm}})}\\ {\frac{{\sin (2\varphi )}}{2}(\cos ({\theta_{imm}}) - 1)}&{{{\sin }^2}(\varphi )\cos ({\theta_{imm}}) + {{\cos }^2}(\varphi )}&{ - \sin (\varphi )\sin ({\theta_{imm}})}\\ {\cos (\varphi )\sin ({\theta_{imm}})}&{\sin (\varphi )\sin ({\theta_{imm}})}&{\cos ({\theta_{imm}})} \end{array}} \right).$$
Finally, the electrical field in the back focal plane as we define in Eq. (3) are given by:
$$(R \cdot {G_{ff}}({r_0},r) \cdot \overrightarrow \mu ) \cdot S \buildrel \Delta \over = {e^{j\psi }} \cdot \left( {\left( {\begin{array}{ccc} {{g_{bfp,1}}}&{{g_{bfp,2}}}&{{g_{bfp,3}}}\\ {{g_{bfp,4}}}&{{g_{bfp,5}}}&{{g_{bfp,6}}}\\ 0&0&0 \end{array}} \right)\overrightarrow \mu } \right) \cdot S,$$
Eq. (18) describes the case of fixed dipoles. Simulating the freely rotating case is simply achieved by super-position (Eq. (2)).

In the case of the scalar approximation, there is only a single component of the electrical field and all the polarization effects are ignored. To relate between the two models, the scalar model can be described by ${g_{bfp,1}} = \sqrt {\frac{{{n_{imm}}}}{{\cos ({\theta _{imm}})}}}$:

$${E_{scalar}} = {g_{bfp,1}} \cdot S \cdot {e^{j\psi }}.$$

Appendix B

To compare between the two MLE cost functions (Poisson and Gaussian) we used the retrieved Tetrapod phase mask (Fig. 2(a)) to localize the z-stack (Fig. 5(c)). Evaluation of the robustness to noise was achieved by adding artificial noise (similar to Section 2.2). We found that the Poisson MLE deteriorates faster than the Gaussian MLE under the addition of artificial noise (Fig. 5(a)) and even under photo-bleaching, which can be seen in Fig. 5(b) where the Poisson MLE has a trend in the precision over the course of a single measurement.

 figure: Fig. 5.

Fig. 5. (a) Precision error on the full axial range as a function of added noise and (b) Precision error in each z position at a single noise multiplier of 12. (c) Full axial range reconstruction with ten images per focal position (510 images) using the different cost functions.

Download Full Size | PPT Slide | PDF

The empirical data suggests that the Gaussian MLE cost is more suitable for localization and PR at high SNR values than Poisson MLE. We attribute this to the apparent sensitivity of the Poisson MLE to exact knowledge of emitter intensity. The derivative of ${C_{G,NLL}}$ is more regularized than the derivative of ${C_{P,NLL}}$ (Eq. (20)), which is crucial because in practice the intensity is estimated from the acquired images, and is not precisely known.

$$\begin{array}{l} \frac{{\partial {C_{P,NLL}}}}{{\partial I_i(n)}} = \frac{1}{{NQ}}\sum\limits_{i = 1}^Q {\sum\limits_{n = 1}^N {1 - \frac{{{Z_i}(n)}}{{I_i(n)}}} } \\ \frac{{\partial {C_{G,NLL}}}}{{\partial I_i(n)}} = \frac{1}{{NQ}}\sum\limits_{i = 1}^Q {\sum\limits_{n = 1}^N {\frac{1}{{(I_i(n) + {\sigma _i}^2)}} - [\frac{{2[{Z_i}(n) - I_i(n)]}}{{(I_i(n) + {\sigma _i}^2)}} + \frac{{{{[{Z_i}(n) - I_i(n)]}^2}}}{{{{(I_i(n) + {\sigma _i}^2)}^2}}}]} } . \end{array}$$
In this work we have compared our results to Zernike polynomial fitting and to using a similar derivation as VIPR with the scalar diffraction model. As stated in our introduction, another common approach is applying variants of the Gerchberg-Saxton algorithm. We note that the scalar approach is equivalent to these methods if the cost function is the L2 norm (of the difference between the simulated and acquired images).

Appendix C

This appendix provides with additional information to Sections 2.3, 2.4 and Fig. 2. Figure 6 describes the results of localizing a short z-stack (a second z-stack taken on the calibration bead) with the three retrieved phase mask from the described methods in Section 2.4. A short z-stack will endure less optical drift and can be used to estimate the accuracy of the PR method. Figure 6(a) shows the difference in the axial position estimation (with MLE fitting) and the stage read where the sole difference is the phase mask. Figures 6(b)–6(c) shows that the wobble was incorporated into the phase mask using our method (with the vectorial and scalar models) compared to the Zernike PR (green) which shows wobble.

 figure: Fig. 6.

Fig. 6. (a) Axial localization mismatch between MLE fit with the three PR masks and the stage reading. (b-c) Lateral localization estimation shows the incorporation of wobble into the phase mask using the pixel-wise approach.

Download Full Size | PPT Slide | PDF

Wobble poses two separate issues for Zernike polynomial fitting: (1) A robust and accurate fit requires the wobble calibration before the optimization (to center the PSFs); unfortunately, a reliable wobble fit needs a matching PSF generator model. Thus, Zernike fitting requires a good initial guess to fit the wobble or an implementation of a slow bootstrap approach to optimize both. (2) The calibration is performed on the coverslip, so implementation in volumetric imaging will require some interpolation or scaling to match the different phase accumulations (immersion oil and sample medium).

Figure 7 describes the localization results of Fig. 2(b) (without additional noise). From these results, we observe that even a small mismatch in the PSF can lead to large deviations in localization precision at certain positions. These are the positions in which the SAF light produces a sharp point (as it is unmodulated by the phase mask). VIPR incorporates SAF into the derivation and thus suffers less precision penalty in this region.

 figure: Fig. 7.

Fig. 7. (a) Precision per focal position of the long z-stack with no additional noise. (b) Average 2D correlation between the measured PSF and the PR prediction per focal position.

Download Full Size | PPT Slide | PDF

Appendix D

In this appendix we demonstrate further generality results, in continuation to Section 3.2. Figure 8 shows the retrieval results on a $4\; \mu m\; $ Tetrapod phase mask which is severely misaligned as described in Section 3.2.

 figure: Fig. 8.

Fig. 8. (top row) – Theoretical misaligned Tetrapod phase mask and experimental images. (middle row) - The retrieved phase mask using VIPR and the resulting PSF simulations. (bottom row) - The retrieved phase mask using centered Zernike polynomials and the resulting PSF simulations.

Download Full Size | PPT Slide | PDF

Figure 9 shows the retrieval results on a DH phase mask, similar to Fig. 2(a). We note that while the DH is not based on Zernike polynomials like the Tetrapod mask, it can still be corrected using Zernike polynomials if we have the prior knowledge of the theoretical phase mask.

 figure: Fig. 9.

Fig. 9. (a) First row is the theoretical simulation of the designed DH phase mask (left) and PSFs (right). Second row shows the calibration measurements, which were used for PR, acquired by imaging 40 nm fluorescent beads. The next 3 rows show the simulated model using the reconstructed phase masks (left) and PSFs from the tested methods, e.g. VIPR, Scalar PR (using our method with a scalar model) and adding Zernike aberrations to the vectorial model, using the prior knowledge of the designed phase mask.

Download Full Size | PPT Slide | PDF

Appendix E

In this appendix we derive an analytical gradient for the case of a fixed dipole, represented by the vector $\overrightarrow \mu = ({\mu _1},{\mu _2},{\mu _3})$. In this case, Eq. (1) is replaced by:

$$I(x,y;\alpha ,\vec{q},P,{\sigma _B}) = (|\sum\limits_{a = 1}^3 {{g_{img,a}}(x,y;\alpha ,\vec{q},P)} {\mu _a}{|^2} + |\sum\limits_{b = 4}^6 {{g_{img,b}}(x,y;\alpha ,\vec{q},P)} {\mu _{b - 3}}{|^2})\otimes B.$$
To generate a gradient ($\frac{{\partial C(I_i(n),{Z_i}(n),{\sigma _i})}}{{\partial P(m)}}$), the only change required is deriving Eq. (21). Thus, Eq. (9) is replaced by Eq. (22):
$$\begin{array}{l} \frac{{\partial I_i(n)}}{{\partial P(m)}} = 2{\mathop{\rm Re}\nolimits} \{ \sum\limits_{a = 1}^3 {[\frac{{\partial {g_{img,a}}(n;\alpha ,\vec{q},P)}}{{\partial P(m)}}(\sum\limits_{\widetilde a = 1}^3 {\overline {{g_{img,\widetilde a}}} (n;\alpha ,\vec{q},P)} {\mu _{\widetilde a}})} ] + \\ \sum\limits_{b = 4}^6 {[\frac{{\partial {g_{img,b}}(n;\alpha ,\vec{q},{P_i})}}{{\partial P(m)}}(\sum\limits_{\widetilde b = 4}^6 {\overline {{g_{img,\widetilde b}}} (n;\alpha ,\vec{q},P)} {\mu _{\widetilde b - 3}})} ]\}\otimes B \end{array}.$$
All the subsequent derivatives and the Fourier transform representations are the same as in the freely rotating case. In Fig. 10, we show the theoretical capabilities of this method. We simulated a z-stack of a fixed dipole $\overrightarrow \mu = (1, - 1,1)$ with a linear polarizer and a Tetrapod phase mask. The proposed gradient in this section was used to recover the phase mask.

 figure: Fig. 10.

Fig. 10. First row is the simulated z-stack of a fixed dipole (right illustration) and a 4 $\; \mu m\; $ Tetrapod phase mask, the second row shows the reconstructed PSFs associated with the reconstructed phase mask on the left.

Download Full Size | PPT Slide | PDF

Appendix F

In this appendix, we describe the sample (Section 4) preparation and provide also with a demonstration of VIPR when used on an aligned $4\; \mu m$ Tetrapod phase mask. Sample preparation included cleaning 22X22 mm, 170 $\mu m$ thick coverslips in an ultrasonic bath with 5% Decon90 at 60°C for 30 min. Next, the coverslips were washed with water then incubated in ethanol absolute for 30 min and finally sterilized with 70% filtered ethanol for 30 min. COS7 cells were grown for 24 hours on the coverslips in 6-well plate in phenol red free Dulbecco's modified eagle medium (DMEM) With 1 g/l D-glucose (low glucose) supplemented with 10% fetal bovine serum, 100 U/ml penicillin 100 ug/ml streptomycin and 2 mM glutamine, at 37°C, and 5% CO2. The cells were fixed with 4% paraformaldehyde and 0.2% glutaraldehyde in PBS, pH 6.2, for 45 min, washed and incubated in 0.3M glycine/PBS solution for 10 minutes. The coverslips were transferred into a clean 6-well plate and incubated in a blocking solution (10% goat serum, 3% BSA, 2.2% glycine, and 0.1% Triton-X in PBS, filtered with 0.45 $\mu m$ PVDF filter unit, Millex) for 2 hours at 4°C. The cells were then immunostained overnight at 4°C with either anti TOMM20-AF647 (Fig. 11) antibody (Abcam, ab209606) or anti-$\alpha $ Tubulin-AF647 (Section 4) antibody (Abcam, ab190573) diluted 1:230 in the blocking buffer, and washed X5 with PBS.

For STORM, a PDMS chamber was attached to the glass coverslip containing fixed immunostained COS7 cells and placed in the microscope setup described in Fig. 1(a). The added blinking buffer was freshly made from 100 mM β-mercaptoethylamine hydrochloride, 20% sodium lactate, and 3% OxyFluor (Sigma, SAE0059), modified from [53]. A glass coverslip was placed on top to prevent evaporation. The sample was illuminated by a high-intensity 638nm 2000mW red dot laser module which passed through a 25 µm pinhole (Thorlabs). Emission light was filtered through a 500 nm Long pass dichroic and a 650 nm long pass (Chroma). 10K images were recorded on a sCMOS camera (Prime95B, Photometrics) instead of the EMCCD to reduce pixel size and enable a large field of view.

 figure: Fig. 11.

Fig. 11. (a) 3D super-resolved image (color encodes depth) of mitochondria in a COS7 cell achieved by training a deep-neural network on simulations of the phase retrieved mask. (i) and (ii) show XZ cross-sections of the white squares, where the hollow shapes of the mitochondria are clearly visible. (b) Two experimental frames (left) and their reconstructions using the phase mask from the PR (right).

Download Full Size | PPT Slide | PDF

Funding

European Research Council (802567); Technion-Israel Institute of Technology; Zuckerman Foundation.

Acknowledgments

We wish to thank Adam Backer for useful discussions. We thank Navitar Optical Solutions for the donation of a Pixelink camera which was used for system alignment.

Disclosures

The authors declare no conflicts of interest.

Code availability

The software will be available at Code 1 (Ref. [54]).

References

1. Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015). [CrossRef]  

2. R. P. Millane, “Phase retrieval in crystallography and optics,” J. Opt. Soc. Am. A 7(3), 394–411 (1990). [CrossRef]  

3. J. C. Dainty and J. R. Fienup, “Phase Retrieval and Image Reconstruction for Astronomy,” Image Recover. theory Appl. 231, 275 (1987).

4. L.-H. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, and L. Waller, “Experimental robustness of Fourier ptychography phase retrieval algorithms,” Opt. Express 23(26), 33214–33240 (2015). [CrossRef]  

5. L. Bian, J. Suo, G. Zheng, K. Guo, F. Chen, and Q. Dai, “Fourier ptychographic reconstruction using Wirtinger flow optimization,” Opt. Express 23(4), 4856–4866 (2015). [CrossRef]  

6. N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017). [CrossRef]  

7. M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007). [CrossRef]  

8. D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019). [CrossRef]  

9. D. Baddeley and J. Bewersdorf, “Biological Insight from Super-Resolution Microscopy: What We Can Learn from Localization-Based Images,” Annu. Rev. Biochem. 87(1), 965–989 (2018). [CrossRef]  

10. B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008). [CrossRef]  

11. Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014). [CrossRef]  

12. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009). [CrossRef]  

13. E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019). [CrossRef]  

14. Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016). [CrossRef]  

15. A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Colored point spread function engineering for parallel confocal microscopy,” Opt. Express 24(24), 27395–27402 (2016). [CrossRef]  

16. M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012). [CrossRef]  

17. A. S. Backer, “Computational inverse design for cascaded systems of metasurface optics,” Opt. Express 27(21), 30308–30331 (2019). [CrossRef]  

18. E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015). [CrossRef]  

19. R. W. Gerchberg and W. O. Saxton, “Practical algorithm for the determination of phase from image and diffraction plane pictures,” Opt. 35, 237–246 (1972).

20. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21(15), 2758–2769 (1982). [CrossRef]  

21. P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017). [CrossRef]  

22. P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018). [CrossRef]  

23. L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019). [CrossRef]  

24. W. Wang, F. Ye, H. Shen, N. A. Moringo, C. Dutta, J. T. Robinson, and C. F. Landes, “Generalized method to design phase masks for 3D super-resolution microscopy,” Opt. Express 27(3), 3799–3816 (2019). [CrossRef]  

25. A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018). [CrossRef]  

26. F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

27. O. Zhang and M. D. Lew, “Fundamental limits on measuring the rotational constraint of single molecules using fluorescence microscopy,” Phys. Rev. Lett. 122(19), 198301 (2019). [CrossRef]  

28. L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015). [CrossRef]  

29. M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013). [CrossRef]  

30. M. Siemons, C. N. Hulleman, RØ Thorsen, C. S. Smith, and S. Stallinga, “High precision wavefront control in point spread function engineering for single emitter localization,” Opt. Express 26(7), 8397–8416 (2018). [CrossRef]  

31. L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge university press, 2012). [CrossRef]  

32. A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014). [CrossRef]  

33. B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004). [CrossRef]  

34. P. Zelger, K. Kaser, B. Rossboth, L. Velas, G. J. Schütz, and A. Jesacher, “Three-dimensional localization microscopy using deep learning,” Opt. Express 26(25), 33166–33179 (2018). [CrossRef]  

35. R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004). [CrossRef]  

36. M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018). [CrossRef]  

37. J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018). [CrossRef]  

38. K. Kreutz-Delgado, “The Complex Gradient Operator and the CR-Calculus,” arXiv:0906.4835 (2009).

39. D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

40. P. Zhao and T. Zhang, “Stochastic Optimization with Importance Sampling,” international conference on machine learning., 1–9 (2015).

41. D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016). [CrossRef]  

42. S. Gopal, “Adaptive sampling for SGD by exploiting side information,” 33rd Int. Conf. Mach. Learn. ICML 2016, 364–372 (2016).

43. G. Zheng, X. Ou, R. Horstmeyer, and C. Yang, “Characterization of spatially varying aberrations for wide field-of-view microscopy,” Opt. Express 21(13), 15131–15143 (2013). [CrossRef]  

44. Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015). [CrossRef]  

45. S. Moser, M. Ritsch-Marte, and G. Thalhammer, “Model-based compensation of pixel crosstalk in liquid crystal spatial light modulators,” Opt. Express 27(18), 25046–25063 (2019). [CrossRef]  

46. E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

47. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006). [CrossRef]  

48. M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008). [CrossRef]  

49. Y. Li, Y. L. Wu, P. Hoess, M. Mund, and J. Ries, “Depth-dependent PSF calibration and aberration correction for 3D single-molecule localization,” Biomed. Opt. Express 10(6), 2708–2718 (2019). [CrossRef]  

50. J. Lin, O. G. Rodríguez-Herrera, F. Kenny, D. Lara, and J. C. Dainty, “Fast vectorial calculation of the volumetric focused field distribution by using a three-dimensional Fourier transform,” Opt. Express 20(2), 1060–1069 (2012). [CrossRef]  

51. J. W. Goodman, “Introduction to Fourier Optics 2nd edition,” (Roberts and Company Publishers, 1996).

52. B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018). [CrossRef]  

53. L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016). [CrossRef]  

54. B. Ferdman, “VIPR - Vectorial Phase Retrieval for microscopy”, github (2020) https://github.com/Borisfer/VIPR-Vectorial-Phase-Retrieval-for-microscopy

References

  • View by:

  1. Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
    [Crossref]
  2. R. P. Millane, “Phase retrieval in crystallography and optics,” J. Opt. Soc. Am. A 7(3), 394–411 (1990).
    [Crossref]
  3. J. C. Dainty and J. R. Fienup, “Phase Retrieval and Image Reconstruction for Astronomy,” Image Recover. theory Appl. 231, 275 (1987).
  4. L.-H. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, and L. Waller, “Experimental robustness of Fourier ptychography phase retrieval algorithms,” Opt. Express 23(26), 33214–33240 (2015).
    [Crossref]
  5. L. Bian, J. Suo, G. Zheng, K. Guo, F. Chen, and Q. Dai, “Fourier ptychographic reconstruction using Wirtinger flow optimization,” Opt. Express 23(4), 4856–4866 (2015).
    [Crossref]
  6. N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
    [Crossref]
  7. M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
    [Crossref]
  8. D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
    [Crossref]
  9. D. Baddeley and J. Bewersdorf, “Biological Insight from Super-Resolution Microscopy: What We Can Learn from Localization-Based Images,” Annu. Rev. Biochem. 87(1), 965–989 (2018).
    [Crossref]
  10. B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
    [Crossref]
  11. Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
    [Crossref]
  12. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
    [Crossref]
  13. E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019).
    [Crossref]
  14. Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
    [Crossref]
  15. A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Colored point spread function engineering for parallel confocal microscopy,” Opt. Express 24(24), 27395–27402 (2016).
    [Crossref]
  16. M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
    [Crossref]
  17. A. S. Backer, “Computational inverse design for cascaded systems of metasurface optics,” Opt. Express 27(21), 30308–30331 (2019).
    [Crossref]
  18. E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015).
    [Crossref]
  19. R. W. Gerchberg and W. O. Saxton, “Practical algorithm for the determination of phase from image and diffraction plane pictures,” Opt. 35, 237–246 (1972).
  20. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21(15), 2758–2769 (1982).
    [Crossref]
  21. P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
    [Crossref]
  22. P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
    [Crossref]
  23. L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019).
    [Crossref]
  24. W. Wang, F. Ye, H. Shen, N. A. Moringo, C. Dutta, J. T. Robinson, and C. F. Landes, “Generalized method to design phase masks for 3D super-resolution microscopy,” Opt. Express 27(3), 3799–3816 (2019).
    [Crossref]
  25. A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
    [Crossref]
  26. F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).
  27. O. Zhang and M. D. Lew, “Fundamental limits on measuring the rotational constraint of single molecules using fluorescence microscopy,” Phys. Rev. Lett. 122(19), 198301 (2019).
    [Crossref]
  28. L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
    [Crossref]
  29. M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013).
    [Crossref]
  30. M. Siemons, C. N. Hulleman, RØ Thorsen, C. S. Smith, and S. Stallinga, “High precision wavefront control in point spread function engineering for single emitter localization,” Opt. Express 26(7), 8397–8416 (2018).
    [Crossref]
  31. L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge university press, 2012).
    [Crossref]
  32. A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
    [Crossref]
  33. B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
    [Crossref]
  34. P. Zelger, K. Kaser, B. Rossboth, L. Velas, G. J. Schütz, and A. Jesacher, “Three-dimensional localization microscopy using deep learning,” Opt. Express 26(25), 33166–33179 (2018).
    [Crossref]
  35. R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004).
    [Crossref]
  36. M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018).
    [Crossref]
  37. J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018).
    [Crossref]
  38. K. Kreutz-Delgado, “The Complex Gradient Operator and the CR-Calculus,” arXiv:0906.4835 (2009).
  39. D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).
  40. P. Zhao and T. Zhang, “Stochastic Optimization with Importance Sampling,” international conference on machine learning., 1–9 (2015).
  41. D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016).
    [Crossref]
  42. S. Gopal, “Adaptive sampling for SGD by exploiting side information,” 33rd Int. Conf. Mach. Learn. ICML 2016, 364–372 (2016).
  43. G. Zheng, X. Ou, R. Horstmeyer, and C. Yang, “Characterization of spatially varying aberrations for wide field-of-view microscopy,” Opt. Express 21(13), 15131–15143 (2013).
    [Crossref]
  44. Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
    [Crossref]
  45. S. Moser, M. Ritsch-Marte, and G. Thalhammer, “Model-based compensation of pixel crosstalk in liquid crystal spatial light modulators,” Opt. Express 27(18), 25046–25063 (2019).
    [Crossref]
  46. E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).
  47. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
    [Crossref]
  48. M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
    [Crossref]
  49. Y. Li, Y. L. Wu, P. Hoess, M. Mund, and J. Ries, “Depth-dependent PSF calibration and aberration correction for 3D single-molecule localization,” Biomed. Opt. Express 10(6), 2708–2718 (2019).
    [Crossref]
  50. J. Lin, O. G. Rodríguez-Herrera, F. Kenny, D. Lara, and J. C. Dainty, “Fast vectorial calculation of the volumetric focused field distribution by using a three-dimensional Fourier transform,” Opt. Express 20(2), 1060–1069 (2012).
    [Crossref]
  51. J. W. Goodman, “Introduction to Fourier Optics 2nd edition,” (Roberts and Company Publishers, 1996).
  52. B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
    [Crossref]
  53. L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
    [Crossref]
  54. B. Ferdman, “VIPR - Vectorial Phase Retrieval for microscopy”, github (2020) https://github.com/Borisfer/VIPR-Vectorial-Phase-Retrieval-for-microscopy

2019 (8)

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019).
[Crossref]

A. S. Backer, “Computational inverse design for cascaded systems of metasurface optics,” Opt. Express 27(21), 30308–30331 (2019).
[Crossref]

L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019).
[Crossref]

W. Wang, F. Ye, H. Shen, N. A. Moringo, C. Dutta, J. T. Robinson, and C. F. Landes, “Generalized method to design phase masks for 3D super-resolution microscopy,” Opt. Express 27(3), 3799–3816 (2019).
[Crossref]

O. Zhang and M. D. Lew, “Fundamental limits on measuring the rotational constraint of single molecules using fluorescence microscopy,” Phys. Rev. Lett. 122(19), 198301 (2019).
[Crossref]

S. Moser, M. Ritsch-Marte, and G. Thalhammer, “Model-based compensation of pixel crosstalk in liquid crystal spatial light modulators,” Opt. Express 27(18), 25046–25063 (2019).
[Crossref]

Y. Li, Y. L. Wu, P. Hoess, M. Mund, and J. Ries, “Depth-dependent PSF calibration and aberration correction for 3D single-molecule localization,” Biomed. Opt. Express 10(6), 2708–2718 (2019).
[Crossref]

2018 (8)

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

M. Siemons, C. N. Hulleman, RØ Thorsen, C. S. Smith, and S. Stallinga, “High precision wavefront control in point spread function engineering for single emitter localization,” Opt. Express 26(7), 8397–8416 (2018).
[Crossref]

A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
[Crossref]

M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018).
[Crossref]

J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018).
[Crossref]

P. Zelger, K. Kaser, B. Rossboth, L. Velas, G. J. Schütz, and A. Jesacher, “Three-dimensional localization microscopy using deep learning,” Opt. Express 26(25), 33166–33179 (2018).
[Crossref]

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

D. Baddeley and J. Bewersdorf, “Biological Insight from Super-Resolution Microscopy: What We Can Learn from Localization-Based Images,” Annu. Rev. Biochem. 87(1), 965–989 (2018).
[Crossref]

2017 (2)

2016 (4)

D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Colored point spread function engineering for parallel confocal microscopy,” Opt. Express 24(24), 27395–27402 (2016).
[Crossref]

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

2015 (6)

E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015).
[Crossref]

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

L.-H. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, and L. Waller, “Experimental robustness of Fourier ptychography phase retrieval algorithms,” Opt. Express 23(26), 33214–33240 (2015).
[Crossref]

L. Bian, J. Suo, G. Zheng, K. Guo, F. Chen, and Q. Dai, “Fourier ptychographic reconstruction using Wirtinger flow optimization,” Opt. Express 23(4), 4856–4866 (2015).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
[Crossref]

2014 (2)

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref]

2013 (2)

M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013).
[Crossref]

G. Zheng, X. Ou, R. Horstmeyer, and C. Yang, “Characterization of spatially varying aberrations for wide field-of-view microscopy,” Opt. Express 21(13), 15131–15143 (2013).
[Crossref]

2012 (2)

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

J. Lin, O. G. Rodríguez-Herrera, F. Kenny, D. Lara, and J. C. Dainty, “Fast vectorial calculation of the volumetric focused field distribution by using a three-dimensional Fourier transform,” Opt. Express 20(2), 1060–1069 (2012).
[Crossref]

2009 (1)

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

2008 (2)

B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
[Crossref]

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

2007 (1)

M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
[Crossref]

2006 (1)

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref]

2004 (2)

R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004).
[Crossref]

B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
[Crossref]

1990 (1)

1987 (1)

J. C. Dainty and J. R. Fienup, “Phase Retrieval and Image Reconstruction for Astronomy,” Image Recover. theory Appl. 231, 275 (1987).

1982 (1)

1972 (1)

R. W. Gerchberg and W. O. Saxton, “Practical algorithm for the determination of phase from image and diffraction plane pictures,” Opt. 35, 237–246 (1972).

Agard, D. A.

B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
[Crossref]

Agrawal, A.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Agronskaia, A. V.

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

Alalouf, O.

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Archetti, A.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Aristov, A.

A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
[Crossref]

Ba, J.

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

Babcock, H.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Backer, A. S.

A. S. Backer, “Computational inverse design for cascaded systems of metasurface optics,” Opt. Express 27(21), 30308–30331 (2019).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref]

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Backlund, M. P.

M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018).
[Crossref]

M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013).
[Crossref]

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Baddeley, D.

D. Baddeley and J. Bewersdorf, “Biological Insight from Super-Resolution Microscopy: What We Can Learn from Localization-Based Images,” Annu. Rev. Biochem. 87(1), 965–989 (2018).
[Crossref]

Bates, M.

B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
[Crossref]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref]

Bernet, S.

Bewersdorf, J.

D. Baddeley and J. Bewersdorf, “Biological Insight from Super-Resolution Microscopy: What We Can Learn from Localization-Based Images,” Annu. Rev. Biochem. 87(1), 965–989 (2018).
[Crossref]

Bi, C.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Bian, L.

Biteen, J. S.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Booth, M. J.

M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
[Crossref]

Broertjes, J.

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

Bu, Y.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Calve, S.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Candès, E. J.

E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015).
[Crossref]

Carlini, L.

L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
[Crossref]

Chao, J.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Chapman, H. N.

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

Chaurasia, A.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

Chen, F.

Chen, M.

Cohen, O.

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

Colabrese, S.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Culurciello, E.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

Dai, Q.

Dainty, J. C.

Den Van Broek, B.

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

Dong, J.

Douglass, K. M.

L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
[Crossref]

Dutta, C.

Eldar, Y. C.

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

Ferdman, B.

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

B. Ferdman, “VIPR - Vectorial Phase Retrieval for microscopy”, github (2020) https://github.com/Borisfer/VIPR-Vectorial-Phase-Retrieval-for-microscopy

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Fienup, J. R.

J. C. Dainty and J. R. Fienup, “Phase Retrieval and Image Reconstruction for Astronomy,” Image Recover. theory Appl. 231, 275 (1987).

J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21(15), 2758–2769 (1982).
[Crossref]

Freedman, D.

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Gerchberg, R. W.

R. W. Gerchberg and W. O. Saxton, “Practical algorithm for the determination of phase from image and diffraction plane pictures,” Opt. 35, 237–246 (1972).

Goodman, J. W.

J. W. Goodman, “Introduction to Fourier Optics 2nd edition,” (Roberts and Company Publishers, 1996).

Gopal, S.

S. Gopal, “Adaptive sampling for SGD by exploiting side information,” 33rd Int. Conf. Mach. Learn. ICML 2016, 364–372 (2016).

Gordon, R.

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Grover, G.

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Guo, K.

Gustafsson, M. G. L.

B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
[Crossref]

Hagen, G. M.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Haimovich, Y.

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

Hanser, B. M.

B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
[Crossref]

Hecht, B.

L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge university press, 2012).
[Crossref]

Heilemann, M.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Henriques, R.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Herbert, A.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Hershko, E.

Hoess, P.

Holden, S.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Holden, S. J.

L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
[Crossref]

Horstmeyer, R.

Huang, B.

B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
[Crossref]

Huang, F.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Hulleman, C. N.

Jalink, K.

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

Jesacher, A.

Ji, N.

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
[Crossref]

Kaser, K.

Kasper, R.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Kenny, F.

Kingma, D. P.

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

Kreutz-Delgado, K.

K. Kreutz-Delgado, “The Complex Gradient Operator and the CR-Calculus,” arXiv:0906.4835 (2009).

Kwok, T.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Landes, C. F.

Landreth, G. E.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Lara, D.

Lee, M. Y.

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

Lelandais, B.

A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
[Crossref]

Lew, M. D.

O. Zhang and M. D. Lew, “Fundamental limits on measuring the rotational constraint of single molecules using fluorescence microscopy,” Phys. Rev. Lett. 122(19), 198301 (2019).
[Crossref]

M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013).
[Crossref]

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Li, X.

E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015).
[Crossref]

Li, Y.

Lin, J.

Liu, N.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Liu, S.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Lord, S. J.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Lukes, T.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Ma, D.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

MacPherson, K. P.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Manley, S.

L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
[Crossref]

Miao, J.

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

Michaeli, T.

E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019).
[Crossref]

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Millane, R. P.

Mlodzianoski, M. J.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

Möckl, L.

L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019).
[Crossref]

Moerner, W. E.

L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019).
[Crossref]

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref]

M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013).
[Crossref]

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Moringo, N. A.

Moser, S.

Mukherjee, A.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Mund, M.

Nahidiazar, L.

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

Needell, D.

D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016).
[Crossref]

Nehme, E.

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Novotny, L.

L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge university press, 2012).
[Crossref]

Ober, R.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Ober, R. J.

R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004).
[Crossref]

Orange, R.

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Ou, X.

Pavani, S. R. P.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Pengo, T.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Petrov, P. N.

L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019).
[Crossref]

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref]

Pham, T. A.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Piestun, R.

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Qu, Q.

J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018).
[Crossref]

Ram, S.

R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004).
[Crossref]

Rensen, E.

A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
[Crossref]

Rieger, B.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Ries, J.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Y. Li, Y. L. Wu, P. Hoess, M. Mund, and J. Ries, “Depth-dependent PSF calibration and aberration correction for 3D single-molecule localization,” Biomed. Opt. Express 10(6), 2708–2718 (2019).
[Crossref]

Ritsch-Marte, M.

Robinson, J. T.

Rodríguez-Herrera, O. G.

Rossboth, B.

Rust, M. J.

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref]

Sage, D.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Sahl, S. J.

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref]

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Sauer, M.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Saxton, W. O.

R. W. Gerchberg and W. O. Saxton, “Practical algorithm for the determination of phase from image and diffraction plane pictures,” Opt. 35, 237–246 (1972).

Schüttpelz, M.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Schütz, G. J.

Sedat, J. W.

B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
[Crossref]

Seefeldt, B.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Segev, M.

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

Shechtman, Y.

E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019).
[Crossref]

M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018).
[Crossref]

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref]

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Shen, H.

Sibarita, J. B.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Siemons, M.

Smith, C. S.

Soltanolkotabi, M.

E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015).
[Crossref]

L.-H. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, and L. Waller, “Experimental robustness of Fourier ptychography phase retrieval algorithms,” Opt. Express 23(26), 33214–33240 (2015).
[Crossref]

Srebro, N.

D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016).
[Crossref]

Stallinga, S.

Sun, J.

J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018).
[Crossref]

Suo, J.

Tang, G.

Thalhammer, G.

Thompson, M. A.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Thorsen, RØ

Tian, L.

Tinnefeld, P.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Twieg, R. J.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Unser, M.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Van De Linde, S.

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Velas, L.

Velmurugan, R.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Waller, L.

Walsworth, R. L.

M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018).
[Crossref]

Wang, W.

W. Wang, F. Ye, H. Shen, N. A. Moringo, C. Dutta, J. T. Robinson, and C. F. Landes, “Generalized method to design phase masks for 3D super-resolution microscopy,” Opt. Express 27(3), 3799–3816 (2019).
[Crossref]

B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
[Crossref]

Wang, Y.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Ward, E. S.

R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004).
[Crossref]

Ward, R.

D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016).
[Crossref]

Weiss, L. E.

E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019).
[Crossref]

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

Wheeler, A.

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

Wright, J.

J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018).
[Crossref]

Wu, Y. L.

Xu, F.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Yang, C.

Ye, F.

Yeh, L.-H.

Yin, P.

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

Zelger, P.

Zhang, O.

O. Zhang and M. D. Lew, “Fundamental limits on measuring the rotational constraint of single molecules using fluorescence microscopy,” Phys. Rev. Lett. 122(19), 198301 (2019).
[Crossref]

Zhang, P.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

Zhang, T.

P. Zhao and T. Zhang, “Stochastic Optimization with Importance Sampling,” international conference on machine learning., 1–9 (2015).

Zhao, P.

P. Zhao and T. Zhang, “Stochastic Optimization with Importance Sampling,” international conference on machine learning., 1–9 (2015).

Zheng, G.

Zhong, J.

Zhuang, X.

B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
[Crossref]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref]

Zimmer, C.

A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
[Crossref]

ACS Nano (1)

B. Ferdman, L. E. Weiss, O. Alalouf, Y. Haimovich, and Y. Shechtman, “Ultrasensitive Refractometry via Supercritical Angle Fluorescence,” ACS Nano 12(12), 11892–11898 (2018).
[Crossref]

Angew. Chem., Int. Ed. (1)

M. Heilemann, S. Van De Linde, M. Schüttpelz, R. Kasper, B. Seefeldt, A. Mukherjee, P. Tinnefeld, and M. Sauer, “Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes,” Angew. Chem., Int. Ed. 47(33), 6172–6176 (2008).
[Crossref]

Annu. Rev. Biochem. (1)

D. Baddeley and J. Bewersdorf, “Biological Insight from Super-Resolution Microscopy: What We Can Learn from Localization-Based Images,” Annu. Rev. Biochem. 87(1), 965–989 (2018).
[Crossref]

Appl. Opt. (1)

Appl. Phys. Lett. (1)

L. Möckl, P. N. Petrov, and W. E. Moerner, “Accurate phase retrieval of complex point spread functions with deep residual neural networks,” Appl. Phys. Lett. 115(25), 251106 (2019).
[Crossref]

Biomed. Opt. Express (1)

Biophys. J. (1)

R. J. Ober, S. Ram, and E. S. Ward, “Localization accuracy in single-molecule microscopy,” Biophys. J. 86(2), 1185–1200 (2004).
[Crossref]

Found Comput Math (1)

J. Sun, Q. Qu, and J. Wright, “A Geometric Analysis of Phase Retrieval,” Found Comput Math 18(5), 1131–1198 (2018).
[Crossref]

IEEE Signal Process. Mag. (1)

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase Retrieval with Application to Optical Imaging: A contemporary overview,” IEEE Signal Process. Mag. 32(3), 87–109 (2015).
[Crossref]

IEEE Trans. Inf. Theory (1)

E. J. Candès, X. Li, and M. Soltanolkotabi, “Phase retrieval via wirtinger flow: Theory and algorithms,” IEEE Trans. Inf. Theory 61(4), 1985–2007 (2015).
[Crossref]

Image Recover. theory Appl. (1)

J. C. Dainty and J. R. Fienup, “Phase Retrieval and Image Reconstruction for Astronomy,” Image Recover. theory Appl. 231, 275 (1987).

J. Microsc. (1)

B. M. Hanser, M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Phase-retrieved pupil functions in wide-field fluorescence microscopy,” J. Microsc. 216(1), 32–48 (2004).
[Crossref]

J. Opt. Soc. Am. A (1)

J. Phys. Chem. B (1)

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref]

Math. Program. (1)

D. Needell, N. Srebro, and R. Ward, “Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm,” Math. Program. 155(1-2), 549–573 (2016).
[Crossref]

Nano Lett. (2)

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref]

M. D. Lew, M. P. Backlund, and W. E. Moerner, “Rotational mobility of single molecules affects localization accuracy in super-resolution fluorescence microscopy,” Nano Lett. 13(9), 3967–3972 (2013).
[Crossref]

Nat. Commun. (1)

A. Aristov, B. Lelandais, E. Rensen, and C. Zimmer, “ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range,” Nat. Commun. 9(1), 2409 (2018).
[Crossref]

Nat. Methods (4)

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref]

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
[Crossref]

D. Sage, T. A. Pham, H. Babcock, T. Lukes, T. Pengo, J. Chao, R. Velmurugan, A. Herbert, A. Agrawal, S. Colabrese, A. Wheeler, A. Archetti, B. Rieger, R. Ober, G. M. Hagen, J. B. Sibarita, J. Ries, R. Henriques, M. Unser, and S. Holden, “Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software,” Nat. Methods 16(5), 387–395 (2019).
[Crossref]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref]

Nat. Photonics (1)

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref]

Opt. (1)

R. W. Gerchberg and W. O. Saxton, “Practical algorithm for the determination of phase from image and diffraction plane pictures,” Opt. 35, 237–246 (1972).

Opt. Express (12)

A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Colored point spread function engineering for parallel confocal microscopy,” Opt. Express 24(24), 27395–27402 (2016).
[Crossref]

E. Hershko, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Multicolor localization microscopy and point-spread-function engineering by deep learning,” Opt. Express 27(5), 6158–6183 (2019).
[Crossref]

L.-H. Yeh, J. Dong, J. Zhong, L. Tian, M. Chen, G. Tang, M. Soltanolkotabi, and L. Waller, “Experimental robustness of Fourier ptychography phase retrieval algorithms,” Opt. Express 23(26), 33214–33240 (2015).
[Crossref]

L. Bian, J. Suo, G. Zheng, K. Guo, F. Chen, and Q. Dai, “Fourier ptychographic reconstruction using Wirtinger flow optimization,” Opt. Express 23(4), 4856–4866 (2015).
[Crossref]

A. S. Backer, “Computational inverse design for cascaded systems of metasurface optics,” Opt. Express 27(21), 30308–30331 (2019).
[Crossref]

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref]

W. Wang, F. Ye, H. Shen, N. A. Moringo, C. Dutta, J. T. Robinson, and C. F. Landes, “Generalized method to design phase masks for 3D super-resolution microscopy,” Opt. Express 27(3), 3799–3816 (2019).
[Crossref]

M. Siemons, C. N. Hulleman, RØ Thorsen, C. S. Smith, and S. Stallinga, “High precision wavefront control in point spread function engineering for single emitter localization,” Opt. Express 26(7), 8397–8416 (2018).
[Crossref]

P. Zelger, K. Kaser, B. Rossboth, L. Velas, G. J. Schütz, and A. Jesacher, “Three-dimensional localization microscopy using deep learning,” Opt. Express 26(25), 33166–33179 (2018).
[Crossref]

S. Moser, M. Ritsch-Marte, and G. Thalhammer, “Model-based compensation of pixel crosstalk in liquid crystal spatial light modulators,” Opt. Express 27(18), 25046–25063 (2019).
[Crossref]

J. Lin, O. G. Rodríguez-Herrera, F. Kenny, D. Lara, and J. C. Dainty, “Fast vectorial calculation of the volumetric focused field distribution by using a three-dimensional Fourier transform,” Opt. Express 20(2), 1060–1069 (2012).
[Crossref]

G. Zheng, X. Ou, R. Horstmeyer, and C. Yang, “Characterization of spatially varying aberrations for wide field-of-view microscopy,” Opt. Express 21(13), 15131–15143 (2013).
[Crossref]

Philos. Trans. R. Soc., A (1)

M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
[Crossref]

Phys. Rev. Lett. (3)

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref]

M. P. Backlund, Y. Shechtman, and R. L. Walsworth, “Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics,” Phys. Rev. Lett. 121(2), 023904 (2018).
[Crossref]

O. Zhang and M. D. Lew, “Fundamental limits on measuring the rotational constraint of single molecules using fluorescence microscopy,” Phys. Rev. Lett. 122(19), 198301 (2019).
[Crossref]

PLoS One (2)

L. Carlini, S. J. Holden, K. M. Douglass, and S. Manley, “Correction of a depth-dependent lateral distortion in 3D super-resolution imaging,” PLoS One 10(11), e0142949 (2015).
[Crossref]

L. Nahidiazar, A. V. Agronskaia, J. Broertjes, B. Den Van Broek, and K. Jalink, “Optimizing imaging conditions for demanding multi-color super resolution localization microscopy,” PLoS One 11(7), e0158884–18 (2016).
[Crossref]

Proc. Natl. Acad. Sci. (1)

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. 106(9), 2995–2999 (2009).
[Crossref]

Proc. Natl. Acad. Sci. U. S. A. (1)

M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U. S. A. 109(47), 19087–19092 (2012).
[Crossref]

Science (1)

B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy,” Science 319(5864), 810–813 (2008).
[Crossref]

Other (9)

L. Novotny and B. Hecht, Principles of Nano-Optics (Cambridge university press, 2012).
[Crossref]

F. Xu, D. Ma, K. P. MacPherson, S. Liu, Y. Bu, Y. Wang, C. Bi, T. Kwok, P. Yin, S. Calve, G. E. Landreth, and F. Huang, “Three dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval,” bioRxiv, 727354 (2019).

B. Ferdman, “VIPR - Vectorial Phase Retrieval for microscopy”, github (2020) https://github.com/Borisfer/VIPR-Vectorial-Phase-Retrieval-for-microscopy

E. Nehme, D. Freedman, R. Gordon, B. Ferdman, L. E. Weiss, O. Alalouf, R. Orange, T. Michaeli, and Y. Shechtman, “ Dense three dimensional localization microscopy by deep learning,” arXiv:1906.09957 (2019).

J. W. Goodman, “Introduction to Fourier Optics 2nd edition,” (Roberts and Company Publishers, 1996).

S. Gopal, “Adaptive sampling for SGD by exploiting side information,” 33rd Int. Conf. Mach. Learn. ICML 2016, 364–372 (2016).

K. Kreutz-Delgado, “The Complex Gradient Operator and the CR-Calculus,” arXiv:0906.4835 (2009).

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

P. Zhao and T. Zhang, “Stochastic Optimization with Importance Sampling,” international conference on machine learning., 1–9 (2015).

Supplementary Material (2)

NameDescription
Visualization 1       Phase retrieval of a 4um axial range Tetrapod PSF using 3 methods: Scalar, Zernike, VIPR.
Visualization 2       Phase retrieval of a Double Helix PSF using 3 methods: Scalar, Zernike, VIPR.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1. (a) Standard PSF engineering optical setup. The emission from a fluorescent molecule is collected by a high NA oil-immersed objective. A 4-f Fourier processing setup extends the optical path and enables the introduction of a phase mask, implemented via LC-SLM which modulates only the polarization transmitted by the Linear Polarizer (LP). (b) (Experimentally measured) A fluorescent bead scanned over 3 µm, imaged with a designed phase mask (using our method) which encodes a four letter PSF at different focal plane positions - demonstrates the generality of pixel-wise PSF engineering. (c) Schematic of our simulation model described in Section 2.1 where the position of the molecule and the phase mask define the PSF.
Fig. 2.
Fig. 2. (a) Optimization evolution of the PR mask. (b) First row is the theoretical simulation of the designed phase mask (left) and PSFs (right). Second row shows the calibration measurements, which were used for PR, acquired by imaging 40 nm fluorescent beads. The next three rows show the simulated model using the reconstructed phase masks (left) and PSFs from the tested methods, e.g. VIPR (blue), Scalar PR (using our method with a scalar model-red) without the SAF light modulation (hence the clear annulus around the PR mask) and adding Zernike aberrations to the vectorial model (green), using the prior knowledge of the designed phase mask. The most dominant aberrations where spherical, of the second and fourth order. (c) Precision error on the full axial range as a function of added noise and (d) precision error in each z position at a single noise multiplier (24). (e) Full axial range reconstruction with ten images per focal position (510 images) using each of the retrieved phase masks. Zoom-ins shows that the Scalar PR (red) suffers from a bias in the axial range scaling. The orange outline shows a single PSF and MLE fits using each of the phase masks.
Fig. 3.
Fig. 3. (a) Top row – theoretical phase mask, created from low amplitude combination of random Zernike polynomials and acquired images used to reconstruct the second row using VIPR. (b) Top row – theoretical machine learned phase mask and acquired images used to reconstruct the second row using VIPR.
Fig. 4.
Fig. 4. (a) Two example experimental frames taken from a microtubule blinking experiment covering a range of ∼ $3.5\; \mu m$ . Distinct PSFs (A-F) from various axial positions were picked by a localization with deep-learning net that was trained on: (d) the phase retrieved mask (by VIPR) from a coverslip z-stack with FluoSpheres(625/645). (b) Simulated PSFs at the axial position matching to emitters A-F using the PR mask with VIPR. (c) Average PSFs from 10 K frames in a range of ${\pm} 25\; nm$ around the stated axial position. (e) Measured PSF in the top of the range with the matching simulation and a spline fit performed on the coverslip data. The difference in shapes shows the importance of incorporating the refractive index mismatch in the optical model as PSFs far from the coverslip experience large deviations from the z-stack data.
Fig. 5.
Fig. 5. (a) Precision error on the full axial range as a function of added noise and (b) Precision error in each z position at a single noise multiplier of 12. (c) Full axial range reconstruction with ten images per focal position (510 images) using the different cost functions.
Fig. 6.
Fig. 6. (a) Axial localization mismatch between MLE fit with the three PR masks and the stage reading. (b-c) Lateral localization estimation shows the incorporation of wobble into the phase mask using the pixel-wise approach.
Fig. 7.
Fig. 7. (a) Precision per focal position of the long z-stack with no additional noise. (b) Average 2D correlation between the measured PSF and the PR prediction per focal position.
Fig. 8.
Fig. 8. (top row) – Theoretical misaligned Tetrapod phase mask and experimental images. (middle row) - The retrieved phase mask using VIPR and the resulting PSF simulations. (bottom row) - The retrieved phase mask using centered Zernike polynomials and the resulting PSF simulations.
Fig. 9.
Fig. 9. (a) First row is the theoretical simulation of the designed DH phase mask (left) and PSFs (right). Second row shows the calibration measurements, which were used for PR, acquired by imaging 40 nm fluorescent beads. The next 3 rows show the simulated model using the reconstructed phase masks (left) and PSFs from the tested methods, e.g. VIPR, Scalar PR (using our method with a scalar model) and adding Zernike aberrations to the vectorial model, using the prior knowledge of the designed phase mask.
Fig. 10.
Fig. 10. First row is the simulated z-stack of a fixed dipole (right illustration) and a 4 $\; \mu m\; $ Tetrapod phase mask, the second row shows the reconstructed PSFs associated with the reconstructed phase mask on the left.
Fig. 11.
Fig. 11. (a) 3D super-resolved image (color encodes depth) of mitochondria in a COS7 cell achieved by training a deep-neural network on simulations of the phase retrieved mask. (i) and (ii) show XZ cross-sections of the white squares, where the hollow shapes of the mitochondria are clearly visible. (b) Two experimental frames (left) and their reconstructions using the phase mask from the PR (right).

Tables (1)

Tables Icon

Table 1. Average run time for a gradient calculation using the analytical gradient vs. a numerical gradient

Equations (24)

Equations on this page are rendered with MathJax. Learn more.

I ( x , y ; α , q , P , σ B ) = w | g i m g , w ( x , y ; α , q , P ) | 2 B ,
g i m g , w ( x , y ; α , q , P ) = F 2 D [ E b f p , w ( x ^ , y ^ ; α , q , P ) ] ,
E b f p , w ( x ^ , y ^ ; α , q , P ) = g b f p , w ( x ^ , y ^ ; α , q ) S ( x ^ , y ^ ; α , q ) exp [ j ψ ( x ^ , y ^ ; α , q , P ) ] ,
ψ ( x ^ , y ^ ; α , q , P ) = P ( x ^ , y ^ ) + l = 1 4 ϕ l ( x ^ , y ^ ; α ) q ( l ) ,
C P , N L L [ I ( n ; P , q i , σ B ) , Z i ( n ) ] = 1 N Q i = 1 Q n = 1 N Z i ( n ) l o g [ I ( n ; P , q i , σ B ) ] I ( n ; P , q i , σ B ) .
I ~ ( n ; P , q i , σ B ) + b g ( n ) N ( I ( n ; P , q i , σ B ) , I ( n ; P , q i , σ B ) + σ i 2 ) .
C G , N L L [ I ( n ; P , q i , σ B ) , Z i ( n ) , σ i ] = 1 2 N Q i = 1 Q n = 1 N l o g [ 2 π ( I ( n ; P , q i , σ B ) + σ i 2 ) ] + [ Z i ( n ) I ( n ; P , q i , σ B ) ] 2 I ( n ; P , q i , σ B ) + σ i 2 .
C I i ( n ) = 1 N f [ I i ( n ) , Z i ( n ) , σ i ] I i ( n ) ,
I i ( n ) P ( m ) = 2 Re { w [ g i m g , w ( n ; P , q i ) P ( m ) g i m g , w ¯ ( n ; P , q i ) ] B } ,
g i m g , w ( n ; P , q i ) P ( m ) = P ( m ) F m ~ [ E b f p , w ( m ~ ; P , q i ) ] = = P ( m ) m ~ = 1 M exp ( j 2 π n m ~ M ) [ E b f p , w ( m ~ ; P , q i ) ] = j exp ( j 2 π n m ~ M ) E b f p , w ( m ~ ; P , q i ) δ m , m ~ ,
C P ( m ) = 2 N w Re ( g i m g , w ( n ; P , q i ) P ( m ) { g i m g , w ¯ ( n ; P , q i ) B f [ I i ( n ) , Z i ( n ) , σ i ] I i ( n ) } ) .
C P ( m ) = 2 N w Re ( E b f p , w ( m ; P , q i ) F n [ j g i m g , w ¯ ( n ; P , q i ) f ( I i ( n ) , Z i ( n ) , σ i ) I i ( n ) B ] ) .
G f f ( r 0 , r ) = A e j ψ ( cos 2 ( φ ) z 2 | | r | | 2 Φ ( 2 ) + sin 2 ( φ ) Φ ( 3 ) sin ( φ ) cos ( φ ) ( z 2 | | r | | 2 Φ ( 2 ) Φ ( 3 ) ) ρ cos ( φ ) z | | r | | 2 Φ ( 1 ) sin ( φ ) cos ( φ ) ( z 2 | | r | | 2 Φ ( 2 ) Φ ( 3 ) ) sin 2 ( φ ) z 2 | | r | | 2 Φ ( 2 ) + cos 2 ( φ ) Φ ( 3 ) ρ sin ( φ ) z | | r | | 2 Φ ( 1 ) ρ cos ( φ ) z | | r | | 2 Φ ( 2 ) ρ sin ( φ ) z | | r | | 2 Φ ( 2 ) ( 1 z 2 | | r | | 2 ) Φ ( 1 ) ) ,
Φ ( 1 ) = ( n i m m n m ) 2 cos ( θ i m m ) cos ( θ m ) t p ( θ m , n i m m , n m ) ,
Φ ( 2 ) = n i m m n m t p ( θ m , n i m m , n m ) ,
Φ ( 3 ) = ( n i m m n m ) cos ( θ i m m ) cos ( θ m ) t s ( θ m , n i m m , n m ) .
ϕ 1 ( x ^ , y ^ ) = k M f x ^ , ϕ 2 ( x ^ , y ^ ) = k M f y ^ ,
ϕ 3 ( x ^ , y ^ ) = k n m cos ( θ m ) , ϕ 4 ( x ^ , y ^ ) = k n i m m cos ( θ i m m ) .
R = n i m m cos ( θ i m m ) ( cos 2 ( φ ) cos ( θ i m m ) + sin 2 ( φ ) sin ( 2 φ ) 2 ( cos ( θ i m m ) 1 ) cos ( φ ) sin ( θ i m m ) sin ( 2 φ ) 2 ( cos ( θ i m m ) 1 ) sin 2 ( φ ) cos ( θ i m m ) + cos 2 ( φ ) sin ( φ ) sin ( θ i m m ) cos ( φ ) sin ( θ i m m ) sin ( φ ) sin ( θ i m m ) cos ( θ i m m ) ) .
( R G f f ( r 0 , r ) μ ) S = Δ e j ψ ( ( g b f p , 1 g b f p , 2 g b f p , 3 g b f p , 4 g b f p , 5 g b f p , 6 0 0 0 ) μ ) S ,
E s c a l a r = g b f p , 1 S e j ψ .
C P , N L L I i ( n ) = 1 N Q i = 1 Q n = 1 N 1 Z i ( n ) I i ( n ) C G , N L L I i ( n ) = 1 N Q i = 1 Q n = 1 N 1 ( I i ( n ) + σ i 2 ) [ 2 [ Z i ( n ) I i ( n ) ] ( I i ( n ) + σ i 2 ) + [ Z i ( n ) I i ( n ) ] 2 ( I i ( n ) + σ i 2 ) 2 ] .
I ( x , y ; α , q , P , σ B ) = ( | a = 1 3 g i m g , a ( x , y ; α , q , P ) μ a | 2 + | b = 4 6 g i m g , b ( x , y ; α , q , P ) μ b 3 | 2 ) B .
I i ( n ) P ( m ) = 2 Re { a = 1 3 [ g i m g , a ( n ; α , q , P ) P ( m ) ( a ~ = 1 3 g i m g , a ~ ¯ ( n ; α , q , P ) μ a ~ ) ] + b = 4 6 [ g i m g , b ( n ; α , q , P i ) P ( m ) ( b ~ = 4 6 g i m g , b ~ ¯ ( n ; α , q , P ) μ b ~ 3 ) ] } B .

Metrics