Open Access
Add to BibSonomy
Abstract
Point spread function (PSF) engineering by phase modulation is a novel approach to three-dimensional (3D) super-resolution microscopy, with different point spread functions being proposed for specific applications. It is often not easy to achieve the desired shape of engineered point spread functions because it is challenging to determine the correct phase mask. Additionally, a phase mask can either encode 3D space information or additional time information, but not both simultaneously. A robust algorithm for recovering a phase mask to generate arbitrary point spread functions is needed. In this work, a generalized phase mask design method is introduced by performing an optimization. A stochastic gradient descent algorithm and a Gauss-Newton algorithm are developed and compared for their ability to recover the phase masks for previously reported point spread functions. The new Gauss-Newton algorithm converges to a minimum at much higher speeds. This algorithm is used to design a novel stretching-lobe phase mask to encode temporal and 3D spatial information simultaneously. The stretching-lobe phase mask and other masks are fabricated in-house for proof-of-concept using multi-level light lithography and an optimized commercially sourced stretching-lobe phase mask (PM) is validated experimentally to encode 3D spatial and temporal information. The algorithms’ generalizability is further demonstrated by generating a phase mask that comprises four different letters at different depths.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Recent years have witnessed the development of wavefront phase manipulation for various optical applications including manipulating plasmonic nanoantennas [1,2], holographic displays [3,4], and engineered PSFs for depth detection [5–9]. Phase-only modulation is advantageous in single-molecule studies because it minimizes lost photons [10], thereby maintaining a high spatial resolution [11]. Super-resolution techniques accomplished by point spread function engineering simultaneously provide large axial detection ranges and high 3D spatial resolutions [5,10,12–18]. To achieve 3D super-resolution detection with such methods, a 4f optical geometry is used, and a phase mask is inserted into the Fourier plane between two lenses along the detection path of a traditional optical microscope. The emission signal is modified in the Fourier domain by the PM within the 4f system [7]. Several previously reported PMs generate PSFs containing depth-dependent information and are used for the imaging of biological structures and dynamics in 3D space [12,15,18–24]. More recently, the popular double helix PM was used to encode temporal dynamics, but at the cost of sacrificing 3D spatial information [25,26].
Designing new or complicated PMs is a phase retrieval problem [5,27–29] in that recovering a signal’s representation in the Fourier domain from only the amplitude in spatial domain is computationally challenging. Previous approaches for phase retrieval include Gerchberg-Saxton algorithms [27,30] and hybrid input-output algorithms [31]. In most cases, a PM is designed by optimizing only a single image in the spatial domain. However, to achieve 3D detection, the continuous evolution of the PSF throughout the desired depth range needs to be considered in the PM design. This requirement limits the application of existing algorithms in the design of novel PMs.
PM design utilizing Gaussian-Laguerre modes is one of the methods that yields a PM design with continuously evolving PSFs [5]. However, this algorithm is only applicable for rotating PSFs, while non-rotating functional PSFs cannot be generated with this method. The above challenges motivate the necessity for a generalized and robust method for the design of PMs suitable for a wide array of imaging applications.
Here we report two new algorithms for PM design using the fundamentals of optimization: a stochastic gradient descent algorithm and a Gauss-Newton algorithm [32]. To confirm the validity of these methods, two well-known PSFs and corresponding PMs are recovered with our method. The patterns generated by our algorithms match well to the previously reported PSFs [5]. We further propose a novel stretching-lobe PSF for super-resolution localization using our algorithms. This PSF is capable of simultaneously encoding sub-frame temporal information while achieving 3D super-localization. Moreover, we showcase the ability to encode arbitrarily complex PSFs by designing a single PM that yields PSFs with the individual letters “R”, “I”, “C”, “E” at different depths. The physical PMs are fabricated via photolithography for proof-of-concept, and the chosen stretching-lobe PM is sourced commercially for final implementation.
2. Methods
2.1 Phase retrieval
One of the most critical applications of PMs lies in super-resolution microscopy, thus we emphasize encoding super-localization information in PSFs, and formatting the PM design problem as a phase retrieval problem. As shown in Fig. 1, PSF engineering is accomplished with a 4f system in a microscope’s detection path. The PM is located at the Fourier plane, which is centered between two lenses. After phase modulation by the PM, the emission light is transferred back to the spatial domain and captured by a camera. The intensity profile of PSFs generated by a given PM is expressed as the following:
in which is the intensity distribution and the subscript denotes the pixel located at row and column; is the discrete fast Fourier transform operation; is a circular pupil intensity mask with a diameter of 2 mm; is the PM pattern in the Fourier domain; ‘’ is the Hadamard product operator; denotes the phase delay in the Fourier domain caused by the defocus effect; denotes the distance of pixels on the PM to the center of the PM; denotes the focal length of the lens; denotes the defocus distance. It should be noticed that Eq. (1) is only an approximation to calculate the PSF from the PM under our current PM quantization (40 by 40 pixels). For a more accurate calculation of the PSF in the continuous domain please refer to reference [33]. Our aim is to calculate the PM, from the intensity distribution of the desired PSFs, , which is equivalent to an inverse process of that in Eq. (1). However, a simple inverse Fourier transform of the PSF cannot reconstruct the PM because the phase information of the PSF is lost and only its amplitude is recovered.
Fig. 1 Layout of a typical PM based 3D super-resolution microscope. The 4f system is shown in the red box consisting of two lenses and a PM at the Fourier plane. Emission light is modulated by the 4f system and projected to a camera.
Here we propose a scheme to retrieve the PM pattern by minimizing the difference between the input PSF profile and the PSF derived from the PM (Eq. (1)). The PM is successfully generated when the two PSFs match with each other as shown in Eq. (2):
in which is the PSF calculated from the PM and is the desired PSF containing different depths. To do so, an optimization method is applied. The L2 norm residual, difference between two PSF intensity profiles, is calculated by:where is an operation of taking the element-wise amplitude squared; zero-padding is not applied in the FFT. Then the corresponding PM is calculated by minimizing the objective function:2.2 Converting phase retrieval to standard optimization problem
Two algorithms are compared in this manuscript to recover PM designs for desired PSFs. One utilizes a stochastic gradient descent and the second relies on a Gauss-Newton method using 2nd order approximations. PM design is simplified by converting the original problem to a standard linear optimization problem. It should be noted that Eq. (5) cannot be directly solved because of the existence of a Fourier transform. In most optimization problems, a gradient of the objective function needs to be calculated, which is computationally challenging when a Fourier transform is applied to the objective function. We first represent the discrete Fourier transform with matrix multiplication.
The 2D discrete Fourier transform is equivalent to the product of the Fourier transform matrix and the PM pattern :
in which denotes the amplitude of signal in the spatial domain and denotes the vectorised PM pattern. The dimension of the Fourier matrix is determined by the image size of the PSF (K) and the image size of the PM (N) in Eq. (7). Thus the residual is expressed by:2.3 Stochastic gradient descent optimization
The stochastic gradient descent algorithm is widely used in solving optimization problems, especially when there are multiple constraints simultaneously applied on the objective functions [34,35]. In PM design, the PSFs at nine different depths consequentially constrain the PM update process, which makes the stochastic gradient descent algorithm ideal for solving this problem. In each iteration the PM is updated only based on the PSF at one depth and consecutively constrains the PM at nine different layers. The gradient is calculated using Newton’s difference quotient. The algorithm scheme is shown in Algorithm 1 below.
Algorithm 1. CStochastic gradient descent
2.4 Gauss-Newton optimization
A Gauss-Newton algorithm is applied to solve the optimization problem in Eq. (5). Gauss-Newton method is well known for solving linear optimization problems with fast convergence speeds. It outperforms Newton’s method in terms of lower computational loads by only updating the Jacobian matrix rather than the second order derivative of the objective function:
in which ∇ is the gradient operator and is the Jacobian matrix. The optimization steps are summarized in Algorithm 2:Algorithm 2. Gauss-Newton
The Gauss-Newton algorithm converges much faster than SGD algorithm because of the utilization of 2nd order approximation (Appendix A, Fig. 8). As the input PSF becomes more complicated, the Gauss-Newton reaches the minimum within a reasonable period of time. Thus we only focus on the Gauss-Newton algorithm in the manuscript for the fabrication and experimental verification of the PM designs.
2.5 Fabrication of PMs using Reactive-Ion Etching (RIE)
The PMs were fabricated on fused silica substrates through nine iterations of photolithography, with RIE (Oxford PlasmaLab 100) following each step [36]. The positive photoresist S1818 (Microchem Corp.) was spin coated onto the entire wafer with a thickness of 2.5 μm followed by a 1 min softbake at 115 °C. The wafer was aligned with the photomask and exposed to a constant dose of 220 mJ/cm2 UV light (EVG 620). After UV exposure, the wafer was developed in MF 319 (Microchem Corp.) for 40 sec, in which the exposed regions were soluble while the unexposed regions remained insoluble. Afterwards, the wafer was rinsed with DI water for 2 min to remove the developer and photoresist residues in the exposed areas. To perform the photolithography, photomasks are patterned on a 4 inch Soda Lime substrate via direct laser writing. The photomasks were fabricated by using a 3D direct laser lithography system (Nanoscribe GmbH, Germany) with a high resolution negative photosensitive resist (IP-L, Nanoscribe GmbH). A total of 9 hard mask patterns are prepared to fabricate a PM consisting of 10 layers with various thicknesses, 127.2 nm for each single layer. The final PMs have pixelated patterns with pixel sizes of either 33 μm (60 by 60 pixels) for the RICE PSFs or 50 μm (40 by 40 pixels) for the double helix (DH) and stretching-lobe PSF patterns. The fabrication process is illustrated in Fig. 2.
Fig. 2 Fabrication process of a PM using photolithography and RIE. A total of nine iterations are involved with each using a different photomask. Patterns are etched to the substrates after photoresists are developed in each iteration of photolithography. Scale bar = 100 μm in SEM image.
The PM consists of n by n pixels, which means there are n rows pixels and each row has n pixels with different heights. The surface profile of each row can be acquired by carrying out profilometer (Veeco Dektak 6M) scanning. After profilometer scanning of each row is complete, we can obtain the surface profiles of the PM.
2.6 Commercial sourcing of optimized PM
A higher resolution (80 by 80 pixels) PM was purchased from Double Helix Optics (Boulder, CO) [37], The commercially-sourced PM was fabricated using gray scale lithography techniques, allowing the etching depth on the PM to be controlled by the exposure time of a laser beam on each pixel, which yields higher depth and lateral resolutions that those used in our proof-of-concept PMs [38].
2.7 Depth measurement and calibration
PSFs of designed PMs are measured by imaging carboxylate-modified polystyrene 100 nm beads (orange fluorescent, max abs/em: 540/560 nm, Invitrogen) at different depths. The fluorescent samples are excited by a 532 nm laser (Coherent, Compass 315M-100SL). The signal is collected by an oil-immersion objective (Carl-Zeiss, alpha Plan-Fluar, N.A. = 1.45, 100 X magnification) and then imaged by a sCMOS camera (Hamamatsu, ORCA-Flash 4.0). Each camera pixel corresponds to 172 nm in real space. The beads are fixed on the coverslip and the objective is equipped with an objective scanner (P-721 PIFOC) to shift and calibrate the samples depth. A high resolution (80 x 80 pixel) PM stretching-lobe PM, purchased from Double Helix LLC, is mounted on a motorized rotary mount (QIOPTIQ, Rotary Mount with Servo Motor) and the rotating speed is 300 rpm.
3. Results and discussion
3.1 Recovering well-established PMs
To test the validity of our algorithms we first generate a PM based on a commonly used engineered PSF, namely the DH PSF. DH PSFs are rotating PSFs, and as discussed earlier, are originally designed in the Gaussian-Laguerre domain [5]. In the Gaussian-Laguerre domain the different modes are orthogonal and form a complete basis set, and the PMs are decomposed into coefficients of the modes. The mode coefficients are optimized to derive the final PM pattern. Or proposed algorithms take a different approach as each pixel in the PM pattern is individually optimized and the final PM is obtained by simultaneously optimizing all pixels. The minimum is reached by iteratively forcing the generated PSFs to be the desired pattern.
Our algorithms are able to generate a PM pattern that can produce DH PSFs similar those previously reported [5]. The initial guess of the PM and the input PSFs are shown in Figs. 3(A) and 3(B). Unlike previous algorithms, our algorithms do not require a priori information. Even random initial guesses as shown in Fig. 3(A) lead to convergence. The input PSFs have two Gaussian-shaped lobes that rotate around the center point (white crosses in Fig. 3(B)). The PM pattern is pixelated containing 40 by 40 pixels. Thus, a total of 1,600 variables are optimized simultaneously in this algorithm. The maximum number of iterations using Gauss-Newton algorithm is set to 20, which converges to the minimum and gives reasonably good results (Appendix A, Fig. 8). Figures 3(C) and 3(D) are the recovered PMs and the corresponding PSFs at different depths using our algorithms. It is worth noting that the produced PSFs have side lobes, which is similar to the previously reported DH PSFs [5]. The theoretical localization precision of the PM in Fig. 3(C) is also calculated from the Cramer-Rao lower bound (CRLB) (Appendix A, Fig. 16). This PM is also recovered using a Gerchberg-Saxton algorithm (Appendix A, Fig. 9) and SGD algorithm (Appendix A, Fig. 10), but the Gerchberg-Saxton algorithm does not yield well defined DH PSFs. The PM is experimentally fabricated using the RIE method (Fig. 3(E)). The fabricated PM is measured (Fig. 3(E)) and the PSFs shown in Fig. 3(F) are the results of 1.0 μm fluorescent beads imaged at different axial depths. The shapes of the obtained PSFs deviate slightly from the simulated PSFs especially for the −1.125 μm depth as shown in Fig. 3(F). We attribute this deviation to missing information during the discretization of the PM in addition to artifacts generated during PM fabrication (Appendix A, Fig. 18). Other potential reasons to the deviation of the PSF shapes might be the refractive index mismatch between the sample and the glass slide, which is a topic that deserves future attention, especially as it has recently been shown that PSF engineering can be used to identify and correct for other optical aberrations [39,40]. Additionally a PM that produces previously published corkscrew PSFs [19] is also successfully recovered using our algorithms (Appendix A Fig. 11).
Fig. 3 PM fabricated using the RIE method and corresponding DH PSFs. (A) Initial guess for the PM pattern. (B) Intensity profiles of the desired PSFs. These intensity profiles are used as the input for the PM design. Each PSF contains two Gaussian distributed lobes and the orientation varies at different depth. The white crosses denote the center position of the lobes, which is the lateral position of the emitter in 3D super-resolution microscopy. (C, D) The recovered PM (C) and its corresponding PSFs at different depths (D). (E, F) The fabricated PM (E) and the measured PSFs at different depths (F). The scale bar in A, C, and E is 500 µm and 1 µm in B, D, and F.
3.2 Stretching-lobe PSF to obtain both time and depth information in high resolution
Similar to rotating PSFs, non-rotating PSFs are able to encode depth information with a response other than PSF orientation. In specific applications the rotation response is already used for detection of information other than depth. For example in our former work [25] the PM is physically rotated 180 degrees within one camera exposure, thereby encoding sub-frame time information in the final PSF. However, if the PM generates PSFs that rotate as a function of depth, the time information cannot be extracted from the data. Therefore when designing a PM capable of 3D super resolution and sub-frame temporal encoding it is highly desired to develop a PM that produces non-rotating PSFs using our algorithms. Non-rotating PSFs were reported previously. However, none of them are calculated based on the desired input PSFs [20,28]. As a result, simultaneous detection of both 3D spatial and time information will not be achievable with existing rotating PSFs and is addressed using the stretching-lobe PSF proposed in this work.
We propose and experimentally test a new stretching-lobe PSF to encode the depth response in 3D super-resolution microscopy into the distances between the two lobes of the PSFs. The desired PSFs are shown in Fig. 4(B). The PSFs consist of two Gaussian distributed lobes with the center between two lobes (white crosses) indicating the lateral position of the emitter. Once again, the PM contains 40 by 40 pixels. Instead of changing the relative orientation between the two lobes, the distance between them changes when the depth of the emitter is changed (Fig. 4(B)). With a random PM initial guess, the generated PM pattern (Fig. 4(C)) and the corresponding PSFs (Fig. 4(D) and Visualization 1) produced from the PM are calculated. Even though different initial guesses lead to slightly different PM patterns, the generated PSFs are very close to each other (Appendix A, Fig. 12). It should be noted that the sharp phase variations in the PM in Fig. 4(E) are not artifacts from the optimization algorithm, but instead these features serve to sharply focus photons into the central portions of the resulting PSF. When smoothing a small arbitrary area of the PM by averaging the values, the photon percentage in the two central lobes drops from 47.3% to 45.8% thereby lowering performance (Appendix A, Fig. 13). Compared to the DH PSFs, the stretching-lobe PSFs have a smaller depth of detection from −0.75 μm to 0.75 μm. Other depth ranges are also examined, but the recovered PSFs are not satisfactory (Appendix A Fig. 14). When the emitter moves beyond the depth-detection range, the intensity will decay rapidly until the PSF is not observed. To verify this PSF experimentally the PM is also fabricated using RIE method (Appendix A Fig. 15) to measure the resulting PSFs (Figs. 4(E), 4(F), and Visualization 2). The obtained PSFs match with the simulated data reasonably well. When physically rotating this particular PM, the PSF orientation will not couple with the depth response, thus this PSF is an ideal candidate for encoding both 3D spatial and sub-frame temporal information.
Fig. 4 PM fabricated using the RIE and corresponding stretching-lobe PSFs. (A) Initial guess for the PM pattern. (B) Intensity profiles of the desired PSF. Each PSF contains two Gaussian distributed lobes and the distance between two lobes varies at different depth. The white crosses denote the center position of the lobes, which is usually the lateral position of the emitter in 3D super-resolution microscopy. (C, D) The recovered PM (C) and its corresponding PSFs at different depths (D). Simulated PSFs of emitter at more depth positions can be found in Visualization 1. (E, F) The fabricated PM (E) and the measured PSFs at different depths (F). Experimental PSFs of emitter at more depth position can be found in Visualization 2. The scale bar in A, C, and E is 500 µm and 1 µm in B, D, and F.
By running Monte Carlo simulations, the localization precision of the stretching-lobe PSFs is calculated to determine the best working distances (Fig. 5). Each fluorescent emitter is simulated to emit photons following Poisson distributions with a mean value of 2,000 photons [14]. Various background noises with Poisson distributions are also considered in this simulation as well. The stretching-lobe PSFs provide reliable 3D position localization within the depth range from −0.6 μm to 0.6 μm. By calculating the CRLB of both the DH PM and stretching PM (Appendix A, Fig. 16), it is clear that the DH PSF still provides a higher depth localization precision and a larger depth detection range. However, the stretching-lobe PSF is able to provide additional information besides 3D localization, such as the temporal information [25,26].
Fig. 5 Localization precision of the stretching-lobe PSFs. Localization precision in both lateral and depth dimensions predicted by 1,000 Monte Carlo simulations per data point with additional Poisson noise of 6 and 12 photons per pixel.
Rotation of the stretching-lobe PM enables the simultaneous acquisition of both 3D spatial and sub-frame temporal information. As discussed in our previous works [25,26], the depth and time response are the same for the DH PSF, leading to a rotational response when either depth or time changes. With the stretching-lobe PM, the depth response is independent of the time response, as shown in Fig. 6. Different depths of the fluorescent emitters are achieved by manually moving the objective. A commercially-sourced stretching-lobe PM exhibits high lateral and depth resolution (Appendix A Fig. 17). The high resolution PM increases the peak intensity of PSFs by 3% compared with that of the PM in low resolution in Fig. 4(C). The PSFs were imaged with a high camera frame rate (100 fps) to show that different orientations of the two lobes uniquely label time. Whereas, the emitter’s depth is encoded in the distance between the two lobes (Fig. 6.). The time-angle rate is 0.56 ms/degree, while the lobe distance-depth rate is 3.125 pixels/μm. It must be noted that in the application of sub-frame temporal retrieval, as we have previously demonstrated [25,26]. If a slower frame rate is used, the arc-lengths of the resulting PSF encodes the surface residence time. The PSFs shown in Fig. 6 represent a single emitter at different depths and times. The development of this PSF design allows for 3D tracking to be performed in addition to resolving sub-frame temporal information all compressed into a 2D image. Using this PSF design for tracking applications will also demand the development of more advanced algorithms [16,41–44] designed for such PSF shapes.
Fig. 6 Final PSFs of stationary green beads when rotating the stretching-lobe PM fabricated using gray scale lithography. PSFs of different orientations (rows) denote different time; PSFs of different lobe distance (columns) denote different depth of the emitter. The scale bar is 1 µm.
3.3 Developing complicated PSFs
In addition to generating 3D PSFs for depth detection and 4D super-resolution, the proposed algorithm can also design arbitrarily complicated PSF patterns. Figure 7(A) shows an initial PM pattern containing a 60 by 60 matrix of random initial guesses. Figure 7(B) are the input PSFs containing four letters at different depths. The number of pixels increases compared with the PMs used for the DH PSF or stretching-lobe PSF due to the fact that the desired PSFs are much more complicated and requires more granularity. Thus a finer spatial control of the PM is necessary in this application. Four different constraints are simultaneously applied to the PM design. It is usually difficult to use traditional phase retrieval approaches [45]. However our algorithm generates a PM based on the complicated PSF inputs of spelling the word “RICE”.
Fig. 7 PM fabricated using the RIE method for complicated PSFs. (A) Initial guess for the PM pattern. (B) Intensity profiles of the desired PSF. Four letters at different depths serve as four constraints. (C, D) The recovered PM (C) and its corresponding PSFs at different depths (D). (E, F) The fabricated PM (E) and the measured PSFs at different depths (F). The scale bar is 500 µm in A, C, and E, while it equals to 1 µm in B, D, and F.
Our algorithm is robust and generalized enough to generate arbitrarily complicated PMs solely based on the input PSF profiles. In Fig. 7(B), four letters are used as the desired PSFs, and the PM pattern can still be recovered (Fig. 7(C)). It is clear that the four letters are successfully recovered by the PM (Fig. 7(D)), although a few artifacts are involved in the end. The PM is fabricated (Fig. 7(E)) and the PSFs are obtained (Fig. 7(F)).
4. Conclusion
In this work we propose two new algorithms to design the PM for arbitrary shape of PSFs. We verify the universality of our algorithms by recovering two published PMs that were designed with a different algorithm. Later, we propose a novel stretching-lobe PSF for 3D super-resolution microscopy and generated the corresponding PM using the faster performing Gauss-Newton method. We also demonstrated the capability of the stretching-lobe PM to simultaneously encode 3D spatial and temporal information. Finally, we also successfully design a PM that corresponds to very complicated PSFs at different depths.
Appendix A Supporting information for phase mask design algorithms
Convergence curve
Fig. 8 Convergence curve of L2 norm residual of (A) Stochastic gradient descent method (B) Gauss-Newton method.
Result of applying Gerchberg-Saxton algorithm
Fig. 9 DH PM recovery result with Gerchberg–Saxton algorithm. The recovery results for each depth layer are averaged in each iteration with equal weight. The algorithm converges at a local minimum.
Result of applying Stochastic Gradient Descent algorithm
Fig. 10 DH PM recovery result with stochastic gradient descent method. In each iteration only one PSF in one depth constrains the optimization. The recovery result is finished within 400 iterations.
Recovery of corkscrew PSF
Fig. 11 The corkscrew PM designed with the proposed method in simulation. (A) The random initial guess where the PM design starts. (B) The desired PSF intensity profiles as the input to the PM design code. Each PSF contains one Gaussian distributed lobe and the orientation varies at different depth. The crosses denote the lateral position of the emitter. (C) The recovered PM from the new algorithm and (D) the corresponding PSFs at different depths. (E) The experimentally fabricated PM and (F) the measured PSFs at different depths. The scale bar is 500 µm in (A, C, and E); the scale bar = 1 µm in (B, D, and F).
PM generated from different initial guesses
Fig. 12 (A) Recovered stretching-lobe PMs with different initial guesses. (B) The corresponding PSFs at the focal plane from each PM shown in A. Scale bar = 1 μm.
Analyses on sharp variations in the PM
Fig. 13 (A) The recovered stretching-lobe PM and (B) the corresponding stretching-lobe PSF in the focal plane. The photons in the two lobes account for 47.3% out of the 2,000 total simulated photons. (C) The PM is smoothed by averaging values within the region denoted by the green block. (D) The corresponding stretching-lobe PSF in the focal plane. The photons in the two lobes account for 45.8% out of the 2,000 total simulated photons.
Phase mask of different depth dependence
Fig. 14 Stretching PM design with different depth range. (A, C, and E) The recovered PM pattern with 1.5 μm, 3.0 μm, and 6.0 μm depth range. (B, D, and F) Representative stretching PSFs of the corresponding PM pattern at different depth layers. The scale bar is 500 µm in (A, C, and E); the scale bar = 1 µm in (B, D, and F).
Scanning electron microscope image of phase mask
Fig. 15 Scanning electron microscope (SEM) image of two regions on stretching-lobe PM fabricated using RIE method. Different depths of the surface feature is achieved by multiple layer light-lithography.
Cramer-Rao lower bound (CRLB) calculation
Fig. 16 Cramer-Rao lower bound for the DH PSF (A, B) and stretching-lobe PSF (C, D). In (A) and (C) the emitted photons are 2,000 and background noise are simulated as a Poisson distribution with a mean value of 6 photons per pixel. In (A) and (C) the emitted photons are 2,000 and background noise are simulated as a Poisson distribution with mean value of 12 photons per pixel.
Stretching-lobe PM of high resolution
Simulation from fabricated PMs
Fig. 18 Simulation of the PSFs from the fabricated PMs. The PMs in A, B, and C are from Fig. 3(E), 4(E), and 7(E) respectively, which is characterized by profilometer.
Signal to noise ratio estimation
The SNR is calculated by
in which denotes the photons from the emitters, denotes the variance of the background noise. When simulating 2,000 photons emitted from the fluorescent emitters, we assume that each lobe of the PSF follows a Gaussian distribution, so that can be estimated. At different SNRs, the variance of the background noise can be calculated by . We assume that the background noise follows a simple Poisson distribution, which is added to the simulated PSFs. The mean background noise values of 6 and 12 photons per pixel results in a SNR of 20 and 10 respectively.
Funding
National Science Foundation (NSF) (CHE 1808382); Welch Foundation (C- 1787); National Science Foundation (NSF) Graduate Research Fellowship Program (GRFP) (1450681).
Acknowledgment
Christy F. Landes thanks the National Science Foundation (NSF) (CHE 1808382) and the Welch Foundation (C- 1787). Nicholas Moringo acknowledges this work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. 1450681.
References
1. N. Yu, P. Genevet, M. A. Kats, F. Aieta, J.-P. Tetienne, F. Capasso, and Z. Gaburro, “Light propagation with phase discontinuities: generalized laws of reflection and refraction,” Science 334(6054), 333–337 (2011). [CrossRef] [PubMed]
2. X. Ni, N. K. Emani, A. V. Kildishev, A. Boltasseva, and V. M. Shalaev, “Broadband light bending with plasmonic nanoantennas,” Science 335(6067), 427 (2012). [CrossRef] [PubMed]
3. H. Sasaki, K. Yamamoto, Y. Ichihashi, and T. Senoh, “Image Size Scalable Full-parallax Coloured Three-dimensional Video by Electronic Holography,” Sci. Rep. 4(1), 4000 (2015). [CrossRef] [PubMed]
4. S. Yamada, T. Kakue, T. Shimobaba, and T. Ito, “Interactive Holographic Display Based on Finger Gestures,” Sci. Rep. 8(1), 2010 (2018). [CrossRef] [PubMed]
5. S. R. P. Pavani and R. Piestun, “High-efficiency rotating point spread functions,” Opt. Express 16(5), 3484–3489 (2008). [CrossRef] [PubMed]
6. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. U.S.A. 106(9), 2995–2999 (2009). [CrossRef] [PubMed]
7. A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical Fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014). [CrossRef] [PubMed]
8. E. R. Dowski Jr. and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34(11), 1859–1866 (1995). [CrossRef] [PubMed]
9. S. Bagheri and B. Javidi, “Extension of depth of field using amplitude and phase modulation of the pupil function,” Opt. Lett. 33(7), 757–759 (2008). [CrossRef] [PubMed]
10. G. Grover, S. Quirin, C. Fiedler, and R. Piestun, “Photon efficient double-helix PSF microscopy with application to 3D photo-activation localization imaging,” Biomed. Opt. Express 2(11), 3010–3020 (2011). [CrossRef] [PubMed]
11. R. E. Thompson, D. R. Larson, and W. W. Webb, “Precise nanometer localization analysis for individual fluorescent probes,” Biophys. J. 82(5), 2775–2783 (2002). [CrossRef] [PubMed]
12. M. D. Lew, M. A. Thompson, M. Badieirostami, and W. E. Moerner, “In vivo three-dimensional superresolution fluorescence tracking using a double-helix point spread function,” Proc SPIE Int Soc Opt Eng 7571, 75710Z–75710Z (2010). [CrossRef]
13. W. E. Moerner, Y. Shechtman, and Q. Wang, “Single-molecule spectroscopy and imaging over the decades,” Faraday Discuss. 184, 9–36 (2015). [CrossRef] [PubMed]
14. M. Badieirostami, M. D. Lew, M. A. Thompson, and W. E. Moerner, “Three-dimensional localization precision of the double-helix point spread function versus astigmatism and biplane,” Appl. Phys. Lett. 97(16), 161103 (2010). [CrossRef] [PubMed]
15. H. Shen, L. J. Tauzin, R. Baiyasi, W. Wang, N. Moringo, B. Shuang, and C. F. Landes, “Single particle tracking: from theory to biophysical applications,” Chem. Rev. 117(11), 7331–7376 (2017). [CrossRef] [PubMed]
16. B. Shuang, W. Wang, H. Shen, L. J. Tauzin, C. Flatebo, J. Chen, N. A. Moringo, L. D. C. Bishop, K. F. Kelly, and C. F. Landes, “Generalized recovery algorithm for 3D super-resolution microscopy using rotating point spread functions,” Sci. Rep. 6(1), 30826 (2016). [CrossRef] [PubMed]
17. J. K. Adams, V. Boominathan, B. W. Avants, D. G. Vercosa, F. Ye, R. G. Baraniuk, J. T. Robinson, and A. Veeraraghavan, “Single-frame 3D fluorescence microscopy with ultraminiature lensless FlatScope,” Sci. Adv. 3(12), e1701548 (2017). [CrossRef] [PubMed]
18. A. von Diezmann, Y. Shechtman, and W. E. Moerner, “Three-dimensional localization of single molecules for super-resolution imaging and single-particle tracking,” Chem. Rev. 117(11), 7244–7275 (2017). [CrossRef] [PubMed]
19. M. D. Lew, S. F. Lee, M. Badieirostami, and W. E. Moerner, “Corkscrew point spread function for far-field three-dimensional nanoscale localization of pointlike objects,” Opt. Lett. 36(2), 202–204 (2011). [CrossRef] [PubMed]
20. S. Jia, J. C. Vaughan, and X. Zhuang, “Isotropic 3D Super-resolution Imaging with a Self-bending Point Spread Function,” Nat. Photonics 8, 302–306 (2014). [CrossRef] [PubMed]
21. Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015). [CrossRef] [PubMed]
22. N. A. Moringo, H. Shen, L. D. C. Bishop, W. Wang, and C. F. Landes, “Enhancing analytical separations using super-resolution microscopy,” Annu. Rev. Phys. Chem. 69(1), 353–375 (2018). [CrossRef] [PubMed]
23. M. A. Thompson, J. M. Casolari, M. Badieirostami, P. O. Brown, and W. E. Moerner, “Three-dimensional tracking of single mRNA particles in Saccharomyces cerevisiae using a double-helix point spread function,” Proc. Natl. Acad. Sci. U.S.A. 107(42), 17864–17871 (2010). [CrossRef] [PubMed]
24. V. Sundaresan, K. Marchuk, Y. Yu, E. J. Titus, A. J. Wilson, C. M. Armstrong, B. Zhang, and K. A. Willets, “Visualizing and calculating tip–substrate distance in nanoscale scanning electrochemical microscopy using 3-dimensional super-resolution optical imaging,” Anal. Chem. 89(1), 922–928 (2017). [CrossRef] [PubMed]
25. W. Wang, H. Shen, B. Shuang, B. S. Hoener, L. J. Tauzin, N. A. Moringo, K. F. Kelly, and C. F. Landes, “Super Temporal-Resolved Microscopy (STReM),” J. Phys. Chem. Lett. 7(22), 4524–4529 (2016). [CrossRef] [PubMed]
26. W. Wang, H. Shen, N. A. Moringo, N. C. Carrejo, F. Ye, J. T. Robinson, and C. F. Landes, “Super-temporal resolved microscopy reveals multistep desorption kinetics of α-lactalbumin from nylon,” Langmuir 34(23), 6697–6702 (2018). [CrossRef] [PubMed]
27. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21(15), 2758–2769 (1982). [CrossRef] [PubMed]
28. Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014). [CrossRef] [PubMed]
29. P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017). [CrossRef] [PubMed]
30. H. H. Bauschke, P. L. Combettes, and D. R. Luke, “Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization,” J. Opt. Soc. Am. A 19(7), 1334–1345 (2002). [CrossRef] [PubMed]
31. J. R. Fienup and C. C. Wackerman, “Phase-retrieval stagnation problems and solutions,” J. Opt. Soc. Am. A 3(11), 1897–1907 (1986). [CrossRef]
32. Å. Björck, Numerical Methods for Least Squares Problems.
33. J. Chao, S. Ram, T. Lee, E. S. Ward, and R. J. Ober, “Investigation of the numerics of point spread function integration in single molecule localization,” Opt. Express 23(13), 16866–16883 (2015). [CrossRef] [PubMed]
34. L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010(Springer, 2010), pp. 177–186. [CrossRef]
35. B. Recht, C. Re, S. Wright, and F. Niu, “Hogwild: A lock-free approach to parallelizing stochastic gradient descent,” in Advances in neural information processing systems(2011), pp. 693–701.
36. F. Ye, B. W. Avants, A. Veeraraghavan, and J. T. Robinson, “Integrated light-sheet illumination using metallic slit microlenses,” Opt. Express 26(21), 27326–27338 (2018). [CrossRef] [PubMed]
37. G. Grover, K. DeLuca, S. Quirin, J. DeLuca, and R. Piestun, “Super-resolution photon-efficient imaging by nanometric double-helix point spread function localization of emitters (SPINDLE),” Opt. Express 20(24), 26681–26695 (2012). [CrossRef] [PubMed]
38. A. Rammohan, P. K. Dwivedi, R. Martinez-Duarte, H. Katepalli, M. J. Madou, and A. Sharma, “One-step maskless grayscale lithography for the fabrication of 3-dimensional structures in SU-8,” Sens. Actuator B-Chem. 153(1), 125–134 (2011). [CrossRef]
39. M. P. Backlund, M. D. Lew, A. S. Backer, S. J. Sahl, G. Grover, A. Agrawal, R. Piestun, and W. E. Moerner, “Simultaneous, accurate measurement of the 3D position and orientation of single molecules,” Proc. Natl. Acad. Sci. U.S.A. 109(47), 19087–19092 (2012). [CrossRef] [PubMed]
40. A. Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2(11), 985–993 (2015). [CrossRef] [PubMed]
41. L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012). [CrossRef] [PubMed]
42. H. P. Babcock and X. Zhuang, “Analyzing single molecule localization microscopy data using cubic splines,” Sci. Rep. 7(1), 552 (2017). [CrossRef] [PubMed]
43. H. Mazidi, J. Lu, A. Nehorai, and M. D. Lew, “Minimizing structural bias in single-molecule super-resolution microscopy,” Sci. Rep. 8(1), 13133 (2018). [CrossRef] [PubMed]
44. Y. Li, M. Mund, P. Hoess, J. Deschamps, U. Matti, B. Nijmeijer, V. J. Sabinina, J. Ellenberg, I. Schoen, and J. Ries, “Real-time 3D single-molecule localization using experimental point spread functions,” Nat. Methods 15(5), 367–369 (2018). [CrossRef] [PubMed]
45. J. Zhang, N. Pégard, J. Zhong, H. Adesnik, and L. Waller, “3D computer-generated holography by non-convex optimization,” Optica 4(10), 1306–1313 (2017). [CrossRef]
