The depth of field of an imaging system refers to the distance range in the vicinity of the focal plane, where the image maintains an acceptable sharpness. The depth of field enforces a general constraint on the allowable sample thickness for obtaining the highest achievable spatial resolution. This limitation can severely deteriorate for high-resolution three-dimensional microscopy, because the typical depth of field can be substantially smaller than the lateral field of view. The focus stacking method tackles this problem by collecting a series of images at multiple object planes, where features at various depths sequentially appear in focus at their corresponding focal planes. These sharp features can be extracted and merged to reconstruct an image with an extended apparent depth of field. Considering that the in-focus features contain sharp details and, thus, have more high frequency information, they can be distinguished through analyzing the frequency components using wavelet or windowed Fourier transforms [1,2]. The focus stacking method has been successfully implemented in optical [3], electron [4], and full-field x-ray microscopy [5].

The multi-slice method [6,7] provides an alternative way to extend the depth of field by decomposing a thick sample into thin slices and modeling the multiple scattering effect slice by slice. This method extracts depth information through numerical propagation, instead of the image stack. The integration of the multi-slice method with the ptychography technique was demonstrated as an effective approach to obtain diffraction-limited resolution from specimens thicker than the depth of field [8], and is able to provide 3D images without sample rotation [9]. These benefits encourage emerging efforts on adopting the multi-slice ptychography method in the x-ray regime for improving both the lateral and depth resolutions [10–15], and exploiting the potential advantages as a novel approach for high-resolution 3D imaging with reduced number of projections beyond the Crowther criterion [16–19].

In this Letter, we present a new approach to effectively extend the depth of field by numerically generating the focus stack from single-slice ptychography reconstructions with propagated illumination wavefronts and obtaining a single sharp image using an image fusion algorithm based on the complex-valued discrete wavelet transform. This approach does not require the repeated measurements of translating the sample along the axial direction as in the conventional focus stacking method, and is computationally more efficient than the multi-slice ptychography method.

Ptychography is a diffraction-based imaging technique, which reconstructs real-space images from datasets collected by scanning a sample across a confined beam with adequately fine steps [20]. The redundant measurements provided by overlapped scan spots make it possible to recover complex-valued images with quantitative absorption and phase contrasts [21–23]. The reconstructed wavefront at the sample exit plane is related to the far-field diffraction pattern by a Fourier transform [24]. Assuming that this wavefront has propagated a relatively short distance, the propagated wavefront will give the same diffraction pattern as the initial wavefront. As a result, the reconstruction algorithm cannot distinguish the initial exit wavefront from its propagated versions. This is known as the propagation ambiguity in the phase retrieval process. For the coherent diffractive imaging method, it has been pointed out that the real-space constraints used in the iterative reconstruction process eliminates the propagation non-uniqueness and effectively determines the plane where the wavefront is reconstructed [25,26]. Similarly, the reconstruction plane in ptychography is determined by the illumination function used in the reconstruction [27]. The features on the reconstruction plane appear sharp as in focus, while the features away from the reconstruction plane that are more than the depth of field lose sharpness, becoming out of focus. In this manner, the optical sectioning can be numerically realized by propagating the incident illumination function to target image planes and reconstructing the focus stack plane by plane, using the same ptychographic dataset.

The sectioning capability, i.e., the depth resolution, is limited by the depth of field for a ptychographic imaging system. Figure 1 illustrated a typical experimental setup for a ptychography measurement using a focused beam. The confined illumination delivered by focusing optics has an intrinsic depth of focus. Within this range, the propagation effect of the illumination wavefront is negligible, as illustrated by the red box in Fig. 1. The depth of focus is determined by the numerical aperture ${\mathrm{NA}}_o$ of the focusing optics by $\lambda /{\mathrm{NA}}_o^2$ [28]. The diffraction- based imaging techniques directly measure the scattering signal from sample. Therefore, it is capable of realizing a larger detection numerical aperture ${\mathrm{NA}}_d$ and using a spatial frequency signal beyond the maximum scattering angle defined by the optics ${\mathrm{NA}}_o$ for achieving better spatial resolution. The enlarged detection numerical aperture ${\mathrm{NA}}_d$ shrinks the depth of field of the imaging system accordingly, as indicated by the green box in Fig. 1. The sample is placed at the focal plane in the left panel of Fig. 1 for direct comparison with the depth of focus and the depth of field, while the ptychography measurements can be performed at defocal planes without sacrificing the achievable resolution. The right panel of Fig. 1 shows a typical far-field diffraction pattern collected with an x-ray beam focused by multilayer Laue lenses (MLLs). The bright region selected by the red frame represents the natural divergence of the focused beam and corresponds to the ${\mathrm{NA}}_o$. The scattering signal actually extends to the edge of the cropped detection array as indicated by the green frame, which defines ${\mathrm{NA}}_d$ and, thus, the depth of field. The narrower depth of field, defined by the detection, enforces a more restrictive limitation on sample thickness and is considered as an undesirable side effect of pursuing high spatial resolution. On the other hand, for the purpose of optical sectioning, a short depth of field is actually beneficial, because it offers better depth sensitivity. As the conventional focus stacking method requires the axial step matching the depth of field to properly capture sample features through the focus, the narrowed depth of field thus demands more sectioning steps, i.e., more measurements. For the ptychography reconstruction, this requirement does not introduce extra burden on collecting more datasets, because the sectioning is numerically conducted by propagating the illumination function with desired steps and running reconstructions accordingly.

 

Fig. 1. Schematic experimental setup for a ptychography experiment. The depth of field in the reconstructed image is defined by the signal with the maximum scattering angle collected by the detector, which is typically narrower than the depth of focus defined by the focusing optics. The right panel shows representative far-field diffraction amplitude using an x-ray beam focused by MLLs. The green and red boxes illustrate the numerical apertures determined by detection and focusing optics, respectively.

Download Full Size | PDF

For detecting the in-focus features, the wavelet transform works better than Fourier transform, since it employs locally oscillating wavelets instead of globally oscillating sinusoids in Fourier transform, allowing the capture of both location and frequency information [29,30]. The complex-valued wavelet transform works better than the real-valued counterpart, because this redundant representation relaxes the inherent constraints in the real-valued case by using individual bases for the real and imaginary components [3,31], and provides more information as the phase of the coefficient carries detailed frequency components, while the magnitude part provides the corresponding weights [32].

We use the dual-tree complex discrete wavelet transform [33,34] for the image fusion, which uses a set of orthonormal and smooth bases and is analytically invertible [30]. The maximum- absolute-value selection rule is used to pick up the most pronounced wavelet coefficients for generating the final sharp image. The image fusion algorithm is adopted and modified from [29] and consists of the following four steps:

The proposed method is validated using an experimental dataset collected at the hard x-ray nanoprobe beamline, National Synchrotron Light Source II (NSLS-II). The sample is a 10 μm thick silicon wafer, with gold and nickel oxide nanoparticles prepared on its front and rear surfaces, respectively. The wafer was scanned across a 12 keV nano-focused x-ray beam produced by MLLs. The details of the experimental setup can be found in Refs. [37,38]. The front surface of the sample was placed 10 μm downstream from the focal plane, and the beam was diverged to ∼100 nm. A Fermat spiral trajectory [39] was used to scan a $1.2\times 1.2{\mathrm{\mu m}}^2$ area with 15 nm radial step size to provide a generous overlapping condition [40–42] to reconstruct two sample planes. A Merlin pixel-array detector [43] was placed 0.5 m downstream and recorded 2035 data frames for the entire 2D ptychography scan. The dataset used in this Letter was taken with a 1 s exposure time per scan point. Typical diffraction amplitude is shown in the right panel of Fig. 1. The cropped $188\times 188$ data array gives a 5 nm reconstruction pixel size and ~1 μm depth of field. The wavefront of the focused beam was determined from a ptychography reconstruction using a dataset collected with only the gold particles inside the field of view.

The dataset was first reconstructed using multi-slice ptychography with 1000 iterations of a difference map algorithm [21] using the single-slice reconstruction results as the starting guess for the object functions. The obtained phase images on the front and rear surface of the sample are shown in Figs. 2(a) and 2(b), respectively. The nanoparticles are sharply reconstructed on their corresponding layers, and they agree well with the Au $L$ and Ni $K$ fluorescence images, as shown in Figs. 2(c) and 2(d), which were measured by a mesh scan over the same area at the same axial position with a 20 nm step size and 1 s dwell time. Multiplying these two reconstructed images (i.e., summing their phases) gives a single projection image with in-focus features over a 10 μm range, as shown in Fig. 2(e), which significantly exceeds the estimated 1 μm depth of field.

 

Fig. 2. Reconstructed phase images from multi-slice ptychography. (a) Recovered image on the front surface. (b) Recovered image on the rear surface. (c) Au $L$ fluorescence map for the gold nanoparticles on the front surface. (d) Ni $K$ fluorescence map for the nickel oxide nanoparticles on the rear surface. (e) Projection image with an extended depth of field obtained by multiplying two reconstructed slices.

Download Full Size | PDF

To generate the focus stack, the x-ray wavefront at the front sample surface was propagated from $-2$ to 12 μm in 1 μm steps to match the depth of field. The front and rear sample surfaces were at $z=0\mathrm{\mu m}$ and $z=10\mathrm{\mu m}$, respectively. Single-slice ptychography reconstructions with 1000 difference map iterations were performed 15 times, each time using one fixed probe from 15 propagated x-ray wavefronts. Figures 3(a)–3(h) display eight of the obtained phase images with 2 μm axial increments. At $z=0\mathrm{\mu m}$ and $z=10\mathrm{\mu m}$ planes, the particles on the front and rear surfaces are sharply reconstructed, each overlaid with a blurry image of the features on the other plane. At other $z$ planes, the image quality on both surfaces is compromised to a certain extend. Using this focus stack, the final image is obtained using the aforementioned algorithm described in the previous section with three-level wavelet transforms. Figure 3(i) shows the merged image with features on both slices sharply presented. The obtained image is consistent with the result given by the multi-slice ptychography. Since the sample only consists of two planes, and the thickness of each plane is thinner than the depth of field, two single-slice reconstructions at $z=0\mathrm{\mu m}$ and $z=10\mathrm{\mu m}$ planes are sufficient to give a merged image with a very similar quality as the one obtained from 15 reconstructions.

 

Fig. 3. Image fusion using complex-valued discrete wavelet transform. (a)–(h) Single-slice reconstruction images with the illumination function propagated from $-2$ to $+12\mathrm{\mu m}$ with a 2 μm step size. (i) Merged image with an extended depth of field using complex-valued discrete wavelet transform.

Download Full Size | PDF

In the ptychography reconstruction process, the majority of the computation time is for Fourier transform. The propagation from the sample to the detector can be described by a Fresnel propagation with one Fourier transform, and the propagation between adjacent planes in the multi-slice case can be modeled by the angular spectrum method which contains two Fourier transforms [44]. Thus, the total computation time in a multi-slice ptychography reconstruction with $N_s$ slices can be estimated as $2(N_s-1)\ast 2\mathrm{FT}+2\mathrm{FT}=(4N_s-2)\mathrm{FT}$, where the first term is for propagations between adjacent slices, the second term is for propagations between the last slice and the detector plane, FT denotes the computation time for one Fourier transform over all data frames, and a general factor of 2 represents the forward and backward propagations in each cycle. However, repeating the single-slice ptychography reconstruction $N_s$ times only takes $N_s\ast 2\mathrm{FT}$. In our case, the entire image fusion process only took less than 1 s. For this two-layer sample, the proposed method increases the computation efficiency by one-third compared with the multi-slice ptychography approach. Considering the number of slices, $N_s$ can become remarkably large for a continuous sample with extended dimension, and the multi-slice ptychography approach demands at least a factor of $N_s$ more memory; numerically generating the focus stack with single-slice ptychography reconstructions in the proposed method is expected to bring a more significant improvement on the computational efficiency.

The freedom and simplicity to numerically tune the axial sectioning spacing of the focus stack make this method very flexible to favor practical applications. Wavelet transform-based image processing techniques, such as denoising by thresholding coefficients [45], can be easily implemented into this method. It should be noted that the numerical sectioning capability of this method relies on an assumption that the probe profile is not altered through the sample thickness, i.e., with negligible multiple scattering effects. Otherwise, the multi-slice approach works better than the focus stacking method [46]. Despite the computational burden, the axially separated slices obtained from multi-slice ptychography enable unique capabilities, such as effectively reducing the projection numbers for tomography [18] or 3D imaging without sample rotation [9].

We present a new method to extend the depth of field for ptychography by numerically generating the focus stack from single-slice ptychography reconstructions with propagated illumination functions and merging the in-focus features with a final sharp image using complex-valued discrete wavelet transform. This method simplifies the data acquisition process compared with the conventional focus stacking method, and it shows the potential to significantly improve the computation efficiency compared with the multi-slice ptychography method. It offers a new opportunity to remove the limitation on sample thickness and obtain high-resolution images from materials with extended dimensions.