Ptychography is a coherent diffractive imaging method that can provide a diffraction-limited, robust reconstruction of the sample’s complex transmission function without the use of high-quality optics. However, the scanning nature of conventional X-ray ptychography unavoidably requires the mechanical motion of either the illumination probe or the sample. In order to avoid overhead related to breaking and acceleration for every scan position, so-called fly-scan methods were developed. Here, we present an improved variant that removes the limitation of continuous scanning along a linear scanning path and allows for ptychographic reconstruction of scans taken along an arbitrary 2D continuous trajectory. We also demonstrate numerically and experimentally that our method provides significantly improved robustness against noise, particularly for larger fly-scan steps, i.e. sample shift during an exposure, which will gain importance with the advent of 4th generation synchrotron sources, where the available coherent flux may be increased by orders of magnitude. Finally, we show that the use of a spiral scan continuous trajectory alleviates significantly raster grid artifacts.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Upcoming 4th generation synchrotron sources  are going to provide significantly increased brightness and, in combination with more efficient X-ray optics, potentially lead to thousand-fold increase in coherent flux . One of the methods that have the potential to benefit most is phase-retrieval high-resolution imaging, in particular, X-ray ptychography . Ptychography can reconstruct information about both absorption and relative phase shift of the imaged specimen and in combination with 3D imaging techniques like tomography, quantitative values of the local refractive index inside the specimen can be obtained [4–7]. Conventional ptychography is a scanning method, for which partially overlapping regions of the imaged specimen are illuminated by a confined probe and sufficiently sampled diffraction patterns are collected. Using constraints given by a common illumination probe and overlap between the adjacent positions, both the complex-valued transmission function of the specimen and the illumination probe can be reconstructed.
Despite continuous improvements in the instrumentation for high-resolution ptychographic X-ray computed tomography (PXCT) [6–8], measurement of a single high-resolution tomogram can take up to 22 hours . The scanning time in  was split roughly equally between exposure time and mechanical overhead, i.e. acceleration between scan positions, of the scanning actuators. However, with advances in synchrotron sources and X-ray optics efficiency, the scanning overhead is soon going to become the critical bottleneck. For ptychography, reducing or eliminating the scanning overhead will be crucial to profit from the increased brightness of 4th generation synchrotron sources.
One potential solution that avoids the mechanical overhead is provided by the so-called fly-scan or continuous ptychography methods that were proposed in recent years and successfully applied for high-resolution ptychography [9–11]. The idea of fly-scan ptychography is based on recording far-field diffraction patterns during the continuous motion of the sample. This acquisition strategy works well provided that the motion during the acquisition does not exceed a handful of resolution elements. The conventional fly-scan methods [9–11] carry out the reconstruction using a coherent mode decomposition of the probe . There are three main limitations in the current fly-scan methods:
- Higher noise sensitivity compared to conventional step-based ptychography
- Reconstruction methods based on mode decomposition are restricted to a linear path with constant velocity
- High computational cost
The first one is mainly caused by additional complexity of the ptychography inversion task due to reduced speckle contrast caused by the continuous motion. Additionally, the limitation of the scan trajectory to straight lines may result in periodic artifacts in the reconstruction  and leads to unavoidable breaking and acceleration at the end of each scanning path line that causes vibrations and overhead. While a short pause after a full line may currently not be critical for the scan performance, as the available coherent flux from synchrotron X-ray sources increases , there will be a substantial need to move to continuous 2D smooth trajectories.
Here, we introduce an alternative approach to fly-scan ptychography that explicitly uses the prior knowledge of the trajectory of the probe across the object and thus avoids the reconstruction of additional illumination modes. This improves its robustness to noise and additionally allows the use of an arbitrary path in 2D.
2. Optimization problem
The ptychography reconstruction can be seen as an optimization task that seeks a common illumination probe Pr and a common complex-valued transmission function Or in order to minimize the difference between the expected diffraction intensity and the measured photon distribution for all scan positions, where r and k are the set of 2D Cartesian coordinates perpendicular to the probe propagation direction in real and reciprocal space, respectively. Assuming a monochromatic spatially coherent illumination, the expected diffraction intensity at the i-th scan position can be expressed as14]. Assuming that the velocity and direction of the translation are constant in all acquired frames, the decoherence effects are identical as well and they can be recovered using ptychography as a sum of mutually incoherent orthogonal illumination modes . This idea is used in the conventional multimodal fly-scan ptychography [9, 10], which describes the expected intensity as 15], the operator has to be constant for all scan positions as well. Consequentially, the scan path is limited to parallel straight lines only. Since periodicity in the scan pattern may result in artifacts in the reconstructed object in the conventional  but also in the fly-scan ptychography [13, 17], aperiodic patterns with a uniform distribution of overlap are preferred. Additionally, as it was already mentioned, the turnaround at the end of each straight line leads to scanning overhead. Both of these problems can be avoided if a spiral-like scan path is used. However, a different fly-scan direction in every scan position will effectively result in variable coherence properties, therefore it is not possible to use the conventional mode reconstruction approach. Instead, we propose here to directly optimize the discretized forward model, given in Eq. 4, that describes the measured intensity as an incoherent sum of diffraction patterns resulting from shifted objects illuminated by an identical probe. Since interferometric positioning systems with a sampling frequency of 1 MHz are available while the frame rate of modern X-ray detectors is limited to several kHz , we can assume that the sample position is precisely known for each discretized position R(ti + j T/N) within the i-th exposure. If sufficiently sampled positions are not available, approximation of the path by a smooth spline between the measured nodes should be sufficient if the velocity and direction changes are slow. Note that a good time synchronization between the sampled positions and the collected frames becomes important in the case of non-straight scan trajectories.
Direct optimization of Eq. 4 provides the same powerful constraint used in the conventional step-based ptychography, namely, a single illumination probe common to all scan positions, thus avoiding the need of recovering additional mutually incoherent illumination modes. The fact that data from the continuous scan are fully defined only by the known trajectory and a single unknown mode leads to a much stronger constraint, which results in improved reconstruction quality and faster convergence compared to the conventional fly-scan ptychography modal reconstruction and allows for an arbitrary scan trajectory and speed.
The collected photon counts per frame can be significantly lower compared to the conventional step-scan based ptychography, in particular if the illumination beam is much larger than the size of the resolution element, due to the requirement of a shorter step size along the fly-scan direction in contrast to the step scan . Therefore, the optimal object and the illumination probe that minimize the distance between the measured data and the model should be preferably sought by statistics-aware ptychography algorithms [19–21] that from our experience handle dose fractionation better between many noisy diffraction patterns. However, the presented method can be adapted to other standard methods such as the difference map (DM)  or the extended ptychographic iterative engine (ePIE)  as well.Eq. 7a and Eq. 7b, the optimization was solved by the least-squares maximum likelihood solver presented in . The computational cost per iteration of the arbitrary-path fly-scan (A-fly) method is comparable with the conventional fly-scan method.
An interesting similarity can be seen between the proposed A-fly method and Richardson-Lucy (RL) deconvolution [23, 24] if the operator is equal to the identity operator, i.e. the propagation distance from the sample to the detector is zero. In that case, the gradient update given by Eq. 7a becomes identical for a positive valued image ur to an update formula of the RL deconvolution applied on a smeared and noisy image relaxed by a positive-valued illumination probe Pr
3. Experimental demonstration
Our A-fly ptychography was experimentally verified at the cSAXS beamline, Swiss Light Source (SLS), Paul Scherrer Institut (PSI), Switzerland. A ptychographic experiment was performed at 6.2 keV photon energy defined by a double-crystal Si(111) monochromator. A 6 µm illumination probe was formed by a Fresnel zone plate with 60 nm outermost zone width and a diameter of 170 µm, fabricated at the Laboratory of Micro and Nanotechnology (LMN), PSI, Switzerland. The diffraction intensities were measured using an Eiger 500k detector . The maximal spatial resolution was limited by the collected diffraction angle to 38 nm. The imaged object was a Siemens star test pattern made of hydrogen silsesquioxane (HSQ) and coated with a 10 nm layer of Ir using atomic layer deposition . Five datasets with line fly-scan trajectory, see Fig. 1, of the test pattern with a different smearing distance were collected. In order to maintain the total imaging dose constant and roughly equal to 800 photons per object pixel, i.e. 0.6 photon/nm2, the sample velocity was kept to 19 µm/s, i.e. 0.5 object pixel per ms, and only the exposure time was changed from 2.5 to 12.5 ms, resulting in a smearing distance between 1.2 and 6.2 object pixels. The spacing along the direction perpendicular to the continuous sample move was 1 µm with pseudo-random offset up to 0.1 µm to reduce periodic artifacts.
Measured data were processed by the conventional multimodal fly-scan reconstruction method  and by the new A-fly method. The initial guess of the illumination probe was provided from the reference step scan shown in Fig. 2(a). For both methods, the number of modes describing coherence properties N was equal to the sample shift per exposure in pixels. However, while for the A-fly method all the illumination modes were assumed to be mutually equal and only shifted along the scan path, the initial guess of the coherence modes for the conventional multimodal fly-scan reconstruction was given as an orthogonal set of N first modes from shifted illumination probes along the average scan direction as used in . The reconstruction with both methods was performed by 200 iterations of the iterative least-squares maximum likelihood method using compact sets (LSQ-MLc) presented in .
The resulting reconstructions were compared with a reference step-scan reconstruction of the same region shown in Fig. 2(b). The spatial resolution given by the crossing of the Fourier ring correlation between the reference and each reconstruction with the 1-bit threshold  was used as the metric of the reconstruction quality. Figure 3 shows the dependence of the spatial resolution on the scanning step. As expected, a clear trend of quality deterioration with increase of the sample shift during the exposure can be seen for both methods. However, the resolution of the classical fly-scan method dropped by 42% when the step increased from 1.2 to 6.2 object pixels while the resolution of the A-fly method dropped only by 14%. The quality improvement between the A-fly and the conventional fly-scan method for the 6.2 pixel step can be seen in Fig. 2(c) and 2(d). Additionally, Fig. 4(a) shows an A-fly reconstruction of a dataset with identical sample velocity along a spiral scan path, with fly-scan step 6.2 pixel and radial spacing of 1 µm. A reconstruction of the same spiral trajectory dataset by the conventional fly-scan method is shown in Fig. 4(b). Compared to the linear trajectory reconstructions in Fig. 2(c) and 2(d), the spiral scan in combination with our A-fly method helped to even further suppress artifacts in the low spatial frequencies.
4. Simulated parameter scans
In order to provide better insight into the difference between the classical multimodal method and the A-fly method, the behavior of the method was tested using realistic simulations. The experimental parameters were identical to the measurements in the previous section and the high quality reference reconstruction of the Siemens star in Fig. 2(a) was used as the object model. Figure 5 shows the reconstruction quality as a function of the imaging dose with a continuous step per exposure equal to 10 pixels. Additionally, a reconstruction from a conventional step scan with 1 µm step along both axes was added for comparison.
Performance of both fly-scan methods is rather comparable for very low photon dose, however the A-fly method follows the trend of the step scan better when the dose is increased. Towards higher photon dose, the A-fly method can provide the same resolution as the conventional fly-scan method with roughly 10-times fewer photons, see for example a resolution of 41 nm in Fig. 5. Despite this, the A-fly method still needs at least 4 times more photons than the conventional step scan to provide similar quality. Note that all methods can provide Nyquist frequency limited resolution with sufficient photon flux.
A second set of simulations depicted in Fig. 6 shows the dependence of the resolution on the fly-scan step, i.e. the shift of the sample during a single exposure, for a constant dose of 1000 photons per pixel. The reconstruction quality of all methods is comparable if the sample moves less than 1 pixel per exposure time. However, the quality difference grows with longer steps for which a higher number of modes is needed to model the reduced speckle contrast in the measured data. This demonstrates that the stronger constraints of the A-fly reconstruction method is an important step towards increased noise robustness of the fly-scan deconvolution task, in particular for larger steps along the fast axis. Given a fixed imaging dose, negligible dead time between frames and fixed real-space overlap in the axis perpendicular to the direction of the continuous move, the only experimental parameter that affects image quality is the sample stage speed. Specifically, the simulated results have the same quality as a step scan if the continuous step is smaller than one resolution element. As soon as the continuous step is larger than one resolution element the quality decreases, as shown in Fig. 6. This implies that for continuous scans the real-space ratio between the step size and beam diameter in the fly-scan direction is not a relevant parameter. Therefore, fly-scan experiments should be designed to approach the 1 pixel step length per exposure as much as it is allowed by the used detector to improve the reconstruction quality.
We have demonstrated that our new variation of the fly-scan ptychography called A-fly method provides better reconstruction quality than the common fly-scan method at the same imaging dose or equivalently same reconstruction quality with longer step size. Additionally, we have demonstrated that our new method allows to scan along non-straight continuous scan trajectory leading to more isotropic imaging quality and reduction of periodic artefacts.
We believe that the A-fly method with continuous 2D trajectories will become an important tool for fast scanning in future high brightness 4th generation synchrotrons, where the positioning overhead of the classical step ptychography and even the line-to-line overhead will become a bottleneck  but at the same time the radiation damage caused by inefficient use of X-ray dose would lead to deteriorated imaging quality. Nowadays, the presented method is important mainly for ptychographic imaging with nanofocused beam [28–30], where the small beam diameter leads to very short exposure time per scan position and thus unacceptable overhead in the classical step-scan ptychography method. Additionally, the presented fly-scan experiments already required up to 400 Hz continuous framerate acquisition for a rather thin sample at the Swiss Light Source (SLS) that is a small 3rd generation synchrotron. Upgrade of SLS to 4th generation in combination with imaging of thicker samples may require continuous framerate >100 kHz to keep the used fly-scan step lengths. Therefore, the framerate requirements will outrun detector speed developments, which already face a strong hurdle to exceed 10 kHz sustained readout. Therefore, the use of longer fly-scan steps may become unavoidable in order to fully use the available coherent X-ray flux. In that case, the presented A-fly method would either help to use longer steps without deterioration of the imaging quality and thus speed up the data acquisition or improve the ptychographic reconstruction quality compared to the conventional fly-scan method using the same scan parameters.
We thank Andreas Menzel for his comments to this manuscript. Preliminary simulations related to this method were presented in .
References and links
3. J. Rodenburg, A. Hurst, A. Cullis, B. Dobson, F. Pfeiffer, O. Bunk, C. David, K. Jefimovs, and I. Johnson, “Hard-X-ray lensless imaging of extended objects,” Phys. Rev. Lett. 98, 034801 (2007). [CrossRef] [PubMed]
4. M. Dierolf, A. Menzel, P. Thibault, P. Schneider, C. M. Kewish, R. Wepf, O. Bunk, and F. Pfeiffer, “Ptychographic X-ray computed tomography at the nanoscale,” Nature 467, 436 (2010). [CrossRef] [PubMed]
5. A. Diaz, P. Trtik, M. Guizar-Sicairos, A. Menzel, P. Thibault, and O. Bunk, “Quantitative X-ray phase nanotomography,” Phys. Rev. B 85, 020104 (2012). [CrossRef]
6. M. Holler, A. Diaz, M. Guizar-Sicairos, P. Karvinen, E. Färm, E. Härkönen, M. Ritala, A. Menzel, J. Raabe, and O. Bunk, “X-ray ptychographic computed tomography at 16 nm isotropic 3D resolution,” Sci. Rep. 4, 3857 (2014). [CrossRef] [PubMed]
7. M. Holler, M. Guizar-Sicairos, E. H. Tsai, R. Dinapoli, E. Müller, O. Bunk, J. Raabe, and G. Aeppli, “High-resolution non-destructive three-dimensional imaging of integrated circuits,” Nature 543, 402–406 (2017). [CrossRef] [PubMed]
8. M. Holler, J. Raabe, R. Wepf, S. H. Shahmoradian, A. Diaz, B. Sarafimov, T. Lachat, H. Walther, and M. Vitins, “OMNY PIN-A versatile sample holder for tomographic measurements at room and cryogenic temperatures,” Rev. Sci. Instrum. 88, 113701 (2017). [CrossRef] [PubMed]
10. P. M. Pelz, M. Guizar-Sicairos, P. Thibault, I. Johnson, M. Holler, and A. Menzel, “On-the-fly scans for X-ray ptychography,” Appl. Phys. Lett. 105, 251101 (2014). [CrossRef]
11. J. Deng, Y. S. Nashed, S. Chen, N. W. Phillips, T. Peterka, R. Ross, S. Vogt, C. Jacobsen, and D. J. Vine, “Continuous motion scan ptychography: characterization for increased speed in coherent X-ray imaging,” Opt. Express 23, 5438–5451 (2015). [CrossRef] [PubMed]
13. X. Huang, H. Yan, M. Ge, H. Öztürk, E. Nazaretski, I. K. Robinson, and Y. S. Chu, “Artifact mitigation of ptychography integrated with on-the-fly scanning probe microscopy,” Appl. Phys. Lett. 111, 023103 (2017). [CrossRef]
15. M. Odstrcil, P. Baksh, S. Boden, R. Card, J. Chad, J. Frey, and W. Brocklesby, “Ptychographic coherent diffractive imaging with orthogonal probe relaxation,” Opt. Express 24, 8360–8369 (2016). [CrossRef] [PubMed]
17. X. Huang, K. Lauer, J. N. Clark, W. Xu, E. Nazaretski, R. Harder, I. K. Robinson, and Y. S. Chu, “Fly-scan ptychography,” Sci. Rep. 59074 (2015).
18. I. Johnson, A. Bergamaschi, H. Billich, S. Cartier, R. Dinapoli, D. Greiffenberg, M. Guizar-Sicairos, B. Henrich, J. Jungmann, D. Mezza, A. Mozzanicaa, X. Shia, and C. Tintia, “Eiger: a single-photon counting X-ray detector,” J. Instrum 9, C05032 (2014). [CrossRef]
19. P. Thibault and M. Guizar-Sicairos, “Maximum-likelihood refinement for coherent diffractive imaging,” New J. Phys. 14, 063004 (2012). [CrossRef]
21. M. Odstrčil, A. Menzel, and M. Guizar-Sicairos, “Iterative least-squares solver for generalized maximum-likelihood ptychography,” Opt. Express 26, 3108–3123 (2018). [CrossRef]
23. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62, 55–59 (1972). [CrossRef]
24. L. B. Lucy, “An iterative technique for the rectification of observed distributions,” Astron. J 79, 745 (1974). [CrossRef]
25. M. Guizar-Sicairos, I. Johnson, A. Diaz, M. Holler, P. Karvinen, H.-C. Stadler, R. Dinapoli, O. Bunk, and A. Menzel, “High-throughput ptychography using Eiger: scanning X-ray nano-imaging of extended regions,” Opt. Express 22, 14859–14870 (2014). [CrossRef] [PubMed]
26. J. Elam, D. Routkevitch, P. Mardilovich, and S. George, “Conformal coating on ultrahigh-aspect-ratio nanopores of anodic alumina by atomic layer deposition,” Chem. Mater 15, 3507–3517 (2003). [CrossRef]
28. J. Deng, D. J. Vine, S. Chen, Y. S. Nashed, Q. Jin, N. W. Phillips, T. Peterka, R. Ross, S. Vogt, and C. J. Jacobsen, “Simultaneous cryo X-ray ptychographic and fluorescence microscopy of green algae,” PNAS 112, 2314–2319 (2015). [CrossRef] [PubMed]
29. J. Deng, D. J. Vine, S. Chen, Q. Jin, Y. S. Nashed, T. Peterka, S. Vogt, and C. Jacobsen, “X-ray ptychographic and fluorescence microscopy of frozen-hydrated cells using continuous scanning,” Sci. Rep. 7445 (2017).
30. J. Deng, Y. P. Hong, S. Chen, Y. S. Nashed, T. Peterka, A. J. Levi, J. Damoulakis, S. Saha, T. Eiles, and C. Jacobsen, “Nanoscale X-ray imaging of circuit features without wafer etching,” Phys. Rev. B 95, 104111 (2017). [CrossRef] [PubMed]
31. M. Odstrcil, A. Diaz, M. Holler, and M. Guizar-Sicairos, “Path toward fast, high-resolution and more photon-efficient X-ray ptychography,” in COSI (OSA, 2017), pp. CW2B.