Abstract
A differential optical sectioning microscopy with structured-illumination (DOSM-SI) with enhanced axial precision is explored in this paper for three-dimensional (3D) measurement. As the segment of data on the linear region of the contrast depth response curve (CDR) is very sensitive to variation of the height information, the DOSM-SI introduces a new CDR2 with an axial shift to intersect the linear region of the CDR1, which is achieved by using two charge-coupled detectors (CCDs) in the optical path. The CCD1 is located on the imaging plane and the CCD2 is displaced from the imaging plane. The difference between the CDR1 and CDR2 for each pixel is defined as the differential depth response curve (DCDR). Further, the zero-crossing point of the DCDR for each pixel is accurately extracted using the line-fitting technique, and finally, the sample surface can be reconstructed with a high resolution and precision. Since the slope around the zero-crossing point of the DCDR is apparently larger than that of near the focal position, an enhanced resolution and precision can be realized in DOSM-SI. The experiments and theoretical analysis are elaborated to demonstrate the feasibility of DOSM-SI.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Functional surfaces with micro-optical components, artificial metamaterials, terahertz devices and MEMS structures have been the focus of research because they are widely used in various fields including integrated circuits, material sciences and medical cures [1,2]. The surface topography of these microstructures needs to be satisfactorily measured since it always has significant influences on the performances of micro products. For a few years, optical sectioning with structured illumination microscopy (OSM-SI) has received much interests in the domains such as biomedicine and semiconductor due to its unique optical sectioning capability and high imaging speed [3–5]. OSM-SI is also a wide-field and non-interferometric tool for three-dimensional profiling of smooth and rough surfaces with nanometer depth resolution [6–9].
The basic principle of OSM-SI is to project a single spatial frequency pattern onto the sample. The contrast distribution of the observed image is first obtained in the x-y plane by implementing the contrast decoding algorithm. After that, the contrast depth response curve (CDR) for each pixel can be obtained while the sample is scanned in the z-direction through the focal plane. The contrast of a pixel becomes maximal when the corresponding object point is in the focal plane of the microscope objective. Consequently, the height information for each pixel can be realized by detecting the peak position from the corresponding CDR.
The algorithms used in OSM-SI are generally classified into the contrast decoding algorithms and the height determination algorithms. Researchers around the world have constantly concentrated on the contrast decoding techniques for improving the imaging efficiency and sensitivity [10,11]. For instance, Neil et.al propose a phase-shifting technique, which is widely used in the conventional OSM-SI and at least three raw images with a predefined phase-shift between two adjacent images are acquired [12]. Santos et al. present a method named HiLo imaging, which utilizes two images obtained under structured illumination to decode an optically sectioned image [13]. Trusiak et.al introduce a new contrast decoding algorithm that combines fast and adaptive two-dimensional empirical mode decomposition method with the Hilbert spiral (HS) transform [14]. The existing height recovery methods are typically based on peak-extraction techniques including the Gaussian function fitting, polynomial fitting, and centroid method, of which the Gaussian fitting method is the most accurate algorithm among these peak-extraction techniques [15].
In OSM-SI, the axial resolution and accuracy can be characterized by the slope around the focus point. Generally, the axial precision of the OSM-SI can be improved by increasing the numerical aperture (NA) or decreasing the spatial frequency of the illuminated pattern since the slope around the peak position is increased [16]. However, the working distance and the field of view is also decreased that inevitably result in low efficiency for large-field measurement. Further, we note that the data near the CDR peak is very insensitivity to variations of the axial position of the sample, so that the peak extraction methods restrict further improvement of the OSM-SI axial resolution and accuracy. In contrast, the linear segment of data on the side of the CDR is very sensitive to variation of the height information, which can be used to reshape the topography with high precision.
In this paper, we explore a new differential optical sectioning microscopy with structured-illumination (DOSM-SI) to enhance the axial accuracy and resolution of the traditional OSM-SI without changing the system parameters such as the spatial frequency of the projected pattern and NA of the objective. Since the side of the contrast depth response curve (CDR) can be considered as linear and very sensitive to variation of the height information, the DOSM-SI introduces a new CDR2 with an axial shift to intersect the linear portion of the conventional CDR1. This is achieved by using two charge-coupled detectors (CCDs) in the optical path, one in the imaging plane and the other CCD before or behind the imaging plane. Then, the differential contrast depth response curve (DCDR) is determined through the difference between two CDRs detected by two differential CCDs. Finally, a linear fit is performed to accurately localize the zero-crossing point of the DCDR and hence the height information for each pixel. As the slope of the DCDR near the zero-crossing point is much larger than that of around the peak position and thus higher accuracy and resolution can be achieved than traditional OSM-SI. The lateral resolution of DOSM-SI can reach 200nm with high NA objective lens. The validity of such DOSM-SI technique is confirmed both theoretically and experimentally in this work, providing that this method is highly promising for high-precision measurement.
2. Measurement principle and simulation
2.1 differential contrast depth response evaluation
The measurement system of the proposed DOSM-SI is shown in Fig. 1. With this technique, a sinusoidal fringe pattern generated by the digital micro-mirror device (DMD) is transmitted through the tube lens 1, splitter mirror 1 and the objective lens onto the sample which located in the focal plane of the microscope objective. The reflected patterns from the sample surface is divided into two beams after passing the splitter mirror 2, and received by two CCDs, respectively. The CCD1 is situated in the imaging plane and the CCD2 is located behind or before the imaging plane. In conventional OSM-SI method, only the CCD1 is used to capture the reflected image, and the light-intensity distribution of the captured signals can be expressed as (ref [12].)
whereis the background intensity,describes the contrast of illuminated sinusoidal fringe pattern,and illustrate the spatial frequency and initial phase of the projected sinusoidal fringe, respectively.Then, the pattern is laterally shifted by a pre-defined phase to decode the contrast distribution. Considering all the phase-shifted patterns, the light-intensity can be written as
here is the intensity distribution of thephase-shift fringe pattern. L denotes the total phase-shift steps and i = 1, 2…L. The contrast distribution can be determined byThe differential contrast distribution is defined as
where the number 1 and 2 indicate the CCD1 and CCD2, respectively. Furthermore, the CDR1 and CDR2 for each pixel are obtained by vertically scanning the sample using a piezo-electric transducer (PZT) stage, as illustrated in Fig. 2(a). The DCDR for one pixel is determined by the difference between the corresponding CDR1 and CDR2, as shown in Fig. 2(b).2.2 height determination and axial response analysis
In this section, we will elaborate how to obtain the height information from the DCDR for each pixel. In OSM-SI, the CDR for a specific pixel that describes the relationship between the contrast and the defocus distance from the focal position is analyzed by P.A.Stokseth [17]. According to the analysis, the normalized CDR is approximately Gaussian shaped, that is
where is the focal position of the object point, denotes the defocus distance from the focal position, i.e. the scanning distance, m is constant and , the FWHM is short for full width at half maximum, and can be expressed ashere is the normalized spatial frequency of the projected pattern and , T denotes the period of the projected pattern, NA shows the numerical aperture of the objective lens, describes the central wavelength of light source, n is refraction index. When the CCD2 has a small displacement from the imaging plane, the shape of the curve of the CDR2 is the same as that of CDR1, and the only effect on the axial response curve is the introduction of an axial shift d. The DCDR can be denoted in the form ofwhere d presents the axial shift of the CDR2 which is determined by the distance between the CCD2 and the imaging plane, and 0<d<2FWHM. Traditional OSM-SI realize the measurement through extracting the focal position from the CDR1 for each pixel using the Gaussian fitting technique. In DOSM-SI, we detect the zero-crossing point of the DCDR for each pixel utilizing the line-fitting method to achieve the 3D measurement, as illustrated in Fig. 3.Further, we will justify the feasibility of DOSM-SI based on the zero-crossing point technique and the validity of line-fitting technique. According to Eq. (7), and suppose that , the zero-crossing point of the DCDR can be denoted as
When the position of CCD2 is determined, the axial shift d is a constant for each pixel. Consequently, the relative height information achieved by the zero-crossing point method is the same as that of the conventional focal position technique, demonstrating the feasibility of DSIM based on the zero-crossing point technique.
Numerical simulation is conducted to justify the validity of linear fitting technique described above. Suppose that the NA of the objective is 0.9, the central wavelength of light source = 580nm and the normalized spatial frequency of the projected pattern = 0.32, the FWHM of the CDR is calculated as 440 nm. To analyze the slopes of the DCDR at the zero-crossing point for different axial shifts d, the Eq. (7) is expanded by Taylor series and expressed as
here (n = 0, 1…) are the coefficients of Taylor series expansion of the DCDR. The coefficients ,,andwith d in the range of 0-2FWHM are shown in Table 1. Table 1 demonstrates that for the given axial shift d, coefficients gradually decrease as the order of Taylor series increases and eventually go to zero, and the coefficient is apparently larger than other coefficients . Consequently, the DCDR has good linearity near the zero-crossing point, providing that the line-fitting can be performed to detect the zero-crossing point.2.3 optimal axial shift
To analyze the influence of the axial shift on the measurement accuracy, mathematical analysis and numerical simulation are conducted. In mathematical terms, the uncertainty can be presented by the Gaussian error propagation law, that is [18]
here is the standard deviation of the surface height, denotes the contrast uncertainty caused by the environmental fluctuations or other factors, N indicates the number of the points for fitting, and gives the depth resolution of the measurement. Therefore, a larger slope implies higher sensitivity and axial resolution of the system. According to the Eq. (7), the slope and the shape near the zero-crossing point of the DCDR are affected by the axial shift d, which directly influence the accuracy and the resolution of the DOSM-SI. To determine the optimal axial shift, the Tylor series coefficient that denotes the slope of the DCDR at the zero-crossing point can be expressed aswhere the Eq. (11) is derived from the MATLAB platform. Figure 4(b) illustrates the curve of Tylor series coefficient as a function of axial displacement d, where the FWHM is set to 440nm and the axial displacement d is changing from 0 to 2 FWHM.The optimum axial shift d is equal to 365nm as the slope of the DCDR near the zero-crossing point is maximum. Simulations are elaborated to validate the analysis of the optimal axial shift. Figure 5(a) demonstrates the CDR1, CDR2 and DCDR when d = 360nm and FWHM = 440nm. The linear portion of the DCDR for different d are presented in Fig. 5(b), the slope at the zero-crossing point initially increases, and then decreases, and is maximum when d = 360nm that is consistent with both the analysis described above and the change of Taylor series coefficient in the Table 1.
2.4 Accuracy analysis
According to the Eq. (10), a larger slope can provide higher sensitivity and axial resolution of the system. The slope curves of the CDR and the DCDR are illustrated in Fig. 6, where FWHM = 440nm, = 480nm and d = 360nm. The slope around the focal position is the smallest, which means the contrast uncertainty leads to a large measurement error. In contrast, the slope at the zero-crossing point of the DCDR is apparently larger than that of around the focal position and thus an improved accuracy can be achieved by the DOSM-SI.
Simulations are investigated to reveal that the DOSM-SI can realize an enhanced accuracy than traditional OSM-SI. Figure. 7(a) shows the simulated step having a height of 100nm. Through scanning the object from 0 nm to 1500 nm at the step pace of 30nm in the z-direction, the patterns are collected by the two CCDs. Figures 7(b) and 7(c) are the captured patterns by the CCD1 and CCD2 at the scanning position of 600nm where the FWHM and the axial shift are set to 440nm and 360nm, respectively. The phase-shifting algorithm is applied to evaluate the modulation distribution for each pixel plane in z-scanning. Random noise of 5% fringe intensity is added in the images to match reality more exactly. All the simulations are performed on MATLAB platform.
In DOSM-SI, we use five points near the zero-crossing point to fit a line and then extract the accurate zero-crossing point. To compare with the conventional OSM-SI, all the scanning points are utilized to fit a Gaussian curve for finding the focal position. The 3D reconstructions achieved by the DOSM-SI and the traditional OSM-SI are shown in Figs. 7(d) and 7(e), respectively. Figure. 7(f) illustrates the cross-section profiles (X = 200) of the reconstructions. With the traditional Gaussian fitting method, the root-mean-square error (RMSE) is 5.8 nm, while the RMSE calculated by the DOSM-SI is 2.1 nm. Then, we change the axial shift d from 120nm to 600nm with the step of 120nm, the root-mean-square errors are 3.8nm, 2.4nm, 2.1nm, 2.2nm, 3.0nm, respectively. The results indicate that the DOSM-SI can provide an enhanced accuracy than the conventional OSM-SI.
3. Experiments
To further verify the feasibility of the developed method, experiments are conducted according to the theory described above. The schematic of the measurement system is shown in Fig. 8, which mainly contains a broadband light source (LED illuminant enveloped by Gaussian function whose central wavelength is 580nm and bandwidth 160nm), a DMD made by TI company that contains 1024 × 768 pixels with the pixel size 13.6μm, the CCD1 and the CCD2 share the same models (WAT-902H) that contains 576 × 768 pixels with the resolution of 6.25 × 6.5μm, a microscopy (100X, NA = 0.9, Olympus), a PZT scanning stage (the resolution is ± 0.5 nm, PI company) and other fixed parts are provided by the Daheng optics company. The period of the projected pattern is eight pixels and the normalized spatial frequency v is calculated as 0.3.
3.1 Axial sensitivity test
Before the measurement, we need to determine the optimal axial shift in the experiment. Firstly, the plate wafer used as a sample moves along the axial direction with a step of 40nm. After that, an image matching technique is performed using the phase-correlation algorithm to make sure that the pixels of the CCD1 image and CCD2 image are corresponding to the same point of the sample [19]. The CDR1, CDR2 and DCDR for each pixel can be decoded by using the phase-shifting algorithm, and the zero-crossing point of the DCDR can be detected. Finally, we adjust the position of the CCD2 using the x-y stage until the zero-crossing point of the DCDR is in the linear region of the CDR, and this position can be approximately considered as the optimal axial shift.
Figure 9(a) shows the DCDR of the measurement system, where the axial shift d = 360nm. To evaluate the axial resolution of the DOSM-SI system, we first move the sample to the position around the zero-crossing point using the PZT stage. Then, the sample is scanned along the axial direction for five times with a step of 1nm. The differential contrast distribution is calculated for each movement. Figure 9(b) illustrates the differential contrast distribution of 100 pixels selected from one line of the acquired image. As shown in Fig. 9(b), almost every time the differential contrast can be distinguished from each other. Therefore, the resolution of the DOSM-SI can reach 1nm.
To compare the resolution of the DOSM-SI with the conventional OSM-SI, we adjust the sample to the position near the focal point. The sample is scanned for 20 times with a step of 1nm. It can be seen from the Fig. 10(b) that the contrast distribution cross each other when the position difference is 1nm, while the contrast distribution can be distinguished from each other for the position difference 4nm, as shown Fig. 10(a). Consequently, the resolution of the OSM-SI is 4nm which is four times lower than DOSM-SI.
3.2 Experiment with a standard sample
To evaluate the 3D imaging capability of DOSM-SI, we employ the commercial Profilometer (DektakXT, the resolution is ± 0.1nm, Bruker company), the conventional OSM-SI technique, and the proposed DOSM-SI to analyze the standard step sample which is formed by plating a chrome layer on the glass substrate. Through scanning the sample from 0 nm to 1600 nm at the step pace of 40 nm in z-direction, the patterns with a period of 8 pixels are captured by two CCDs and saved in the computer. Generally, the more phase-shift steps used result in a more accurate contrast evaluation. Meanwhile, the number of the captured images will also increase. Here, a sinusoidal fringe pattern is shifted over the specimen by /4 phase for each z-position to decode the contrast distribution. Figure 11(a) shows the image detected by the CCD1 at the scanning position of 640nm, where the sample is clear imaged on the CCD1. Figure 11(b) denotes the image captured by the CCD2 at the scanning position of 1000nm.
Then, the zero-crossing point of the DCDR for each pixel is extracted using the linear fitting method. The 3D reconstruction of the sample is realized through combining the corresponding zero-crossing position with the scanning pace, as shown in Fig. 12(b). To estimate the accuracy of the proposed method, we make a comparison among the commercial Profilometer, conventional OSM-SI and the DOSM-SI. The step height measured by the commercial Profilometer is selected as the standard height which is 137 nm, as illustrated in Fig. 12(a). To compare with the OSM-SI, we remove the splitter mirror 2 from the measurement system and use the Gaussian fitting technique to reshape the sample. The cross-section profiles presented in Fig. 12(c) show that the height information obtained by the conventional OSM-SI (the blue line) and the DOSM-SI (the red line) are nm and nm, respectively. The results validate that the proposed DOSM-SI achieves an improved accuracy than the traditional OSM-SI. Furthermore, seven repetitive experiments are carried out to estimate the repeatability of the proposed method. The average height obtained from the seven measurements is shown in Fig. 12 (d), which indicates that the repeatability of the measurements is within 1.5 nm.
3.3 Experiment on rough surface
To demonstrate the feasibility of DOSM-SI technique to achieve the rough surface topography, an object with highly rough surface is utilized as the sample in the experiment, as shown in the Fig. 13(a). The reconstruction of the rough surface is illustrated in Fig. 13(b). The DOSM-SI is also a powerful tool for rough surface profiling.
4. Conclusion
There are several comments can be concluded from the above analysis, i.e.
- a) To improve the accuracy of the OSM-SI, high-frequency patterns and a high numerical aperture objective are used to produce a thinner FWHM of the CDR, and a better accuracy can be achieved since the slope around the focal position is improved. Meanwhile, the slope around the zero-crossing of the DCDR is increased, and thus the optimum system parameters used in OSM-SI are also applicable to DOSM-SI.
- b) Since the image matching technique used in the developed method can cause system error, the sub-pixel image matching method will significantly improve the precision of the DOSM-SI.
In conclusion, we explore a novel DOSM-SI method with high axial resolution and precision for 3D measurement. Since the side of CDR can be considered as linear and very sensitive to variation of the height information. The proposed method introduces a new CDR2 to intersect the linear region of the CDR1, this is achieved by using two CCDs in the optical path, one located in the imaging plane and the other CCD placed before or behind the imaging plane. The zero-crossing point of the DCDR for each pixel is accurately determined by solving the equations of the fitting lines, and then, the sample surface can be reconstructed. The theoretical analysis and experiments validate that the accuracy and resolution are enhanced significantly, demonstrating the feasibility of DOSM-SI for high-precision measurement.
Funding
National Natural Science Foundation of China under Grant 61675206, Grant 61875201, Grant 61604154, Grant 61605232, Grant 61605212 and in part by the Sichuan Science and Technology Program under Grant 18ZDZX014, Grant 2014GZ0113, Grant 2017JY0243 and The Instrument Developing Project of the Chinese Academy of Sciences (YZ201616).
Acknowledgments
The authors gratefully thank Y. He and P .H for their assistance during the experiment.
References
1. H. A. Abdelaal, “Functional surfaces for tribological applications: inspiration and design,” Surf. Topogr. 4(4), 014004 (2016).
2. F. A. Müller, C. Kunz, and S. Gräf, “Bio-inspired functional surfaces based on laser-induced periodic surface structures,” Materials (Basel) 9(6), 476 (2016). [CrossRef] [PubMed]
3. J. Mertz, “Optical sectioning microscopy with planar or structured illumination,” Nat. Methods 8(10), 811–819 (2011). [CrossRef] [PubMed]
4. D. Xu, T. Jiang, A. Li, B. Hu, Z. Feng, H. Gong, S. Zeng, and Q. Luo, “Fast optical sectioning obtained by structured illumination microscopy using a digital mirror device,” J. Biomed. Opt. 18(6), 060503 (2013). [CrossRef] [PubMed]
5. J. Y. Lin, R. P. Huang, P. S. Tsai, and C. H. Lee, “Wide-field super-resolution optical sectioning microscopy using a single spatial light modulator,” J. Opt. A, Pure Appl. Opt. 11(1), 015301 (2009). [CrossRef]
6. M. Vogel, Z. Yang, A. Kessel, C. Kranitzky, C. Faber, and G. Häusler, “Structured-illumination microscopy on technical surfaces: 3d metrology with nanometer sensitivity,” Proceedings of SPIE - The International Society for Optical Engineering, 8082. (2011) [CrossRef]
7. G. Haeusler, M. Vogel, Z. Yang, A. Kessel, and C. Faber, “SIM and Deflectometry: New Tools to Acquire Beautiful, SEM-like 3D Images,” Applied Industrial Optics: Spectroscopy, Imaging & Metrology, Optical Society of America (2011).
8. H. Wang, J. Tan, C. Liu, J. Liu, and Y. Li, “Wide-field profiling of smooth steep surfaces by structured illumination,” Opt. Commun. 366, 241–247 (2016). [CrossRef]
9. Z. Xie, Y. Tang, Y. Zhou, and Q. Deng, “Surface and thickness measurement of transparent thin-film layers utilizing modulation-based structured-illumination microscopy,” Opt. Express 26(3), 2944–2953 (2018). [CrossRef] [PubMed]
10. N. Hagen, L. Gao, and T. S. Tkaczyk, “Quantitative sectioning and noise analysis for structured illumination microscopy,” Opt. Express 20(1), 403–413 (2012). [CrossRef] [PubMed]
11. K. Wicker and R. Heintzmann, “Single-shot optical sectioning using polarization-coded structured illumination,” J. Opt. 12(8), 084010 (2010). [CrossRef]
12. M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997). [CrossRef] [PubMed]
13. S. Santos, K. K. Chu, D. Lim, N. Bozinovic, T. N. Ford, C. Hourtoule, A. C. Bartoo, S. K. Singh, and J. Mertz, “Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle,” J. Biomed. Opt. 14(3), 030502 (2009). [CrossRef] [PubMed]
14. M. Trusiak, K. Patorski, and T. Tkaczyk, “Optical sectioning microscopy using two-frame structured illumination and Hilbert-Huang data processing,” 19th Polish-Slovak-Czech Optical Conference on Wave and Quantum Aspects of Contemporary Optics, International Society for Optics and Photonics (2014).
15. P. Heidingsfelder, J. Gao, and P. Ott, “Theoretical analysis of photon noise limiting axial depth resolution for three-dimensional microscopy by incoherent structured illumination,” Opt. Eng. 51(10), 103203 (2012). [CrossRef]
16. D. Dan, B. Yao, and M. Lei, “Structured illumination microscopy for super-resolution and optical sectioning,” Chin. Sci. Bull. 59(12), 1291–1307 (2014). [CrossRef]
17. P. A. Stokseth, “Properties of a Defocused Optical System,” J. Opt. Soc. Am. 59(10), 1314–1321 (1969). [CrossRef]
18. A. Possolo and C. Elster, “Evaluating the uncertainty of input quantities in measurement models,” Metrologia 51(3), 339–353 (2014). [CrossRef]
19. H. Bay, T. Tuytelaars, and L. V. Gool, “SURF: Speeded up robust features,” Proceedings of the 9th European conference on Computer Vision - Volume Part I. Springer, Berlin, Heidelberg (2006). [CrossRef]