Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Speckle pattern shifting deflectometry based on digital image correlation

Open Access Open Access

Abstract

A method based on digital image correlation (DIC) for the surface shape measurement of specular surface by shifting a speckle pattern, which is displayed on an LCD screen, is proposed in this paper. With this method, the deformed information of test surface is encoded within the displacement distribution between the two recorded speckle images before and after the speckle pattern shifted. The displacement distribution is calculated by the DIC algorithm, then the slope data and the surface shape are obtained. The principle and algorithm of speckle pattern shifting deflectometry (SPSD) are described in detail. The correctness and feasibility of the proposed method are verified by simulation, and the source of error is analyzed as well. Finally, the shape of an acrylic plastic plate and a silicon wafer are measured. The experimental result of the proposed method is compared with that of PMD, and the figure error is around 1μm RMS with a measured diameter of about 100mm. This method has the advantages of fast measurement, simple device, low cost and needlessness of reference element. It provides a new approach to measure the shape of specular surface.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Specular surface metrology has a wide range of application, such as optics manufacturing, the evaluation of wafer surface quality, and car body surface measurement, etc. Traditional surface measurement is usually accomplished by interferometer, however, its limited measurement range, large cost, and high sensitivity to environmental influence, make it difficult to fulfill online measurement task. As a competitive metrology, phase measuring deflectometry (PMD) with high accuracy and large dynamic range has attracted an extensive attention [13] for more than one decade. The general measuring process of PMD is to use an LCD screen to display several sinusoidal fringes, a camera next to the screen records the fringe images reflected off the test surface, and the phase distribution related to the test surface is obtained by phase shifting algorithm. Based on the phase distribution, once the slope of the test surface is obtained, the surface can be reconstructed by slope integration method. In order to obtain the phase distribution accurately, N (N ≥ 3) step phase shifting algorithm is usually used, and it is necessary to acquire 2N fringe patterns in two orthogonal directions to complete a measurement, which significantly reduces the efficiency of measurement. To reduce the number of images, some researchers proposed to use color fringe [4] or orthogonal fringe [57], and Tang et al. [8] proposed a speckle pattern deflectometry (SPD) on the basis of PMD by using speckle pattern instead of sinusoidal fringe. In Tang’s SPD, a distorted speckle pattern reflected off the test surface is compared to that reflected off the planar reference element, with the two images the displacement of the speckle is calculated using a two-dimensional digital image correlation technique (2D DIC) [911], and the slope of the test surface is obtained from the speckle displacement. The fringe pattern used in the phase shift algorithm is sensitive to noise [12], but the speckle image used in the DIC technique is robust against noise [11,13]. However, it requires extra efforts to align the reference element to the same position and orientation as the test [8], which if not satisfied will introduce errors, moreover, it is inconvenient for on-line measurement. To overcome the shortcomings of SPD, a novel slope measurement method without using reference element for specular surface based on the 2D DIC algorithm is proposed in this paper, in which the slope and shape of test surface can be accomplished only by shifting speckle pattern displayed on a screen.

Recently, some methods have been published which are related to the three-dimensional shape specification of the object surface by translating speckle pattern and using 2D DIC algorithm. In the work of Y. H. Huang et al. [14], a pinhole camera is used to record the different proportion of the speckles with different distances, and the surface shape of a bulb is successfully measured at an accuracy of sub-millimeter level by translating the test element with a speckle pattern on its surface. On the basis of Huang’s method, Xu et al. [15] introduce a planar mirror between the measured element and the camera to approximately execute the in-plane translation of the test element with speckle by rotating the plane mirror, and the surface shapes of both flat and wedge are separately measured. However, these work mainly focus on the diffuse reflection surface [1618], and the randomly distributed speckle pattern is sprayed on the test surface, which may damage the measured object, especially for a specular surface. The translation of the speckle pattern is achieved by mechanical devices in their work, of which it is difficult to ensure the accuracy of translation, while in our method the speckle pattern on the LCD screen can be shifted in the unit of screen pixel, which significantly improves the translation accuracy.

In the method proposed in this paper, the speckle pattern is displayed on the LCD screen, and the speckle pattern reflected off the test element is captured by a pinhole camera. When the speckle pattern is shifted on the LCD screen, the surface shape information is encoded in the displacement distribution between the two captured speckle images before and after the speckle pattern shifted on the LCD screen. The displacement distribution of the speckle image is calculated by the DIC algorithm, from which the slope and figure of the test surface is obtained. Only needing to shift the speckle pattern two times, the measurement has a high efficiency. No reference element is needed, and the measurement accuracy can achieve close to 1um RMS. The principle and algorithm of the method are introduced. The feasibility of the method is verified by numerical simulation and the source of error is analyzed. In the experiment, an acrylic plastic plate and a wafer are measured successfully.

2. Principles

2.1 The principle of 2D DIC

The basic principle of 2D DIC is to track the same point between two digital images, and then obtain the displacement of each point in the image. In our case, the images of speckle pattern [1921] displayed on the LCD screen before and after shifting are recorded in turn by a camera, as depicted in Fig. 1. The two images before and after shifting are referred as reference and shifted image, respectively. In order to calculate the displacement of the point $R({{x_0},{y_0}} )$ in the reference image, $M \times N$ pixels around point R are selected to act as a reference subset of rectangle (region in red rectangle in Fig. 1(a)), and the $P({{x_i},{y_j}} )$ is an arbitrary point in the reference subset. In the shifted image Fig. 1(b), the position of the subset is moved and the subset shape is deformed due to the distortion of specular surface. For a target subset centered at point $R^{\prime}({{x_0}^{\prime},{y_0}^{\prime}} )$, usually the changes in the subset are described with a first-order shape function:

$$\begin{array}{l} {x_i}^{\prime} = {x_i} + du + {u_x}\Delta x + {u_y}\Delta y\\ {y_j}^{\prime} = {y_j} + dv + {v_x}\Delta x + {v_y}\Delta y, \end{array}$$
where $\varDelta x$ and $\varDelta y$ are the coordinate differences between the arbitrary point $P({{x_i},{y_j}} )$ and the center point $R({{x_0},{y_0}} )$ in the reference subset, $du$ and $dv$ are the displacements of $R({{x_0},{y_0}} )$ in the x direction and the y direction, respectively, and ${u_x}$, ${u_y}$, ${v_x}$, ${v_y}$ are the first order differential of the displacement. In order to find the target subset corresponding to the reference subset in the shifted image, the correlation criterion is used to evaluate the similarity between the reference subset and the target subset, the commonly used correlation criteria are ZNCC and ZNSSD, which can be converted to each other [22], and ZNCC is described as:
$${C_{ZNCC}}\textrm{=}\frac{{\sum\limits_{i = 1}^M {\sum\limits_{j = 1}^N {[{f({{x_i},{y_j}} )- \bar{f}} ]\times [{g({{x_i}^{\prime},{y_j}^{\prime}} )- \bar{g}} ]} } }}{{\sqrt {\sum\limits_{i = 1}^M {\sum\limits_{j = 1}^N {{{[{f({{x_i},{y_j}} )- \bar{f}} ]}^2}} } } \times \sqrt {\sum\limits_{i = 1}^M {\sum\limits_{j = 1}^N {{{[{g({{x_i}^{\prime},{y_j}^{\prime}} )- \bar{g}} ]}^2}} } } }},$$
where $f({{x_i},{y_j}} )$ and $g({{x_i}^{\prime},{y_j}^{\prime}} )$ represent the gray values within the reference subset and the shifted subset separately, and $\bar{f}$ and $\bar{g}$ are the averages of $f({{x_i},{y_j}} )$ and $g({{x_i}^{\prime},{y_j}^{\prime}} )$, respectively. Using the correlation criterion to find the target subset with the highest similarity to the reference subset, the point $R^{\prime}({{x_0}^{\prime},{y_0}^{\prime}} )$ corresponding to the point $R({{x_0},{y_0}} )$ is determined, and the displacements $du$ and $dv$ of the point $R({{x_0},{y_0}} )$ can be obtained, simultaneously. The mean bias error of sub-pixel displacement calculation of DIC algorithm is approximately 0.01 pixel [23,24].

 figure: Fig. 1.

Fig. 1. Schematic of a subset before and after shifting: (a) reference image, (b) shifted image.

Download Full Size | PDF

2.2 The principle of speckle pattern shifting deflectometry

The schematic of speckle pattern shifting deflectometry (SPSD) test system proposed in this paper is shown in Fig. 2. This test system consists of four parts: an LCD screen with a speckle pattern as structured light source, a CCD camera with an external pinhole (working as a projection center of pinhole camera), a flat as a test element, and a computer for controlling and data processing. When the speckle pattern is shifted along the x direction in the unit of screen pixel on the LCD screen, two speckle patterns reflected off the test surface before and after shifting are captured by the pinhole camera, respectively. Here, the center of a speckle subset is displayed at the position of the sampling point ${S_0}$ on the LCD screen, by which the ray emitted is reflected at the point ${M_0}$ on the test surface and reaches the pinhole camera, and the light will illuminate the point ${C_0}$ on the camera sensor. If the world coordinates of the sampling point ${S_0}$ is obtained by a geometric measurement tool, such as point source microscope (PSM) with ruler, and the pixel coordinates of point ${C_0}$ on the captured image (In the following, point ${C_0}$ is used as the starting point in SPSD.) is given, the world coordinate of the mirror pixel ${M_0}$ on the test surface which is related to the CCD pixel ${C_0}$ can be obtained after the relationship between the CCD pixel ${C_0}$ and the mirror pixel ${M_0}$ is determined by camera calibration [3,25], thus, the slope at point ${M_0}$ can be calculated by substituting these coordinates into Eq. (3) [3,26] as follows,

$$\begin{array}{l} \tan {\alpha _x}({x_m},{y_m}) = \frac{{\frac{{{x_s} - {x_m}}}{{{d_{m2s}}}} + \frac{{{x_c} - {x_m}}}{{{d_{m2c}}}}}}{{\frac{{{z_{m2s}} - W({x_m},{y_m})}}{{{d_{m2s}}}} + \frac{{{z_{m2c}} - W({x_m},{y_m})}}{{{d_{m2c}}}}}},\\ \tan {\alpha _y}({x_m},{y_m}) = \frac{{\frac{{{y_s} - {y_m}}}{{{d_{m2s}}}} + \frac{{{y_c} - {y_m}}}{{{d_{m2c}}}}}}{{\frac{{{z_{m2s}} - W({x_m},{y_m})}}{{{d_{m2s}}}} + \frac{{{z_{m2c}} - W({x_m},{y_m})}}{{{d_{m2c}}}}}}, \end{array}$$
where $\tan {\alpha _x}$ and $\textrm{tan}{\alpha _y}$ are defined as the slope of the mirror pixel in the x and y directions, respectively, ${x_c}$, ${y_c}$ and ${z_c}$ are the coordinates of the pinhole camera, ${z_{m2s}}$ and ${z_{m2c}}$ are the distance from the test surface to the LCD screen and pinhole camera in the z direction. These coordinates can be obtained by geometric measurement. ${x_m}$ and ${y_m}$ are the coordinates of the mirror pixel of the test surface, ${x_s}$ and ${y_s}$ are the coordinates of the sampling point on the LCD screen, ${d_{m2s}}$ and ${d_{m2c}}$ are the distances from the mirror pixel of the test surface to the corresponding sampling point and the mirror pixel to the pinhole camera. For near-flat surfaces, the height $W({{x_m},{y_m}} )$ is small and negligible compared to ${d_{m2s}}$ and ${d_{m2c}}$, so the height $W({{x_m},{y_m}} )$ is assumed to be zero in the calculation process.

 figure: Fig. 2.

Fig. 2. Schematic of the SPSD test system.

Download Full Size | PDF

We assume that the sampling points ${S_0}$, ${S_1}$ and ${S_2}$ on the LCD screen are three of a series of equally spaced points distributed along the x direction, and the distance between these points is equal to the shifting distance of the speckle pattern as shown in the bottom of Fig. 2. With the world coordinates of ${S_0}$ known, the world coordinates of ${S_1}$, ${S_2}$, etc. can be calculated. After the speckle pattern is shifted, the center of the speckle subset in red is shifted from the position of ${S_0}$ to that of ${S_1}$, correspondingly the ray emitted by it will be reflected at another point ${M_1}$ on the test surface and illuminate the point ${C_1}$ on the camera sensor. The displacement from the reference subset at ${C_0}$ to the target or test subset at ${C_1}$ can be calculated from the two captured images before and after shifting the speckle pattern by the DIC algorithm, that is, the coordinate differences between the ${C_0}$ and ${C_1}$ in the unit of camera pixel will be obtained. Since the pixel coordinates of the point ${C_0}$ are known, the pixel coordinates of the point ${C_1}$ can be obtained, and then the world coordinates of the point ${M_1}$ on the test surface corresponding to the point ${C_1}$ are determined from the calibrated correspondence between the CCD pixel ${C_1}$ and the mirror pixel ${M_1}$. The slope at point ${M_1}$ can be obtained from Eq. (3). Similarly, the center of the speckle subset in green at the position of ${S_1}$ is shifted to the position of ${S_2}$, and the ray emitted by it will be reflected at another point ${M_2}$ on the test surface and illuminate the point ${C_2}$ on the camera sensor. The displacement from ${C_1}$ to ${C_2}$ can be also calculated by DIC algorithm. Since the pixel coordinates of point ${C_1}$ has been worked out, the pixel coordinates of point ${C_2}$ can be obtained as well. So, the world coordinates of the point ${M_2}$ are obtained from the correspondence between the CCD pixel ${C_2}$ and the mirror pixel ${M_2}$, and then the slope at the point ${M_2}$ is obtained by the Eq. (3).

As shown in Fig. 3, starting from the pixel coordinate of CCD pixel ${C_0}$ corresponding to the mirror pixel ${M_0}$ (in red) on the test surface, after the world coordinates of a row of mirror pixels along the x direction, such as ${M_1}$, ${M_2}$, ${M_3}$ and ${M_4}$, are determined as mentioned method above, the speckle pattern is then shifted in the y direction. Then the CCD pixels corresponding to the row of mirror pixels, which have been determined in the x direction shifting process, are treated as new starting points to obtain the world coordinates of the other mirror pixels in the y direction until this process cover over the entire region of interest (ROI). Vice versa, the speckle pattern can also be shifted in y direction first to acquire the positions of a column of mirror pixels and then shifted in x direction.

 figure: Fig. 3.

Fig. 3. Schematic of the acquisition path of the mirror pixel M on the test surface.

Download Full Size | PDF

As mention above, the surface shape of the test element will lead to an unequally spaced mirror pixel distribution or unpredicted speckle distortion if the test element weren’t an idea flat, therefore, the SPSD by shifting speckle pattern is proposed to find the pixel coordinate differences between the neighboring mirror pixels using the DIC algorithm, thereby the world coordinates of the mirror pixel are deduced, and the slope data are eventually calculated. In PMD, the fringe projection and phase shifting algorithm are used to find the coordinates of the point $({{x_s},{y_s}} )$ corresponding to the mirror pixel $({{x_m},{y_m}} )$, that is, the normal of the known mirror pixel is calculated. On the contrary, our method focus on how to find the position of mirror pixel by tracking the same point of speckle between the two images recorded before and after shifting, essentially, because the correspondence between the CCD pixel, the sampling point on the LCD screen and the mirror pixel is unique when the test system geometry is determined. As shown in Fig. 2, from the perspective of the inverse ray of the PMD, the point ${C_1}$ on the camera sensor sees the speckle at ${S_1}$ through the mirror pixel ${M_1}$. But for SPSD, only if the speckle at the position of ${S_0}$ is just shifted to the position of ${S_1}$, the mirror image of it reflected by ${M_1}$ can be seen through the line of sight of ${C_1}$, and the mirror pixel is moved from ${M_1}$ to . After the speckle displacement is calculated by DIC, the amount of change in the position of the mirror pixel can be obtained indirectly. Theoretically, this method can be used for a shift magnitude in random. Based on this property, the SPSD intends to find the coordinates of the mirror pixel $({{x_m},{y_m}} )$ corresponding to the sampling point $({{x_s},{y_s}} )$ using the speckle pattern shifting and DIC algorithm. This method is somehow similar to the deflectometry using a point source array [27]. In contrast, SPSD has a higher resolution, which is determined by the high precision characteristics of the DIC algorithm. DIC algorithm can calculate the displacement of speckle at approximately 0.01 pixel accuracy. Even if the displacement of the speckle is less than one pixel, DIC can still calculate the pixel coordinate difference between ${C_0}$ and ${C_1}$. Theoretically, the resolution of SPSD has an advantage over the physical resolution of camera.

3. Speckle pattern shifting deflectometry algorithm

According to the principle of SPSD, a starting point is required before the measurement. The position of the starting point can be obtained using a fiducial point, that is, a bright point is displayed on the LCD screen, then its world coordinates $({{x_{s0}},{y_{s0}}} )$ with the PSM is measured, and the pixel coordinates of the bright spot image on the camera sensor is recorded. Then, the speckle pattern is shifted in the x direction and the y direction on the LCD screen, and the shifting distance is defined as $nl$ ($n$ is the number of screen pixel, and l is the size of the screen pixel). At the same time, the speckle images reflected off the test surface before and after shifting are captured by the pinhole camera. The algorithm of SPSD is introduced in detail:

  • (1) Set a ROI according to the position and size of the test surface on the image captured by the camera;
  • (2) obtain the pixel coordinates $({u,v} )$ of the starting point. The DIC algorithm is used to calculate the displacement $({d{u_x},d{v_x}} )$ and $({d{u_y},d{v_y}} )$ of each point in ROI after speckle pattern shifting along the x and y direction, respectively. Meanwhile, according to the size of ROI and the shifting distance $nl$, the world coordinates of the equally spaced sampling points on the LCD screen are expressed as follows:
    $$\begin{array}{l} {x_{s(i,j)}} = {x_{s0}} + nli\\ {y_{s(i,j)}} = {y_{s0}} + nlj, \end{array}$$
    where i, j are the sequence of samples along the x and y direction;
  • (3) obtain the displacement $({d{u_x},d{v_x}} )$ at the pixel coordinates $({u,v} )$, and then obtain the pixel coordinates of the next CCD pixel as $({u + d{u_x},v + d{v_x}} )$;
  • (4) determine if the pixel coordinates $({u + d{u_x},v + d{v_x}} )$ exceed the ROI. If not exceed, $u = u + d{u_x}$, $v = v + d{v_x}$ is executed and then repeat step 3); otherwise, the pixel coordinates $({u,v} )$ of CCD pixel on the calculated one row path are taken as the new starting point, and go to the next step;
  • (5) obtain the displacement $({d{u_y},d{v_y}} )$ at the pixel coordinates $({u,v} )$, and then obtain the pixel coordinates of the next CCD pixel as $({u + d{u_y},v + d{v_y}} )$;
  • (6) determine if the pixel coordinates $({u + d{u_y},v + d{v_y}} )$ exceed the ROI. If the ROI is not exceeded, $u = u + d{u_y}$, $v = v + d{v_y}$ is executed and then repeat step 5); otherwise, go to the next step;
  • (7) using the bicubic interpolation algorithm (because the calculated pixel coordinates of the CCD pixel are not all at the integer pixel position), the world coordinates of the mirror pixel $({{x_m},{y_m}} )$ are obtained from the camera calibration data, and then the slope of the test surface is calculated by Eq. (3).
The whole procedure of SPSD algorithm is illustrated in Fig. 4.

 figure: Fig. 4.

Fig. 4. Flow chart of speckle pattern shifting deflectometry (SPSD) algorithm.

Download Full Size | PDF

4. Numerical simulation

In order to verify the feasibility of the method, a simulation is implemented based on the Korsch ray tracing equations [28]. As shown in Fig. 5, the LCD screen and the camera pinhole are both on the same $xoy$ plane, ${z_{m2s}}$ and ${z_{m2c}}$ being $1600mm$. The diameter of the test element is about $138.8mm$. The pixel size of the LCD screen is $0.2705mm$. The camera pixel size is $3.75 \times {10^{ - 3}}mm$, the angle between the camera optical axis and the z direction is ${5^o}$. With the known surface shape of the test element, the Korsch ray tracing equations can be used to obtain the sampling point coordinates $({{x_s},{y_s}} )$ and the mirror pixel coordinates $({{x_m},{y_m}} )$ of the test surface corresponding to each CCD pixel. The captured speckle images are obtained through a grayscale interpolation procedure of the speckle pattern displayed on the screen based on the sampling point coordinates $({{x_s},{y_s}} )$ obtained by ray tracing. The speckle pattern on the screen is shifted with the same distance in the x and y direction separately, and three speckle images before and after shifting are generated. This process is equivalent to the imaging process of the camera. The one of two pre-known surfaces is an ideal flat and the other is a quadric surface with a radius of curvature $90000mm$ which is a near-flat element, and the starting point is at the center point of the test surface. The slopes of the test surface are calculated directly with the ray tracing result, as the actual result.

 figure: Fig. 5.

Fig. 5. Schematic of the test system.

Download Full Size | PDF

The actual slope maps of the ideal flat acting as the pre-known surface are shown in Figs. 6(a) and 6(b). The shifting distance of the speckle pattern is $5l$, the slope maps calculated by SPSD are shown in Figs. 6(d) and 6(e). The slope data is integrated using the Zernike modal method [29,30] to reconstruct the surface shape. The surface shape reconstructed from the actual slope and the slope by SPSD are shown in Figs. 6(c) and 6(f), respectively. The RMS differences between the actual and slopes by SPSD in the x and y direction are $5.2 \times {10^{ - 5}}rad$ and $3.1 \times {10^{ - 5}}rad$, respectively. And the RMS error of the reconstructed surface shape is about $0.5um$, which is close to 1 wavelength ($0.633um$). As for the quadric surface, with the shifting distance also being $5l$, the actual and slope maps by SPSD are shown in Figs. 7(a), 7(b) and 7(d), 7(e), respectively, from which the actual and SPSD surface results are reconstructed as shown in Figs. 7(c) and 7(f), respectively. The RMS differences of the slopes in the x and y direction are $2.1 \times {10^{ - 5}}rad$ and $1.1 \times {10^{ - 5}}rad$, respectively. The RMS error of the reconstructed surface shape between the two methods is $0.3um$. The actual and the SPSD method’s slopes are very close both in the distribution and RMS value. And the reconstructed surface results show good consistency, both in height distribution and RMS value. Based on the analysis of the slope calculation results under two pre-known surface shape, our method can accurately measure the slope of the test surface with an accuracy of $5.2 \times {10^{ - 5}}rad$ (for the flat).

 figure: Fig. 6.

Fig. 6. The ideal flat as the pre-known surface: (a) and (b) are the actual slope in the x and y direction calculated directly by Eq. (3), respectively, and (c) is the reconstructed surface shape from Figs. 6(a) and 6(b). (d) and (e) are the slope in the x and y direction calculated by SPSD, respectively, and (f) is the reconstructed surface shape from Figs. 6(d) and 6(e).

Download Full Size | PDF

 figure: Fig. 7.

Fig. 7. The quadric surface as the pre-known surface: (a) and (b) are the actual slope in the x and y direction calculated directly by Eq. (3), respectively, and (c) is the reconstructed surface shape from Figs. 7(a) and 7(b). (d) and (e) are the slope in the x and y direction calculated by SPSD, respectively, and (f) is the reconstructed surface shape from Figs. 7(d) and 7(e).

Download Full Size | PDF

For the perfect flat, both the slope and height should be zero, however, the calculated slope value varies within a certain range, as indicated in Figs. 6(d) and 6(e), and the maximum reaches the order of magnitude of ${10^{ - 4}}rad$. To recognize the source of error, further analysis is carried out. According to the principle of SPSD, the position of each mirror pixel is determined by the displacement calculated by the DIC algorithm. For an ideal flat, ignoring the keystone distortion, the displacement of each point should be equal. Figure 8 shows the displacement distribution calculated by the DIC algorithm with a $5l$ shifting distance of the speckle pattern in the x direction. However, instead of the displacement amount being a constant, and there is a fluctuation of about 0.015 pixel around the mean value. Considering that the displacement calculation of DIC algorithm is approximately 0.01 pixel, along with that the interpolation algorithm used to generate the captured speckle images also induces errors, the error of the displacement amount is unavoidable.

 figure: Fig. 8.

Fig. 8. The displacement calculated by the DIC algorithm: (a) and (b) are displacement in the x and y direction, respectively.

Download Full Size | PDF

In the simulation, the size of the mirror pixel corresponding to a single camera pixel is around $0.375mm$, assuming that the error of the DIC is 0.015 pixel, the error of the mirror pixel world coordinate is $5.625 \times {10^{ - 3}}mm$. According to Eq. (3), the error of the slope is $3.51 \times {10^{ - 6}}rad$. This error is accumulated along the acquisition path of the mirror pixel, as illustrated in Fig. 3, it will induce slope errors with the order of magnitude of ${10^{ - 5}}rad$. As the error introduced by DIC is random, the slope error may decrease after being accumulated multiple times, that is, the accumulation of slope error is not a simple linear increase. It dominantly causes the distribution of the slope in Figs. 6(d) and 6(e) and is the main source that affects the accuracy of this method.

On the other hand, the shifting distance of the speckle pattern also affect the accuracy of reconstructed surface. Figure 9 shows the error of reconstructed surface with different shifting distances of speckle pattern. The shifting distances are $3l$, $4l$, $5l$, $6l$ and $7l$. For an ideal flat, as the increase of shifting distance, the reconstruction error of the surface shape in RMS decrease from 2.4 $um$ to around $0.8um$. For a quadric surface, the RMS error is close to $2um$ at $3l$ due to the error accumulation along the acquisition path where there are 165 sampling points from the starting point to the edge of the ROI. As the shifting distance exceeds $4l$, the RMS error is around $0.8um$. However, as the shifting distance increase to $6l$ and $7l$, the resolution reduces to 82 and 69 sampling points from the starting point to the edge of the ROI, respectively, and the average displacement of the speckle image exceeds 2 pixels, resulting in the loss of some detail information of the test surface. In contrast, when the shifting distance is $4l$ or $5l$, 122 and 97 sampling points in the ROI, respectively, and the average displacement of the speckle image is less than 2 pixels. Weighing the pros and cons, to achieve a high accuracy with a relative high resolution, the shifting distance of $4l$ or $5l$ in this simulation test system is recommended.

 figure: Fig. 9.

Fig. 9. The error of reconstructed surface with different shifting distances of speckle pattern.

Download Full Size | PDF

Without the loss of generality, the relationship between the shifting distance of the speckle pattern on the screen and the average displacement of speckle image is derived as follows:

$${\textrm{z}_{m2c}} \approx \frac{{d\cdot f}}{{K\cdot \overline {n^{\prime}} \cdot {l_c}}},$$
$$n \approx ROUND[\frac{{\overline {n^{\prime}} \cdot {l_c}\cdot \textrm{(}{\textrm{z}_{\textrm{m2c}}}\textrm{+}{\textrm{z}_{\textrm{m2s}}})}}{{l\cdot f}}],$$
where d is the radius of the test surface; ${l_c}$ is the size of the pinhole camera pixel; K is the number of the sampling points from the starting point to the edge of the ROI; f is the focal length; $\overline {n^{\prime}} $ is the average displacement of speckle image in the unit of camera pixel; the ROUND function rounds the number to the nearest integer. For a SPSD test system, in order to guarantee the detection resolution meanwhile minimize the accumulation error, K and $\overline {n^{\prime}} $ do not exceed 122 and 2, respectively. And ${z_{m2c}}$ is determined according to Eq. (5). Then, according to Eq. (6), the shifting distance of the speckle pattern on the LCD screen is properly chosen.

5. Experiments

In order to verify the method proposed in this paper in practical application, an acrylic plastic plate is measured first. The tested region of it is about $94.3mm$ in diameter, and the measurement result of it is compared with PMD’s result. The setup is shown in Fig. 10. A 21.3 inch LCD screen with a resolution of $1600 \times 1200$ pixels is used to display the speckle pattern. A pinhole camera with a focal length of 16mm and a resolution of 1296×966 pixels is used to record the speckle images. The test elements are placed parallel to the LCD screen. The z direction distances from the test surface to the LCD screen and the pinhole camera are $1499.8mm$ and $1479.5mm$, respectively. The internal and external parameter matrices of the pinhole camera are obtained by Zhang’s calibration method, and the correspondence between the CCD pixel and the mirror pixel on the test surface is established. The starting point is chosen as close as possible to the center of ROI to reduce the influence of error accumulation.

 figure: Fig. 10.

Fig. 10. Experiment setup of SPSD test system.

Download Full Size | PDF

The speckle pattern is shifted in turn by 5 screen pixels($5l$) in the x and y directions on the LCD screen, and the displacement distribution of the speckle image captured by the pinhole camera before and after the shifting is calculated by the DIC algorithm, as shown in Fig. 11. The slope maps calculated by the SPSD are shown in Figs. 12(a) and 12(b). As a comparison the slope maps are calculated using PMD, as shown in Figs. 12(d) and 12(e). SPSD and PMD slope datum show a good agreement with each other. The surface is reconstructed with the Zernike modal method. The reconstructed surface shape of SPSD and PMD is illustrated in Figs. 12(c) and 12(f), respectively. The RMS difference between the two surface results is $0.6um$. The comparison of Zernike polynomial coefficients of the two methods from the 4th to the 37th terms is demonstrated in Fig. 13, which indicates a good agreement.

 figure: Fig. 11.

Fig. 11. The displacement calculated by the DIC algorithm: (a) and (b) are the displacements in the x and y direction calculated by the DIC algorithm when the speckle pattern is shifted in the x direction, respectively. (c) and (d) are the displacements in the x and y direction calculated by the DIC algorithm when the speckle pattern is shifted in the y direction, respectively.

Download Full Size | PDF

 figure: Fig. 12.

Fig. 12. The measurement results of the acrylic plastic plate: (a) and (b) are the slope in the x and y direction calculated by SPSD, respectively, and (c) is the reconstructed surface shape using SPSD. (d) and (e) are the slope in the x and y direction calculated by PMD, respectively, and (f) is the reconstructed surface shape using PMD (Piston, tip and tilt terms of the surface shape are removed.).

Download Full Size | PDF

 figure: Fig. 13.

Fig. 13. The comparison of 4-37 Zernike polynomials coefficients.

Download Full Size | PDF

A wafer deformed by external force is also measured. The measurement range is $96.2mm$ in diameter. With the same experimental setup, the speckle pattern is shifted by 4 pixels in the x and y direction on the LCD screen. The slope in x and y direction and surface shape of the wafer surface obtained by our SPSD setup are shown in Figs. 14(a) and (c), respectively. the corresponding results of PMD are shown in Figs. 14(d) and (e) the RMS difference between the reconstructed figures is $0.8um$. In Fig. 15, it shows the comparison of Zernike polynomial coefficients of the two methods from the 4th to 37th terms for the wafer. Likewise, the Zernike coefficients of SPSD and PMD are very close to each other. This further verifies the feasibility of our method and that the measurement accuracy of SPSD can reach about $1um$ RMS.

 figure: Fig. 14.

Fig. 14. The measurement results of the wafer: (a) and (b) are the slope in the x and y direction calculated by SPSD, respectively, and (c) is the reconstructed surface shape using SPSD. (d) and (e) are the slope in the x and y direction calculated by PMD, respectively, and (f) is the reconstructed surface shape using PMD (Piston, tip and tilt terms of the surface shape are removed.).

Download Full Size | PDF

 figure: Fig. 15.

Fig. 15. The comparison of 4-37 Zernike polynomials coefficients.

Download Full Size | PDF

6. Conclusions

In this paper, the SPSD test system based on DIC is proposed. The principle of SPSD is described in detail. Compared with SPD, this method doesn’t need reference element. Compared to PMD, only three images are needed to complete a measurement. It has the advantages of fast measurement, low cost, and independence on the quality of reference element. The numerical simulation is carried out to verify the correctness of the method, and the source of the error is analyzed. What the shifting distance of the speckle pattern on the LCD screen should be chosen to minimize the error while ensuring the measurement resolution is suggested. The surface shapes of the acrylic plastic plate and the wafer are measured in the experiment. Compared with the measurement results of PMD, the effectiveness of the method is verified, and the measurement accuracy can reach about $1um$ RMS. The test element in our method is near-flat, and it is not applicable if the mirror curvature is too large. We will keep investigating surface shape measurement of large curvature element in next work.

Funding

National Natural Science Foundation of China (61875142); Equipment Research Program in Advance of China (JZX2017-1570/Y464); National Key R&D Program of China (2017YFB1002900).

References

1. M. C. Knauer, J. Kaminski, and G. Hausler, “Phase measuring Deflectometry: a new approach to measure specular free-form surfaces,” Proc. SPIE 5457, 366–376 (2004). [CrossRef]  

2. Y. Tang, X. Su, Y. Liu, and H. Jing, “3d shape measurement of the aspheric mirror by advanced phase measuring deflectometry,” Opt. Express 16(19), 15090–15096 (2008). [CrossRef]  

3. E. Kewei, D. Li, L. Yang, L. Yang, G. Guo, M. Li, X. Wang, T. Zhang, and Z. Xiong, “Novel method for high accuracy figure measurement of optical flat,” Opt. Laser Eng. 88, 162–166 (2017). [CrossRef]  

4. X. Chen, C. Lu, M. Ma, X. Mao, and T. Mei, “Color-coding and phase-shift method for absolute phase measurement,” Opt. Commun. 298-299, 54–58 (2013). [CrossRef]  

5. C. Jin, D. Li, E. Kewei, M. Li, P. Chen, R. Wang, and Z. Xiong, “Phase extraction based on iterative algorithm using five-frame crossed fringes in phase measuring deflectometry,” Opt. Laser Eng. 105, 93–100 (2018). [CrossRef]  

6. L. Huang, C. Ng, and A. Asundi, “Dynamic three-dimensional sensing for specular surface with monoscopic fringe reflectometry,” Opt. Express 19(13), 12809–12814 (2011). [CrossRef]  

7. P. Xie, M. Tang, and X. Wei, “Three-dimensional shape measurement of specular surfaces by orthogonal composite fringe reflection,” Proc. SPIE 8200, 820014 (2011). [CrossRef]  

8. H. Tang, D. Li, L. Li, P. Chen, R. Wang, and Q. Wang, “Planar Object Surface Shape Speckle Pattern Deflectometry Based on Digital Image Correlation,” Acta Opt. Sin. 39(2), 0212006 (2019). [CrossRef]  

9. W. H. Peters and W. F. Ranson, “Digital imaging technique in experimental stress analysis,” Opt. Eng. 21(3), 427–431 (1982). [CrossRef]  

10. M. A. Sutton, J. J. Orteu, and H. Schreie, Image Correlation for Shape, Motion and Deformation Measurements Basic Concepts, Theory and Applications (Springer, 2009).

11. B. Pan, “Digital image correlation for surface deformation measurement: historical developments, recent advances and future goals,” Meas. Sci. Technol. 29(8), 082001 (2018). [CrossRef]  

12. C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: a review,” Opt. Laser Eng. 109, 23–59 (2018). [CrossRef]  

13. Z. Jiang, K. Qian, H. Miao, J. Yang, and L. Tang, “Path-independent digital image correlation with high accuracy, speed and robustness,” Opt. Laser Eng. 65, 93–102 (2015). [CrossRef]  

14. Y. H. Huang, C. Quan, C. J. Tay, and L. J. Chen, “Shape measurement by the use of digital image correlation,” Opt. Eng. 44(8), 087011 (2005). [CrossRef]  

15. X. Xu, K. Wang, and G. Gu, “An improved method for shape measurement using two-dimensional digital image correlation,” Optik 124(20), 4097–4099 (2013). [CrossRef]  

16. F. Zhong, R. Kumar, and C. Quan, “A cost-effective single-shot structured light system for 3D shape measurement,” IEEE Sens. J. 19(17), 7335–7346 (2019). [CrossRef]  .

17. F. Zhong, R. Kumar, and C. Quan, “RGB laser speckles based 3D profilometry,” Appl. Phys. Lett. 114(20), 201104 (2019). [CrossRef]  

18. D. Khan, M. Shirazi, and M. Kim, “Single shot laser speckle based 3D acquisition system for medical applications,” Opt. Laser Eng. 105, 43–53 (2018). [CrossRef]  

19. B. Pan, H. M. Xie, B. Q. Xu, and F. Dai, “Performance of sub-pixel registration algorithms in digital image correlation,” Meas. Sci. Technol. 17(6), 1615–1621 (2006). [CrossRef]  

20. B. Pan, Z. Lu, and H. Xie, “Mean intensity gradient: An effective global parameter for quality assessment of the speckle patterns used in digital image correlation,” Opt. Laser Eng. 48(4), 469–477 (2010). [CrossRef]  

21. B. Pan, H. Xie, Z. Wang, K. Qian, and Z. Wang, “Study on subset size selection in digital image correlation for speckle patterns,” Opt. Express 16(10), 7037–7048 (2008). [CrossRef]  

22. B. Pan, H. Xie, and Z. Wang, “Equivalence of digital image correlation criteria for pattern matching,” Appl. Opt. 49(28), 5501–5509 (2010). [CrossRef]  

23. B. Pan and B. Wang, “Digital Image Correlation with Enhanced Accuracy and Efficiency: A Comparison of Two Subpixel Registration Algorithms,” Exp. Mech. 56(8), 1395–1409 (2016). [CrossRef]  

24. J. Blaber, B. Adair, and A. Antoniou, “Ncorr: Open-Source 2D Digital Image Correlation Matlab Software,” Exp. Mech. 55(6), 1105–1122 (2015). [CrossRef]  

25. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). [CrossRef]  

26. P. Su, R. Parks, L. Wang, R. Angel, and J. Burge, “Software configurable optical test system: a computerized reverse Hartmann test,” Appl. Opt. 49(23), 4404–4412 (2010). [CrossRef]  

27. K. E, A Study of High Accuracy Surface Figure Measurement Based on Phase Measuring Deflectometry (Sichuan University, 2017).

28. D. Korsch, Reflective Optics (Academic Press, 1991).

29. M. Li, D. Li, C. Zhang, E. Kewei, Q. Wang, and H. Chen, “Modal wavefront reconstruction from slope measurements for rectangular apertures,” J. Opt. Soc. Am. A 32(11), 1916–1921 (2015). [CrossRef]  

30. C. Zhao and J. H. Burge, “Orthonormal vector polynomials in a unit circle, Part I: basis set derived from gradients of Zernike polynomials,” Opt. Express 15(26), 18014–18024 (2007). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1.
Fig. 1. Schematic of a subset before and after shifting: (a) reference image, (b) shifted image.
Fig. 2.
Fig. 2. Schematic of the SPSD test system.
Fig. 3.
Fig. 3. Schematic of the acquisition path of the mirror pixel M on the test surface.
Fig. 4.
Fig. 4. Flow chart of speckle pattern shifting deflectometry (SPSD) algorithm.
Fig. 5.
Fig. 5. Schematic of the test system.
Fig. 6.
Fig. 6. The ideal flat as the pre-known surface: (a) and (b) are the actual slope in the x and y direction calculated directly by Eq. (3), respectively, and (c) is the reconstructed surface shape from Figs. 6(a) and 6(b). (d) and (e) are the slope in the x and y direction calculated by SPSD, respectively, and (f) is the reconstructed surface shape from Figs. 6(d) and 6(e).
Fig. 7.
Fig. 7. The quadric surface as the pre-known surface: (a) and (b) are the actual slope in the x and y direction calculated directly by Eq. (3), respectively, and (c) is the reconstructed surface shape from Figs. 7(a) and 7(b). (d) and (e) are the slope in the x and y direction calculated by SPSD, respectively, and (f) is the reconstructed surface shape from Figs. 7(d) and 7(e).
Fig. 8.
Fig. 8. The displacement calculated by the DIC algorithm: (a) and (b) are displacement in the x and y direction, respectively.
Fig. 9.
Fig. 9. The error of reconstructed surface with different shifting distances of speckle pattern.
Fig. 10.
Fig. 10. Experiment setup of SPSD test system.
Fig. 11.
Fig. 11. The displacement calculated by the DIC algorithm: (a) and (b) are the displacements in the x and y direction calculated by the DIC algorithm when the speckle pattern is shifted in the x direction, respectively. (c) and (d) are the displacements in the x and y direction calculated by the DIC algorithm when the speckle pattern is shifted in the y direction, respectively.
Fig. 12.
Fig. 12. The measurement results of the acrylic plastic plate: (a) and (b) are the slope in the x and y direction calculated by SPSD, respectively, and (c) is the reconstructed surface shape using SPSD. (d) and (e) are the slope in the x and y direction calculated by PMD, respectively, and (f) is the reconstructed surface shape using PMD (Piston, tip and tilt terms of the surface shape are removed.).
Fig. 13.
Fig. 13. The comparison of 4-37 Zernike polynomials coefficients.
Fig. 14.
Fig. 14. The measurement results of the wafer: (a) and (b) are the slope in the x and y direction calculated by SPSD, respectively, and (c) is the reconstructed surface shape using SPSD. (d) and (e) are the slope in the x and y direction calculated by PMD, respectively, and (f) is the reconstructed surface shape using PMD (Piston, tip and tilt terms of the surface shape are removed.).
Fig. 15.
Fig. 15. The comparison of 4-37 Zernike polynomials coefficients.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

x i = x i + d u + u x Δ x + u y Δ y y j = y j + d v + v x Δ x + v y Δ y ,
C Z N C C = i = 1 M j = 1 N [ f ( x i , y j ) f ¯ ] × [ g ( x i , y j ) g ¯ ] i = 1 M j = 1 N [ f ( x i , y j ) f ¯ ] 2 × i = 1 M j = 1 N [ g ( x i , y j ) g ¯ ] 2 ,
tan α x ( x m , y m ) = x s x m d m 2 s + x c x m d m 2 c z m 2 s W ( x m , y m ) d m 2 s + z m 2 c W ( x m , y m ) d m 2 c , tan α y ( x m , y m ) = y s y m d m 2 s + y c y m d m 2 c z m 2 s W ( x m , y m ) d m 2 s + z m 2 c W ( x m , y m ) d m 2 c ,
x s ( i , j ) = x s 0 + n l i y s ( i , j ) = y s 0 + n l j ,
z m 2 c d f K n ¯ l c ,
n R O U N D [ n ¯ l c ( z m2c + z m2s ) l f ] ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.