1. Introduction

Fringe projection profilometry [1–4], having advantages of high speed, noncontact, and full-field, is widely used in three-dimensional (3D) shape measurement. In its implementation, a projector casts sinusoidal fringe patterns onto a measured surface, and a camera at a different angle captures the deformed patterns caused by the depth variation of the surface. Measuring the fringe phases enables us to reconstruct the 3D shape of the measured surface. Therefore, the system calibration, which determines the mapping relationship between the phase map and 3D point coordinates, is crucial for guaranteeing the measurement accuracy. Conventionally, the reference-plane-based methods are used to determine this mapping relationship between fringe phases and object heights [5,6]. This phase-to-height conversion is usually implicitly represented with a polynomial [7–9] or with a function deduced from the system geometry [10–14]. In [14], for example, by projecting uneven fringes, a one-coefficient relationship is deduced for simplifying the data processing. Compared with these techniques, a more general approach is to explicitly calibrate all the parameters of the camera and the projector, including their intrinsic and extrinsic parameters, as well as their lens distortion coefficients.

As is well known, camera calibration techniques have been extensively developed in the field of photogrammetry or computer vision [15,16]. With them, a specially designed target, e.g., usually a checkerboard with known square size, is placed in the field of view of the camera, in order to help determine the correspondences between camera pixels and the object points. The camera parameters are estimated from these correspondences by using optimization algorithms. For enhancing the calibration accuracy, some efforts are made by use of bundle adjustment strategy [17,18].

Although a projector has seemingly the same model as that of a camera, projector calibration remains challenging because the projector, unlike a camera, cannot capture images, leading to an obstacle to knowing the correspondences between projector pixels and object points. For solving this problem, a well-calibrated camera is usually used as an aid [19–24]. Taking [22] for example, the projector to be calibrated casts a checkerboard pattern onto a calibration board, and the image is captured by a camera. By extracting the featured points from these images and using the camera parameters, the spatial coordinates of the object points are calculated, so that the correspondences between projector pixels and object points are determined in such an indirect way. In doing so, many factors may induce uncertainties in the calibration results. Especially, an optical lens generally has a point spread function (PSF) equivalent to the impulse response of a low-pass filter. Such low-pass properties of lenses make the captured images blurred to a certain extent [25]. For suppressing these uncertainties, [23] suggested a two-step method that estimates the coarse parameters by identifying the captured markers, and then achieve more accurate results via iterative adjustments.

Instead of projecting specially designed patterns onto the calibration board, using sinusoidal fringe patterns are more suitable for the projector calibration purpose [26–34]. With them the projector pixels are coded as fringe phases thus being less sensitive to the low-pass properties of the projector and camera lenses. In practice, however, the phases at edges or in dark areas cannot be measured accurately, causing a problem of how the phases at the featured points are measured reliably. For solving this problem, [30] and [31] use a red-blue checkerboard, instead of the popularly used black-white ones, as the calibration board thus avoiding measuring the phases in dark areas. In this case, the phase measuring at square corners are still affected by the low-pass properties of the devices [25]. Another option is to use a board having a circle array, with which the circle centers are considered to be the featured points [32,33]. Although the image of each circle, due to perspective projection, does not exactly have a shape of ellipse [35], its centroid is extracted and approximately used as a featured point. The results of this method strongly depend on the ellipse segmentation, and using sub-pixel edge detection is helpful for improving the accuracy [34]. Besides, the existing methods using sinusoidal fringes suffer from the phase measuring errors induced by such factors as noise, device nonlinearities, and illumination fluctuations.

From the proceeding discussion, we know that developing an accurate system calibration method for fringe projection profilometry remains challenging. This paper presents, to the best of our knowledge, a novel method for calibrating the measurement system having lens distortions. This method uses a checkerboard as the calibration board. By projecting sinusoidal fringes onto it, and fitting the fringe phases using a rational function which is demonstrated to be exact in representing the phase map of a plane board, the phases at corner points are accurately extracted and, further, the corresponding projector pixels are determined. An iterative strategy is used for restraining effects of lens distortions on the phase fitting results. As a result, the projector parameters, including the intrinsic and extrinsic parameters, and the radial and tangential distortion coefficients, are determined accurately. For compensating for the effects of the lens distortions on measurement, this paper also suggests a pre- and a post-compensation method. Both are experimentally demonstrated to be effective in improving the measurement accuracies.

2. Model of measurement system

2.1 Camera model

In fringe projection profilometry, the measurement system mainly consists of a camera and a projector. The standard camera model has been well established in the field of computer vision and can be found in the related literature [36–38]. In this section, we repeat deriving the camera model just for the convenience in using its concepts, notations, and formulas.

Calibrating a camera is to obtain its intrinsic and extrinsic parameters. The intrinsic parameters include the focal length, the principal point and the pixel skew factor; and the extrinsic parameters include a rotation matrix and a translation vector.

As schemed is Fig. 1, Q is an object point having coordinates (X_w, Y_w, Z_w) in the world coordinate system and coordinates (X_c, Y_c, Z_c) in the camera coordinate system. Q produces its image at the point q_c on the image plane of the camera. In the image coordinate system, q_c has pixel coordinates (u, v). The correspondence between the camera coordinates and the image coordinates is represented with

 

Fig. 1. The models of the projector-camera system.

Download Full Size | PDF

In measurement practice, distortions of the camera lens may decrease the measurement accuracy. In the presence of lens distortions, the image of a point deviates from its correct position. These distortions should be described in the camera coordinate system, but we have to use the coordinates scaled by 1/f_c instead, because the lens focal length f_c cannot be separated during calibration. In this case, the relation between the ideal coordinates of a camera pixel (x_c, y_c) and the distorted coordinates $({\hat{x}_\textrm{c}},{\hat{y}_\textrm{c}})$ is represented with

The parameters of a camera, including its intrinsic matrix A_c and extrinsic matrix [R_c, T_c], as well as its lens distortion coefficients, are determined by using the technique known as camera calibration. With it, a target having specially designed markers with known sizes is usually used. For example, the black-white checkerboard as shown in Fig. 1 is the most popularly used target for calibration, with its square corners being the featured points. By capturing the images of the target at various positions and angles and by extracting their featured points, the correspondences between the object points and their image positions on the imaging plane of the camera are easy to determine. Using these correspondences, the camera parameters are estimated according to the model in Eqs. (1) through (4). With this technique, although data processing involves complex nonlinear optimizations, such software is readily available, making it easy to use in many applications. Throughout this paper, we use the standard method proposed by Zhang [16] to calibrate the camera, with the camera parameters being estimated using iterative Levenberg-Marquardt algorithm.

2.2 Projector model

A projector has the same model as the camera, with inverse directions of light rays. As shown in Fig. 1, the point Q in the projector coordinate system has coordinates (X_p, Y_p, Z_p). It is illuminated by the light source at q_p on the image plane of the projector, with its pixel coordinates being (s, t). Similar to Eq. (1), their relation is

Despite having the same model as that of a camera, a projector is much more difficult to calibrate, because it has inverse directions of light rays and cannot capture images, leading to an obstacle to knowing the correspondences between projector pixels and object points. This paper mainly focuses on solving the issue how the correspondences between the projector pixels and the featured points on the calibration board are accurately established. When the correspondences are obtained, the projector parameters will be estimated in a similar fashion to that of camera calibration.

2.3 Projector-camera model

As shown in Fig. 1, the measurement system is based on the principle of triangulation, and works like a binocular stereovision system. For the convenience purpose, we define H_c=A_c[R_c, T_c], so that Eq. (3) is restated as

In fringe projection technique, the projector pixel (s, t) are coded using fringe phases. In the next section, we shall introduce phase-measuring method and then show how the phases can be converted into projector pixels. Note that, the equation system in Eq. (10) contains four equations involving three unknowns. In measurement, only one of the last two equations necessarily remains in the equation system, depending on the fringe orientations. If vertical fringes (perpendicular to s-axis) are used, the third equation remains. Whereas if horizontal fringes (perpendicular to t-axis) are used, we select the last equation instead. Generally, the fringes roughly perpendicular to the system baseline (which connects the projector and camera lens centers) are used in measurement, for achieving a high phase sensitivity.

3. Iterative calibration method

3.1 Phase measuring

Fringe projection technique measures the 3D shape of an object by projecting sinusoidal fringes onto it and measuring the phases. In this work, we calibrate the system also by measuring the phases of fringes projected on the calibration board. We use phase-shifting technique for recovering the fringe phases. Presume for the moment that vertical fringes are used, so that a sequence of phase-shifting fringe patterns is generated in computer. Among them, the mth (m = 0, 1, …, M-1) frame has a form of

3.2 Determining projector pixels corresponding to featured points

As mentioned previously, the difficulty of calibrating a projector lies in getting the correspondences between projector pixels and featured points on the calibration board, because the projector cannot capture images. We overcome this issue by measuring the fringe phases on the calibration board using the method introduced above.

Here, we select a black-white checkerboard as the calibration board, which is the most popularly used in engineering and thus being commercially available. The square corners on this checkerboard serve as the featured points. Referring to Fig. 1, when the image of the calibration board is captured by the camera, the corners are extracted using a corner detection operator, like Harris detector [41]. We denote the camera pixel coordinates of the extracted square corners as (U_n, V_n) with the subscript n being the index related to the nth corner point on the checkerboard. This subsection focuses on determining the corresponding projector pixels, (S_n, T_n), of these corner points.

In 3D measurement, only fringes of one direction are used. When calibrating a projector, we have to project both vertical and horizontal fringe patterns onto the calibration board, as shown in Figs. 2(a) and 2(e), respectively. On their right side, Figs. 2(b) and 2(f) are the recovered phase maps. From them, we see that the regions of black squares have been segmented out by thresholding fringe modulations, where the calculated phases are inaccurate because of the low reflectivity there. In fact, the measured phases at square corners also have prohibitively large errors caused by the low-pass properties of the camera.

 

Fig. 2. The columns, from left to right, show the captured fringe patterns on the calibration board, the measured phase map with the black areas having been segmented out, the least-squares fitting results using rational functions, and the residuals calculated by subtracting the fitting results from the measured phases. The top and bottom rows correspond to the results of using vertical and horizontal fringes, respectively.

Download Full Size | PDF

For estimating accurate phases at square corners, we exploit the fact that, in the absence of lens distortions, the fringe phases on a plane board theoretically have a distribution of rational function [42], viz.,

Figures 2(c) and 2(g) give the fitting results of Figs. 2(b) and 2(f), respectively. On their right side, Figs. 2(d) and 2(h) show the residuals calculated by subtracting the fitting results in Figs. 2(c) and 2(g) from the measured phases in Figs. 2(b) and 2(f), respectively. We observe that large phase errors appear at the corners and borders of each square, which are induced by the low-pass properties of lenses. As mentioned in the introduction section, an optical lens has the low-pass property because its point spread function (PSF) is generally equivalent to the impulse response of a low-pass filter. At the same time, the low-pass properties of lenses are also sourced from the defocuses of lenses. Reference [25] modeled the lenses using Gaussian low-pass filters and deeply analyzed the effects of their low-pass properties on the phase measuring results, revealing that the phase measuring results inevitably have larger errors at edges, just as we see in Figs. 2(d) and 2(h). This type of errors cannot be suppressed by improving the phase-shifting algorithm, e.g., by increasing the number of phase shifts. This fact means that the measured phases at the square corners cannot be used to determine their pixel coordinates. For this reason, we calculate the pixel coordinates of the corners from the fitting results.

By substituting the camera pixel coordinates of the square corners, i.e., (U_n, V_n), into Eq. (15) whose coefficients have been estimated through Eq. (16) and by combing Eq. (14), the corresponding projector pixel coordinates are calculated as

 

Fig. 3. Arrays of extracted square corners on the image planes of (a) camera and (b) projector

Download Full Size | PDF

This method, by fitting the phase map, estimates coordinates of the projector pixels corresponding to the featured points on the calibration board. The results are much more accurate than those obtained from the phase values directly measured, because they are immune to the low-pass properties of lenses. Simultaneously, the phase errors induced by such factors as noise, projector and camera nonlinearities, and illumination fluctuations have also been averaged out in fitting the rational function. Although we use a checkerboard in this work, this function fitting method is suitable for enhancing the accuracy in extracting the featured points when selecting other types of calibration boards (e.g., that having circle array). Note that, with this function fitting method, the lens distortions still induce errors in extracting the projector pixels of the corners, because they deform the phase maps.

3.3 Iterative estimation of projector parameters

In fringe projection profilometry, system calibration generally involves a complex procedure. For make it clear and easy to follow, we summarize its steps here.

The camera is used to capture a sequence of images of the checkerboard when it is placed at various positions and angles. In parallel to doing it, the projector is used to cast both horizontal and vertical fringe patterns onto the checkerboard, and the same camera captures the deformed patterns. Instead of capturing the image of the checkerboard, its photograph can also be obtained by averaging the captured fringe patterns. From the captured fringe patterns, we recover the fringe phases and exclude invalid areas by using the method in Section 3.1. As a result, for the checkerboard at each position, we have a photograph and a pair of phase maps.

Following Zhang’s technique [16], we calibrate the camera, extracting corner pixels from each photograph and then estimating the camera parameters from the correspondences between the square corners on the checkerboard and their camera pixels. Using the calibrated distortion coefficients of the camera lens, the geometric distortions on the photographs and on the phase maps of the checkerboard are easy to correct. Assuming (u, v) to be a pixel in the corrected pattern, substituting its scaled camera coordinates (x_c, y_c) into Eq. (4) results in $({\hat{x}_\textrm{c}},{\hat{y}_\textrm{c}})$ and, further, $(\hat{u},\hat{v})$, i.e., the pixel in the distorted pattern. By simply assigning the value (e.g., gray level or phase) at $(\hat{u},\hat{v})$ to (u, v), we have the image or the phase maps with the camera distortions having been corrected. By doing so, the camera lens distortions will not affect calibrating the projector parameters.

After correcting effects of camera lens distortions, we calculate, from each pair of phase maps $\varPhi$_s(u, v) and $\varPhi$_t(u, v), the projector pixels corresponding to camera pixels (u, v) through the formulas s=λ_s$\varPhi$_s(u, v)/2π and t=λ_t$\varPhi$_t(u, v)/2π. From each photograph of the checkerboard, we extract the corner pixels (U_n, V_n). These are basic data for calibrating the projector.

Using the method in Section 3.2, we fit the phases $\varPhi$_s(u, v) and $\varPhi$_t(u, v) using rational functions of (u, v), resulting in the coefficients {a₀, a₁, …, a₇}. By substituting (U_n, V_n) into Eq. (17) having coefficients {a₀, a₁, …, a₇}, we calculate (S_n, T_n), namely the projector pixel coordinates corresponding to the square corners on the checkerboard. Through numerical optimization with Zhang’s technique [16], we estimate the projector parameters from the correspondences between the square corners on the checkerboard and their projector pixels. These parameters include the intrinsic parameters {κ_s, κ_t, s₀, t₀} and the lens distortion coefficients {k_p1, k_p2, p_p1, p_p2}.

If the estimated {k_p1, k_p2, p_p1, p_p2} are not zero, the projector has lens distortions, in which case the measured phases of the calibration board are not exactly distributed as rational functions. This fact means that the projector lens distortions may induce errors in the calibration result. We use the following iterative strategy to solve this problem.

Step 1. Use the calibration results just obtained as the initial values, i.e., {κ_s, κ_t, s₀, t₀}⁽⁰⁾ and{k_p1, k_p2, p_p1, p_p2}⁽⁰⁾, with the superscripts representing the number of iterations.

Step 2. If {κ_s, κ_t, s₀, t₀}⁽ⁱ⁾ and {k_p1, k_p2, p_p1, p_p2}⁽ⁱ⁾ are calculated, correct the effects of the projector lens distortions on the phase maps. Substituting ${x_\textrm{p}} = {{[s - s_0^{(i)}]} \mathord{\left/ {\vphantom {{[s - s_0^{(i)}]} {k_s^{(i)}}}} \right.} {k_s^{(i)}}}$ and ${y_\textrm{p}} = {{[t - t_0^{(i)}]} \mathord{\left/ {\vphantom {{[t - t_0^{(i)}]} {k_t^{(i)}}}} \right.} {k_t^{(i)}}}$ into Eq. (7), we calculate ${\hat{x}_\textrm{p}}$ and ${\hat{y}_\textrm{p}}$ on the left side. Then by using ${\hat{s}^{(i + 1)}} = {\hat{x}_\textrm{p}}\kappa _s^{(i)} + s_0^{(i)}$ and ${\hat{t}^{(i + 1)}} = {\hat{y}_\textrm{p}}\kappa _t^{(i)} + t_0^{(i)}$, we have the updated projector pixel coordinates. Using them, we repeat calculating the phase maps through $\varPhi _s^{(i + 1)}(u,v) = {{2\pi {{\hat{s}}^{(i + 1)}}} \mathord{\left/ {\vphantom {{2\pi {{\hat{s}}^{(i + 1)}}} {{\lambda_s}}}} \right.} {{\lambda _s}}}$ and $\varPhi _t^{(i + 1)}(u,v) = {{2\pi {{\hat{t}}^{(i + 1)}}} \mathord{\left/ {\vphantom {{2\pi {{\hat{t}}^{(i + 1)}}} {{\lambda_t}}}} \right.} {{\lambda _t}}}$ where effects of the projector lens distortions are partially eliminated.

Step 3. Estimate more accurate projector parameters. Fitting $\varPhi _s^{(i + 1)}(u,v)$ and $\varPhi _t^{(i + 1)}(u,v)$ using rational functions, resulting in the coefficients {a₀, a₁, …, a₇}⁽ⁱ⁺¹⁾. By substituting (U_n, V_n) into Eq. (17) having these coefficients, we calculate more accurate (S_n, T_n)⁽ⁱ⁺¹⁾. Through numerical optimization, we estimate the projector parameters, including {κ_s, κ_t, s₀, t₀}⁽ⁱ⁺¹⁾ and {k_p1, k_p2, p_p1, p_p2}⁽ⁱ⁺¹⁾, by corresponding (S_n, T_n)⁽ⁱ⁺¹⁾ to the square corners on the checkerboard.

Step 4. Repeat implementing Step 2 and 3 until the algorithm converges, i.e., until the variations of the estimated parameters between two consecutive iterations are below a preset threshold. The threshold is determined empirically. Here, we set a threshold of 10⁻⁴ which guarantees the iterative procedure achieves a satisfied accuracy.

3.4 Results

We apply the proposed technique to a measurement system. In this system, the camera (AVT Stingray F-125B) has a lens KOWA LM12JC with its focal length being 12 mm, and its image size is 1024 × 768 pixels with the pixel size being 3.75 × 3.75 μm². The projector (Philips PPX 4010) has a resolution of 854 × 480 pixels. A black-and-white checkerboard having 9 × 12 squares is used as the calibration board, with each square having a size of 25 × 25 mm².

During the calibration, we use the camera to capture the fringe patterns of the checkerboard at fifteen different positions and angles. At each pose, two sequences of fringe patterns, having vertical and horizontal fringe directions, respectively, are projected onto the checkerboard. In each sequence, the fringes have three different frequencies with the full pattern being covered by 1, 8, and 64 fringe periods, respectively, for the phase-unwrapping purpose. For each fringe frequency, we use three-step phase shifting technique, and the photograph of the checkerboard without fringes is obtained by simply averaging the captured fringe patterns. As a result, we capture totally 270 patterns for one calibration. Harris corner detector is used to extract the featured points from the images. In phase measuring, we use the temporal technique to unwrap the phase maps. With it, the fringe pattern having a single fringe does not have phase ambiguities. Using its phase map as a reference allows us to unwrap a phase map of a pattern having several fringes, by use of the multiple relation between their frequencies. Furthermore, using the unwrapped phase map of a lower frequency pattern as a reference allows us to unwrap a phase map of a higher frequency [43].

Following Zhang’s calibration technique, the camera parameters are estimated using iterative Levenberg-Marquardt algorithm. As a result, the intrinsic parameters are {η_u, η_v, u₀, v₀}={3415.0950, 3415.6276, 523.4285, 393.0799}, the coefficients of lens distortions are{k_c1, k_c2, p_c1, p_c2}={−0.0844, −0.0350, 0.0019, −0.0005}, and the extrinsic matrix is

 

Fig. 4. Reprojection errors of (a) the camera and (b) the projector.

Download Full Size | PDF

4. Projector lens distortion correction

4.1 Pre-compensation technique

After calibrating the system, we can use it to measure an object through the following steps. First, generate sinusoidal phase-shifting fringe patterns using Eq. (11); second, project the generated fringe patterns onto the measured object surface, and capture the deformed patterns; third, calculate the wrapped phase map through Eq. (13), and unwrap them by using multi-frequency temporal phase unwrapping algorithm; fourth, calculate the project pixel coordinates through Eq. (14); and last, calculate 3D coordinates of the measured surface by using Eq. (10).

In the above procedure, using the calibrated coefficients of camera lens distortions, it is easy to eliminate the image distortions induced by them. The procedure has been described in the third paragraph of Section 3.3. After correcting the camera lens distortions, the measurement accuracy is mainly affected by the projector lens distortions. Some techniques allow us to circumvent this problem. For example, [45] recovers the depth map by use of the cross-ratio invariance, at an expense of much more time for searching for corresponding points in three reference phase maps. Therefore, correcting the projector lens distortions is more feasible in practical measurement. However, correcting distortions of a projector lens is more complex than correcting those of a camera lens. In this paper, we suggest two solutions, i.e., the pre- and post-compensation methods, for this issue.

Here, the pre-compensation method is derived. With it, the fringes are actively curved in computer when generating them. Because a projector has the same model as the camera with inverse directions of light rays, generating a fringe pattern deformed by Eq. (7) allows us to get undistorted fringes in the measurement space.

Assuming $(\hat{s},\hat{t})$ to be a pixel on the pre-corrected fringe patterns, with its scaled projector coordinates being ${\hat{x}_\textrm{p}} = {{(\hat{s} - {s_0})} \mathord{\left/ {\vphantom {{(\hat{s} - {s_0})} {{\kappa_s}}}} \right.} {{\kappa _s}}}$ and ${\hat{y}_\textrm{p}} = {{(\hat{t} - {t_0})} \mathord{\left/ {\vphantom {{(\hat{t} - {t_0})} {{\kappa_t}}}} \right.} {{\kappa _t}}}$. For determining the fringe intensity at this pixel, we have to solve the system of nonlinear equations in Eq. (7) for (x_p, y_p). By defining the initial values as $[x_\textrm{p}^{(0)},y_\textrm{p}^{(0)}] = [{\hat{x}_\textrm{p}},{\hat{y}_\textrm{p}}]$, the iterative solution for this nonlinear equation system is represented with

In a special application when measuring a batch of objects having the same shape, generating adaptive fringe patterns using similar technique is helpful for highlighting the flaws on the objects [46].

4.2 Post-compensation technique

The post-compensation technique projects standard fringe patterns onto the object without curving them in advance, and corrects the projector lens distortions in data processing stage.

In measurement, only fringes of one direction is used. Following from Section 3.1, we presume for the moment that vertical fringes perpendicular to s-axis are projected onto the measured object. Ignoring the projector lens distortions, the fringe phases are calculated. These calculated phases are affected by the projector lens distortions and denoted as ${\hat{\varPhi }_s}(u,v)$, corresponding to the coordinates $\hat{s} = {{{\lambda _s}{{\hat{\varPhi }}_s}(u,v)} \mathord{\left/ {\vphantom {{{\lambda_s}{{\hat{\varPhi }}_s}(u,v)} {2\pi }}} \right.} {2\pi }}$, i.e., the horizontal coordinates of distorted projector pixels. By substituting these matched pixel coordinates (u, v) and $\hat{s}$ into the first three equations of Eq. (10), the 3D coordinates of the measured object point are calculated as $({\hat{X}_w},{\hat{Y}_w},{\hat{Z}_w})$. Equation (7) implies that the projector lens distortions are two-dimensional functions of pixel coordinates. For correcting them, both the horizontal and vertical coordinates must be available. According to Eq. (9), we calculated the vertical coordinate of the distorted projector pixel as

4.3 Results

Here we experimentally verify the effectiveness of the proposed method by measuring objects. Firstly, we measure a standard cylinder in order to exam the accuracies of the methods. The cylinder has a nominal diameter of $\varPhi$149.48 mm, with a deviation of tolerance ± 0.020 mm. We use the system schemed in Fig. 1, which is calibrated in Section 3.4, to do the measurement.

In phase measuring stage, we use phase-shifting algorithm to analyze fringe patterns and use temporal phase-unwrapping technique to get absolute phase maps, as described in Section 3.1. In this procedure, vertical fringes perpendicular to $s$-axis are projected onto the object. The number of phase shifts is three and the phase increment between consecutive frames is 2π/3 radians. Figure 5 shows the results, with the panels in each row, from left to right, being a fringe pattern, the wrapped and the unwrapped phase map. The first row is obtained by projecting standard sinusoidal fringe patterns onto the cylinder, without considering the projector lens distortions. The second row shows the results of using the pre-compensation method in Section 4.1. In the second row, the fringe patterns have been purposely deformed in fringe generating stage, according to the coefficients of the projector lens distortions, so the projector lens distortions will not affect the measurement results.

 

Fig. 5. Phase measuring results. The panels in each row, from left to right, show a fringe pattern, the wrapped phase map, and the unwrapped phase map. The first row shows the results of projecting standard sinusoidal fringe patterns. The second row shows the results of using the pre-compensation fringe patterns. The colorbars have the unit of radian.

Download Full Size | PDF

After measuring the phases, we reconstruct 3D shape of the cylinder. The achieved accuracy depends not only on phase measurement accuracy, but also on the accuracies of system calibration and projector lens distortion correction. Figure 6(a) shows the reconstructed surface using the standard sinusoidal fringes in Fig. 5(a) without correcting the projector lens distortions. Figure 6(c) gives the results calculated from the same standard fringes, but the effects of projector lens distortions have been corrected through the post-compensation method suggested in Section 4.2. Figure 6(e) is the result of using the pre-compensation technique in Section 4.1, where the projector lens distortions have been corrected in fringe generating stage by actively deforming the fringes according to the calibrated coefficients. For highlighting the differences between these results, Fig. 7 plots cross-sections of these reconstructed surfaces at the same position. From these cross-sections, we observe that the pre- and the post-compensation results are very close to each other and deviate from the one without correcting the projector lens distortions.

 

Fig. 6. The top row shows the 3D reconstructed surfaces accompanying with their fitted least-squares cylinders. The bottom row gives the radial contours of residual errors, which is obtained by subtracting the nominal radius of the cylinder from the measured radiuses. (a) and (b) Using the standard fringe patterns without correcting the projector lens distortions. (c) and (d) Using the same patterns but the projector lens distortions are corrected using the post-compensation method. (e) and (f) Using the pre-compensation fringe patterns. The x, y, and z axes, and the colorbar have the unit of millimeter; and θ has the unit of degree.

Download Full Size | PDF

 

Fig. 7. Cross-sections of the reconstructed surfaces in Fig. 6. The pre- and the post-compensation results are very close to each other and deviate from the one without correcting the projector lens distortions.

Download Full Size | PDF

In accompany with the reconstructed surfaces, Fig. 6 also shows, in green, the least-squares cylinders obtained by fitting the measured point clouds. Table 1 compares their radiuses, from which we see that, after correcting the projector lens distortions, the radiuses of the fitted least-squares cylinders are very close to its nominal value. The above results demonstrate that using the proposed technique enables improving the measurement accuracy of global size of the object. For examining their effectiveness in improving the measurement accuracy of local profile, the second row of Fig. 6 shows the contours of the radial errors obtained by subtracting the nominal radius from the measurement results. Table 1 also lists the RMS values of these residual errors. It is evident that the residual errors by using two compensation techniques are much smaller than that without correcting the projector lens distortions. These results demonstrate that the system has been accurately calibrated by using the proposed technique, and that both the pre- and post-compensation techniques are effective in correcting the projector lens distortions thus being helpful for improving the measurement accuracies.

Table 1. Radiuses and RMS errors of least-squares cylinders (mm)

View Table

From the results just obtained, we know that both the pre- and post-compensation techniques can achieve almost the same accuracies. From the view of practice, the pre-compensation is more useful in measurement, because it allows us to measure different objects, without repeating correcting the distortions of projector lens, just by generating one sequence of fringe patterns. However, when we want to change the fringe frequency for getting proper resolutions adapting to different objects, the post-compensation technique shows a higher flexibility.

More experimental results are provided for further evaluating feasibility of our proposed techniques in measuring a complex object. Figure 8 show the results of measuring a mechanical part having a more complex shape than a cylinder. The top row shows the captured fringes, and below them are the reconstructed 3D shapes of the object. Note the first two columns share exactly the same fringe patterns, but in Fig. 8(d) the projector lens distortions are not corrected and in Fig. 8(e) these distortions are corrected by using the post-compensation technique. Figure 8(f) uses different fringe patterns which have been corrected using the pre-compensation technique. For highlighting the differences between these reconstructed shapes, Figs. 8(g) and 8(h) show the gaps between Fig. 8(e) and Fig. 8(d), and between Fig. 8(f) and Fig. 8(d), respectively. The maximum gap is about 0.45 mm, demonstrating that the projector lens distortions may severely deform the reconstructed shapes. Figure 8(i) shows the difference between Fig. 8(e) and Fig. 8(f), implying that both the pre- and post-compensation method have almost equal effects in correcting the projector lens distortions. Figure 9 having the same layout as that of Fig. 8 shows the results of measuring a plaster bust. These results demonstrate that the proposed calibration method works well for measuring complex objects.

 

Fig. 8. Measurement results of a mechanical part. (a) and (b) are the same captured patterns when projecting standard fringes onto the object without correcting the projector lens distortions. (c) is the captured pattern when projecting curved fringes according to the calibrated coefficients of the projector lens distortions. Below them, (d) shows the reconstructed shape without correcting the projector lens distortions, (e) and (f) are the results with the projector lens distortions are corrected using pre- and post-compensation methods, respectively. In the bottom row, (g), (h), and (i) show the gaps between (e) and (d), between (f) and (d), and between (e) and (f), respectively.

Download Full Size | PDF

 

Fig. 9. Measurement results of a plaster bust. The layout is the same as that of Fig. 8.

Download Full Size | PDF

5. Discussions

With the proposed techniques, there are some issues worth discussing. For example, random noise, projector nonlinearity, and illumination fluctuation may degrade the accuracies of calibration and measurement by inducing errors in phase measuring results. Even though, the proposed calibration technique is less sensitive to these error-inducing factors, because it can average out the induced errors by fitting the phase maps using rational functions in the least-squares sense. In measurement, however, we must cope with these errors carefully. Among them, the nonlinearity [47] of the projector is the most crucial factor affecting the measurement accuracy. This projector nonlinearity induces ripple-like artifacts having a frequency M times higher than the fringes with M being the number of phase shifts [48,49]. Therefore, increasing M is helpful for restraining the effects of the projector nonlinearity [50]. In this work, we use a more effective method that depends on few fringe patterns and enables recognizing and removing the errors directly from the calculated phase maps [51]. Random noise in fringe patterns [52] always affects the measurement accuracy. Generally, noise in a pattern is induced by complicated physical factors, and can be simply modeled with the Gaussian distribution [53] according to the central limit theorem from the theory of probability. When noise is present, the variance of the induced phase errors is proportional to the variance of noise, and inversely proportional to the number of phase shifts and to the square of modulations [54]. Therefore, increasing the number of phase shifts can compress the influence of noise, at the expense of time duration for image capturing. The illumination fluctuation induces ripple-like phase errors having the same frequency as fringes, which can be corrected with the aid of fringe histograms [55]. In fringe projection profilometry, phase sensitivity is also a main factor determining the measurement resolution and accuracy. This phase sensitivity mainly depends on the system geometry, and simultaneously is associated with fringe pitch and fringe orientation [56]. For a fixed system, using fringes having finer pitch or with a direction perpendicular to the epipolar lines on the projector plane, can achieve a higher phase sensitivity.

Another issue is regarding efficiency. With this calibration method, the time duration is mainly expended in capturing the images, especially when the checkerboard position is adjusted manually. In our experiment, for example, we have to spend several minutes to change the checkerboard poses and capture the required images. In comparison with the existing system calibration techniques based on phase measuring, however, the proposed technique does not require capturing extra images. Although it involves an additional iterative procedure for data processing, increase in calibration time is not so noticeable, thanks to the development of computer technology. For example, we use a computer (Dell Precision T7910) having Intel Xeon CPU with 2.40 GHz and 32G RAM, and the overall data processing for calibration can be completed within three minutes.

6. Conclusion

In fringe projection profilometry, the difficulty in calibrating a system lies in determining projector parameters. This paper has proposed a solution for this issue based on phase measuring. It exploits the fact that the fringe phases on a plane board theoretically have a distribution of rational function, and estimates the projector parameters by implementing the phase fitting and the parameter estimating alternately. In comparison with the existing system calibration techniques based on phase measuring, this technique offers some superior performances. For example, it applies the most popularly used checkerboard as the calibration target, having a properly compat with software readily available. It extracts the featured points from fitted phase map, being less sensitive to the low-pass property of the projector lens. It enables averaging out phase errors induced by such factors as noise, projector nonlinearities, and illumination fluctuations. It does not require capturing extra images, having a satisfied efficiency. We experimentally validate this technique by calibrating a system and then using it to measure objects. In measurement, the projector lens distortions are corrected, according to the calibrated coefficients, by using a pre- and a post-compensation technique thus improving the measurement accuracy.