Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Color crosstalk compensation method for color phase-shifting fringe projection profilometry based on the phase correction matrix

Open Access Open Access

Abstract

Color phase-shifting fringe projection profilometry is one of the single-shot three-dimensional shape measurement techniques. The color crosstalk of the projector-camera system yields undesired phase errors when using phase-shifting method. In this paper, a color crosstalk compensation method based on phase correction matrix is proposed. In this method, the phase correction matrix is established to compensate the deviations between the actual phase-shift values in the acquired fringes and the standard ones in the ideal fringes. Only two fringe patterns are utilized to obtain the phase correction matrix. The quadratic equations for calculating the actual phase-shift values of the fringes in the three color channels are derived. The actual phase-shift values and the corresponding standard ones are employed to form the equilibrium equations for computing the phase correction coefficients in the matrix. Experimental results demonstrate the feasibility of the proposed method and it can effectively reduce the induced overall phase error caused by the color crosstalk.

© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Fringe projection profilometry (FPP) has been widely used in 3D shape measurement due to its low cost, noncontact, high resolution and high measurement efficiency [1,2], which can be categorized into Fourier Transform Profilometry (FTP) and Phase-shifting fringe projection Profilometry (PSP) [35]. FTP is capable of reconstructing the 3D shape of objects with single shot. However, it is susceptible to complex surfaces that can easily cause spectral aliasing in phase demodulation [6]. PSP can implement the pixel-wise phase measurements with high accuracy. However, the multi-shot nature of PSP limits its efficiency [7,8]. To improve the measuring speed of PSP, the color phase-shifting fringe projection profilometry (color PSP) approach has been presented. The three-step phase-shift fringes are encoded into one color fringe pattern as its RGB (red, green, and blue) components. The color pattern is projected onto the object. The deformed color pattern is captured by a RGB camera, which is utilized to restore the deformed three-step phase-shift fringes by color isolation for single-shot measurement [912].

However, the existence of color crosstalk, which is mainly caused by the spectral response of the color camera and projector, yields undesired phase errors when using phase-shifting algorithm. Many studies have been performed on compensating the color crosstalk effect. Huang et al. [13] proposed a compensation method using crosstalk matrix. Nine different uniform color fringes were projected to obtain the matrix. However, the variation of the intensities in the nine captured images introduced errors in the calculation of the proportion of the three-channel modulation intensities. Moreover, the method is not efficient due to the large number of projected fringe patterns. Pan et al. [14]proposed an improved crosstalk compensation method. Three color fringes were encoded and projected onto a white plate. The average and modulation intensities in the three color channels were calculated respectively. The intensities in one channel was selected as the reference to correct the ones in the other two channels. However, this method ignores the effect on the phase-shift value from color crosstalk. Thus, the phase errors cannot be completely eliminated. Zhang et al. [15] employed four-step phase-shift fringes to be encoded in the four color patterns. The multi-frequency heterodyne method was utilized to calculate the optimal fringe period for the three channels. Twelve color patterns were projected on to a white plate. The ratios of the modulation intensities was calculated and thus the crosstalk matrix is obtained. The variation of the intensities is inevitable during the image acquisition process, which makes the average intensities in the fringes hard to be subtracted. Wang et al. [16] proposed the Hilbert transform to eliminate the phase errors resulting from the color crosstalk. However, this method ignored the effect of crosstalk on the phase-shift values in the three channels, which brings residual phase errors in dealing with the composite fringe with severe color crosstalk.

Aiming at the above issue, this paper proposed an improved color crosstalk compensation method based on phase correction matrix. We analyzed the aspects in each color channel affected by color crosstalk, including the average intensity, the modulation intensity and the phase-shift value. A simple and effective method for calculating the actual phase-shift values of the captured color fringe was investigated. The phase correction matrix was employed to rectify the actual phase-shift values to the standard ones. The equations were established for calculating the elements in the phase correction matrix by the actual phase-shift values. The proposed method is composed of the following steps: To begin with, one gray fringe pattern and one three-step phase-shift color fringe pattern were projected onto a white plate and captured by the camera successively. The second step is the gamma correction to eliminate the nonlinear crosstalk. After that, the normalization of the average intensity and modulation intensity was performed on the captured fringes after the gamma correction. Next, the quadratic equations were established to calculate the actual phase-shift values, which deviates from the standard ones due to the color crosstalk. Subsequently, the elements in the phase correction matrix were calculated to compensate the deviation from the actual phase-shift values to the standard ones. Finally, the phase correction matrix was employed to reduce the phase errors resulting from the color crosstalk. The simulation experiments were designed to evaluate our method. The accuracy of our method was verified by measuring a pair of ceramic balls and a step workpiece. In additional, 3D reconstruction experiments were carried out to verify the performance of our method, which was compared with traditional phase-shift method, the method using color crosstalk matrix [13] and the Hilbert transform method [16].

The main contributions of our method are shown as follows:

  • • Only two fringe patterns are utilized to obtain the phase correction matrix, including one gray fringe pattern and one color pattern carrying the three-step phase-shift fringes.
  • • The quadratic equations for calculating the actual phase-shift values of the fringes in the three color channels are established. With the help of the fundamental frequency components, which can be easily obtained from the captured gray fringe pattern, the quadratic equations can be accurately solved.
  • • The actual phase-shift values and the corresponding standard ones are employed to form the equilibrium equations for computing the phase correction coefficients. The phase errors resulting from the color crosstalk could be eliminated for the color pattern with arbitrary fringe frequency.

The remainder of this paper is organized as follows. Section 2 outlines some about preliminaries the color phase-shifting fringe projection profilometry and color crosstalk model. Section 3 provides the detailed descriptions of our method. Section 4 presents the simulation experiments. Section 5 discusses the 3D reconstruction results. The paper ends with some concluding remarks in Section 6.

2. Preliminaries

2.1 Color phase-shifting fringe projection profilometry

Figure 1 depicts the schematic diagram of the color phase-shifting fringe projection profilometry. The color fringe pattern is projected onto the scene by the projector, which carries the phase-shift fringes in the three color channels. The deformed color fringe is collected by the color camera. The phase-shift fringe in each color channel can be extracted by color isolation to obtain the wrapped phase map. Then wrapped phase is unwrapped to get a continuous phase distribution for the 3D reconstruction.

 figure: Fig. 1.

Fig. 1. Schematic diagram of color phase-shifting fringe projection profilometry.

Download Full Size | PDF

The intensities of the three-step phase-shift fringes encoded in the color pattern is expressed as:

$${I_n}(x,y) = {A_0}(x,y) + {B_0}(x,y)\cos [\phi (x,y) + {\delta _n}],$$
where n represents the red (r), green (g) and blue (b) channels. Ideally, δn are -2π/3, 0 and 2π/3, respectively; In are the intensities. $\phi (x,y)$ is the phase distribution that carries the height information. A0 is the average intensity. B0 is the modulation intensity. $\phi (x,y)$ can be calculated as:
$$\phi (x,y) = {\tan ^{ - 1}}\left[ {\frac{{\sqrt 3 ({I_r}(x,y) - {I_b}(x,y))}}{{2{I_g}(x,y) - {I_r}(x,y) - {I_b}(x,y)}}} \right],$$
$\phi (x,y)$ is wrapped in the range of (-π, π] due to the arctangent function. Phase unwrapping should be carried out to retrieve the continuous phase distribution.

2.2 Color crosstalk model

There are two main phase error sources distorting the ideal sinusoidal fringe patterns, gamma nonlinearity effect and overlaps between each color channels. Combining the gamma nonlinearity and color crosstalk, the acquired fringe pattern can be expressed using as

$$\left[ {\begin{array}{{c}} {I_r^{\prime}(x,y)}\\ {I_g^{\prime}(x,y)}\\ {I_b^{\prime}(x,y)} \end{array}} \right] = \textrm{C}\left[ {\begin{array}{{c}} {{{[{{I_{or}}(x,y)} ]}^{\gamma R}}}\\ {{{[{I_{og}}(x,y)]}^{\gamma G}}}\\ {{{[{{I_{ob}}(x,y)} ]}^{\gamma B}}} \end{array}} \right] = \left[ {\begin{array}{{ccc}} {{C_{11}}}&{{C_{12}}}&{C{}_{13}}\\ {{C_{21}}}&{{C_{22}}}&{{C_{23}}}\\ {{C_{31}}}&{{C_{32}}}&{{C_{33}}} \end{array}} \right]\left[ {\begin{array}{{c}} {{{[{{I_{or}}(x,y)} ]}^{\gamma R}}}\\ {{{[{I_{og}}(x,y)]}^{{\gamma^G}}}}\\ {{{[{{I_{ob}}(x,y)} ]}^{\gamma B}}} \end{array}} \right],$$
where $I_r^{\prime}(x,y)$, $I_g^{\prime}(x,y)$ and $I_b^{\prime}(x,y)$ are the RGB channels of the acquired pattern. γR, γG and γB are the gamma factors. Ior, Iog and Iob are the RGB channels of the projected pattern. C denotes the crosstalk matrix. Cij (i,j = 1,2,3) are the crosstalk coefficients.

The nonlinearity effect can be reduced by the gamma correction [17,18]. Assuming that the gamma nonlinearity has been corrected, $I_r^{\prime}(x,y)$, $I_g^{\prime}(x,y)$ and $I_b^{\prime}(x,y)$ can be simply expressed as [19]:

$$I_n^{\prime}(x,y) = A_n^{\prime}(x,y) + B_n^{\prime}(x,y)\cos [\phi (x,y) + {\delta _n} + {\varepsilon _n}],$$
$$A_n^{\prime} = {A_0}({C_{n,1}} + {C_{n,2}} + {C_{n,3}}),$$
$$B_n^{\prime} = {B_0}\sqrt {\frac{1}{2}[{{({C_{n,1}} - {C_{n,2}})}^2} + {{({C_{n,2}} - {C_{n,3}})}^2} + {{({C_{n,1}} - {C_{n,3}})}^2}]} ,$$
$$\tan ({\theta _n}) = \frac{{ - \sqrt 3 ({C_{n,1}} - {C_{n,3}})}}{{2{C_{n,2}} - {C_{n,1}} - {C_{n,3}}}},$$
where $I_n^{\prime}(x,y)$ denotes the intensity in each RGB channel of a pixel in the acquired fringe pattern. $A_n^{\prime}({x,y} )$ and $B_n^{\prime}({x,y} )$ denote the average intensity and modulation intensity. ɛr and ɛg is positive, ɛb is negative, for the detailed proof process, please refer to Appendix A. θn= δn + ɛn denotes the actual phase-shift values in the three channels.

The frequency of the fringe pattern is not affected by the color crosstalk. However, the variations of the average intensity, modulation intensity and phase-shift value in each channel are induced by the color crosstalk. The actual phase-shift values cannot be ignored during the color crosstalk compensation.

3. Proposed method

In this section, the color crosstalk compensation method based on phase correction matrix is elaborated in detail. As shown in Fig. 2, the whole process can roughly be divided into six steps. At the first stage, one gray fringe pattern and one three-step phase-shift color fringe pattern were projected onto a white plate and captured by the camera successively. The second step is the gamma correction. Thus, only linear crosstalk remains in the color fringes. Next, the normalization of the average and modulation intensities was implement to eliminate the intensity errors resulting from the color crosstalk [20]. However, the absolute phase differences between the three-step fringes in RGB channels were not equal to 2π/3. The deviation of the actual phase-shift values from the standard ones was the residual issue. After that, the quadratic equations were established to accurately calculate the actual phase-shift values with the help of the fundamental frequency components, obtained from the captured gray fringe pattern. Subsequently, the elements in the phase correction matrix were calculated to compensate the deviation from the actual phase-shift values to the standard ones. Finally, the phase correction matrix was employed to compensate the deviation of the phase-shift values.

 figure: Fig. 2.

Fig. 2. Flowchart of color crosstalk compensation method based on phase correction matrix.

Download Full Size | PDF

The calculation of the actual phase-shift values in each channel and the establishment of the phase correction matrix are descripted in detail as follows.

3.1 Calculation of the actual phase-shift values

The method in [20] was employed to implement the intensity normalization. First, the high-pass filtering is operated on $I_n^{\prime}(x,y)$ to remove the average intensity, resulting in $I_n^{HP}(x,y)$ . Subsequently, $I_n^{HP}(x,y)$ is subjected to the Hilbert transform. The Hilbert transform can shift the phase of negative frequency components by π/2, and the phase of positive frequency components by −π/2, resulting in $I_n^{HT}(x,y)$ [16]. Due to the presence of high-pass filtering and Hilbert transformation, filtering effects on frequency domain information occur, resulting in loss of detailed information during the reconstruction of complex surfaces. The expressions of $I_n^{HP}(x,y)$ and $I_n^{HP}(x,y)$ are as follows.

$$I_n^{HP}(x,y) = B_n^{\prime}(x,y)\cos [\phi (x,y) + {\theta _n}],$$
$$I_n^{HT}(x,y) = B_n^{\prime}(x,y)\sin [\phi (x,y) + {\theta _n}],$$

Finally, the normalized fringes are obtained through Eq. (10). The intensities of the fringes in the three color channels can be expressed as:

$$\begin{aligned} {{\hat{I}}_n}(x,y) &= \frac{{I_n^{HP}(x,y)}}{{\sqrt {{{(I_n^{HP}(x,y))}^2} + {{(I_n^{HT}(x,y))}^2}} }}\\ &= \cos [\phi (x,y) + {\theta _n}], \end{aligned}$$

By expanding Eq. (10), a quadratic equation with independent variable θn can be established as:

$$\textrm{co}{\textrm{s}^2}{\theta _n} - 2{\hat{I}_n}(x,y)\cos \phi (x,y)\cos {\theta _n} + \hat{I}_n^2(x,y) - {\sin ^2}\phi (x,y) = 0,$$
where $\Delta = 4{\sin ^2}\phi ({x,y} )({1 - \hat{I}_n^2({x,y} )} )> 0$,which means the equation has two solutions as:
$$\cos {\theta _n} = {\hat{I}_n}(x,y)\cos \phi (x,y) \pm \sqrt {({1 - {{\cos }^2}\phi (x,y)} )({1 - \hat{I}_n^2(x,y)} )} ,$$

The standard three-step phase-shift values are −2π/3, 0 and 2π/3. Generally, the actual phase-shift values in the RGB channels are limited within the following ranges: cosθr ranging from −2π/3 to −2π/3+π/6, cosθg ranging from 0 to 0+π/6, cosθb ranging from 2π/3 to 2π/3-π/6. Therefore, the accurate solution of cosθn in Eq. (12) can be obtained when the fundamental frequency component $\cos \phi $ is known.

To obtain the fundamental frequency components $\cos \phi $, a gray fringe pattern is projected onto the white plate. The intensity of the gray fringe pattern is expressed as:

$$I(x,y) = {A_0} + {B_0}\cos \phi (x,y),$$

The intensity of the gray fringe pattern is normalized. Then the fundamental frequency component can be easily obtained:

$${I^N}(x,y) = \cos \phi (x,y),$$

To sum up, the actual phase-shift values θn in each channel can be obtained according to Eq. (12) and Eq. (14).

3.2 Establishment of the phase correction matrix

The phase correction matrix was established to compensate the deviations between the actual phase-shift values θn to the standard ones δn. The matrix is expressed as:

$$\left[ {\begin{array}{{c}} {{{\tilde{I}}_r}}\\ {{{\tilde{I}}_g}}\\ {{{\tilde{I}}_b}} \end{array}} \right] = \textrm{A}\left[ {\begin{array}{{c}} {{{\hat{I}}_r}}\\ {{{\hat{I}}_g}}\\ {{{\hat{I}}_b}} \end{array}} \right] = \left[ {\begin{array}{{ccc}} 1&{{a_{12}}}&{{a_{13}}}\\ {{a_{21}}}&1&{{a_{23}}}\\ {{a_{31}}}&{{a_{32}}}&1 \end{array}} \right]\left[ {\begin{array}{{c}} {{{\hat{I}}_r}}\\ {{{\hat{I}}_g}}\\ {{{\hat{I}}_b}} \end{array}} \right]$$
where ${\hat{I}_r}$, ${\hat{I}_g}$ and ${\hat{I}_b}$ are the normalized intensities of the acquired fringes with actual phase-shift values in RGB channels. ${\tilde{I}_r}$, ${\tilde{I}_g}$ and ${\tilde{I}_b}$ are the normalized intensities of the ideal fringes with standard phase-shift values in RGB channels. A is the phase correction matrix.

θn and δn are employed to form the equilibrium equations for computing the phase correction coefficients. Taking the red channel as an example, the computation process of the phase correction coefficients a12 and a13 is as follows. By expanding Eq. (15), the equilibrium equation between the ideal fringe and acquired fringe in the red channel can be expressed as:

$${\tilde{I}_r}(x,y) = {\hat{I}_r}(x,y) + {a_{12}}{\hat{I}_g}(x,y) + {a_{13}}{\hat{I}_b}(x,y),$$

Thus, the relationship between δr and θn can be expressed as:

$$\cos [\phi (x,y) + {\delta _r})] = \cos [\phi (x,y) + {\theta _r}] + {a_{12}}\cos [\phi (x,y) + {\theta _g}] + {a_{13}}\cos [\phi (x,y) + {\theta _b}],$$

According to the sum-to-product formula of trigonometric functions, the linear equations h1(a12, a13) of a12 and a13 can be derived as:

$$h({a_{12}},{a_{13}}):\left\{ {\begin{array}{{c}} {\cos {\theta_r} + {a_{12}}\cos {\theta_g} + {a_{13}}\cos {\theta_b} ={-} \frac{1}{2}}\\ {\sin {\theta_r} + {a_{12}}\sin {\theta_g} + {a_{13}}\sin {\theta_b} ={-} \frac{{\sqrt 3 }}{2}} \end{array}} \right.$$

The a12 and a13 can be easily calculated by substituting the known θn in to Eq. (18). Likewise, the coefficients for the green and blue channels can be obtained.

4. Simulation

Two simulation experiments were carried out to verify the effectiveness of the proposed method. The algorithms were implemented on the MATLAB 2022 platform. Gamma correction was assumed to be completed and thus the gamma value of each channel was 1. The ideal pattern was generated using the following parameters. The average intensity A0 and modulation intensity B0 of the pattern were both set to 0.5. The pattern resolution was 500 × 500 pixels. The fringe period was 25 pixels.

The phase error induced by the color crosstalk was added into the ideal pattern to generate the crosstalk pattern. The crosstalk matrix was given as Eq. (19).

$$\textrm{C} = \left[ {\begin{array}{{ccc}} {1.00}&{0.36}&{0.08}\\ {0.15}&{1.00}&{0.20}\\ {0.06}&{0.25}&{1.00} \end{array}} \right],$$

The phase-shift values in the RGB channels were calculated by the proposed method as θr, θg and θb. Then the phase correction matrix was obtained by the proposed method as:

$$\textrm{A} = \left[ {\begin{array}{{ccc}} {1.00}&{ - 0.2381}&{0.1289}\\ {0.0301}&{1.00}&{ - 0.0246}\\ {0.0731}&{ - 0.1589}&{1.00} \end{array}} \right],$$

Figure 3 provides the phase errors of the simulated crosstalk pattern. As shown in Fig. 3(a), the root mean square error (RMSE) in the wrapped phase map extracted from the pattern was 0.1277 rad. After normalization of the average and modulation intensities, RMSE was reduced to 0.1090 rad, as shown in Fig. 3(b). RMSE was approximately reduced to 0 rad by using the proposed phase correction matrix, as shown in Fig. 3(c). Figure  3(d) presents one cross-section of the phase errors, $\Delta \phi $ for (a), $\Delta {\phi ^N}$ for (b) and $\Delta {\phi ^A}$ for (c), respectively. $\Delta {\phi ^A}$ was close to 0 along the whole cross-section.

 figure: Fig. 3.

Fig. 3. Phase errors of the simulated crosstalk pattern. (a) phase errors of the simulated crosstalk pattern, (b) phase errors after intensity normalization, (c) phase errors by using the proposed phase correction matrix and (d) one cross-section of the phase errors.

Download Full Size | PDF

To simulate the actual captured color pattern, an uneven surface was generated by the peaks function in MATLAB to produce the deformed crosstalk pattern. The pattern resolution is 500 × 500 pixels. The fringe period is changed to 50 pixels. The same crosstalk matrix was still as Eq. (19). Besides, the effect of the nonuniform illumination was also simulated as

$$R(x,y) = 0.5 + 0.5 \times G(x,y)/\max [G(x,y)],$$
where G(x, y) denotes the 2D Gaussian function with a size of 500 × 500 pixels and a standard deviation of 150 pixels.

Furthermore, the Gaussian noise with two different variances was added to the simulated image. Figure  4(a) to (d) presents the phase errors of the deformed crosstalk pattern without noise; Fig. 4(e) to (h) presents the phase errors of the pattern with Gaussian noise (σ=0.0001); Fig. 4(i) to (l) presents the phase errors of the pattern with Gaussian noise (σ=0.001).

 figure: Fig. 4.

Fig. 4. Phase errors of the deformed crosstalk pattern without noise and with Gaussian noise (σ=0.0001 and 0.001). (a)(e)(i) phase errors of the deformed crosstalk pattern without correction. (b)(f)(j) phase errors after intensity normalization. (c)(g)(k) phase errors by using the proposed phase correction matrix. (d)(h)(l) one cross-section of the phase errors.

Download Full Size | PDF

In Fig. 4(a), (e), (i), RMSE of the wrapped phase map without correction was 0.0977 rad, 0.1158 rad and 0.1304 rad, respectively. After normalization of the intensities, RMSE was reduced to 0.0701 rad, 0.0790 rad and 0.0801 rad, respectively, as shown in Fig. 4(b), (f), (j). In Fig. 4(c), (g), (k), RMSE was reduced to 0.0015 rad, 0.0071 rad and 0.0149 rad, respectively, by using the same phase correction matrix as Eq. (20). Figure 4(d), (h), (l) present one cross-section of the phase errors, $\Delta \phi $ for (a), (e), (i), $\Delta {\phi ^N}$ for (b), (f), (j) and $\Delta {\phi ^A}$ for (c), (g), (k), respectively. $\Delta {\phi ^A}$ was the smallest along the whole cross-section. The RMSE error was reduced by about 98.5%, 93.9% and 88.6%, respectively.

The simulation results proved that the same phase correction matrix A is capable of eliminating the phase errors from color crosstalk even the fringe frequency varies. Furthermore, the proposed method is still effective when nonuniform illumination, irregular deformation and noise occur.

5. Experiments

In this section, 3D reconstruction experiments were carried out to verify the performances of the proposed method, compared with the traditional phase-shift method, the method using color crosstalk matrix [13]and the Hilbert transform method [16]. The experimental setup was consisted of one color camera (MV-CA020-10UC, 1624 × 1240 pixels) equipped with 16 mm lenses and one DLP projector (ANHUA M26EGLC, 1920 × 1080 pixels) as shown in Fig. 5. System calibration was implemented by using the method in [21]. The phase-shift values in the three channels of the color pattern were −2π/3, 0, 2π/3. The fringe frequency was 70.

 figure: Fig. 5.

Fig. 5. Calibrated CPSP 3D Measurement System.

Download Full Size | PDF

Firstly, gamma correction was performed on each color channel by using the method in [17]. The gamma value of the gray fringe was γGray =1.8722, while the gamma values of the color fringes were γr = 1.9231, γg = 1.8583 and γb = 1.9886. The actual phase-shift values of the fringes in the three channels were calculated by using our method, θr = −107.7275°, θg = 6.1187°and θb = 108.4390°. The obtained correction matrix was expressed as:

$${\textrm{A}_1} = \left[ {\begin{array}{{ccc}} 1&{ - 0.1631}&{0.1100}\\ {0.0481}&1&{ - 0.0641}\\ {0.0684}&{ - 0.1639}&1 \end{array}} \right],$$

Figure 6(a) shows the captured color pattern on the white plate. Figure 6(b) and (c) present one cross section (500th row) before and after compensation, including intensity normalization and phase correction. Both the average and the modulation intensities of the three channels were obviously different in the captured color pattern. Moreover, the phase-shift values in the three color channels also deviated from the standard ones. After compensation by the proposed method, the intensities were equal and the phase-shift values were corrected to the ideal standard values.

 figure: Fig. 6.

Fig. 6. Captured color pattern on the white plate and the one cross-section of the phase errors (a) captured color pattern, (b) the cross-section of the phase errors before compensation, (c) the cross-section of the phase errors after compensation.

Download Full Size | PDF

5.1 Accuracy analysis

To verify the accuracy of our method, a pair of standard ceramic balls and a step workpiece were measured. Figure 7(a) to Fig. 7(b) provides the parameters of the standard objects. The radii of ceramic ball A and B are 25.4030 mm and 25.4070 mm, respectively. The distance between the centers of the two balls is 120.0570 mm. The step height of the workpiece is 25 mm. The manufacturing error of the ceramic ball is less than 5µm and the error of the workpiece is less than 50µm.

 figure: Fig. 7.

Fig. 7. Accuracy analysis experiments of the 3D measurement system. (a) two ceramic standard spheres to be measured. (b) a stepped workpiece. (c-e) 3D reconstruction results of spheres A, B and stepped workpiece. (f-h) plots of the difference distribution between the reconstructed and fitted data of spheres A, B and stepped surface 3. (i-j) quantitative histograms of error plots (f-g). (k) error box line plots of surface 1, surface 2, surface 3 and surface 4.

Download Full Size | PDF

The 12-step phase-shift method was employed in these experiments. Three color patterns were generated, which carry three groups of three-step phase-shift fringe. The frequencies of the three groups were 105, 99, and 94, respectively. The simulation experiments proved that the proposed method is effective for the color patterns with different fringe frequencies. Thus, the same phase correction matrix A1 was utilized to compensate the errors for the three color patterns.

The 3D reconstruction results were shown in Fig. 7(c) to (e). The fitted radii of ball A and B were 25.5121 mm and 25.3853 mm, respectively. The center distances of the two balls was 120.1184 mm. The distances between the step surfaces were 25.0845 mm, 25.0905 mm and 25.0806 mm. Figure 7(f) and (g) presents the deviations between the 3D points and the fitted sphere. Figure 7(h) presents the deviations between the 3D points and the fitted plane of the third step surface. Figure  7(i) and (j) provides the quantitative histograms of the deviations in Fig. 7(f) and (g). The RMSE of the 3D reconstruction results of the two balls are 42 µm and 32 µm, respectively. Figure 7(k) provides box plots of the fitting errors for the step surfaces. The RMSE were 31 µm, 36 µm, 41 µm and 40 µm, respectively.

5.2 Comparison with existing methods

3D reconstruction experiments of real objects were designed to evaluate the performance of the proposed method, comparing with other three methods. The three-frequency heterodyne method was employed to unwrap the phase map extracting from the three-step phase-shift fringes. The fringe frequencies were 105, 99, and 94, respectively. The obtained phase correction matrix A1 was utilized to compensate the errors from the color crosstalk.

A smooth sphere was measured. Figure 8(a) shows the unwrapped phase map directly obtained by the traditional three-step phase-shift method. The phase map was not completely unwrapped. Periodic phase errors occurred in a relatively large region. Figure 8(b) provides the unwrapped phase map obtained by the crosstalk matrix method [13]. The effect of eliminating the phase errors from the color crosstalk was slight. Figure 8(c) presents the unwrapped phase map obtained by the Hilbert transform method [16]. There were still some residual phase errors. Figure 8(d) illustrates the unwrapped phase map obtained by our method. The phase errors were well compensated. Furthermore, the 20-step phase-shift method was utilized to measure the same sphere to gather the ground truth of the unwrapped phase map. The deviation maps between the unwrapped phase maps obtained by the four method and the ground truth were shown in Fig. 8(e) to (h). The RMSE of the four methods were 0.3844 rad, 0.2675 rad, 0.2169 rad and 0.0821 rad, respectively. By contrast, the RMSE of our method is reduced by 78.6%, 69.3%, and 62.1%, respectively.

 figure: Fig. 8.

Fig. 8. Unwrapped phase maps (a) original crosstalk fringe, (b) crosstalk matrix method, (c) Hilbert transform method, (d) our method, and (e) ∼(h) are phase error maps of (a) ∼ (d).

Download Full Size | PDF

In the measurement experiment, a set of colored patterns is projected onto the object for measurement. Figure  9(a) displays the deformed color fringe patterns captured by the camera. The 20-step phase-shift method was employed to measure the same mask, aiming to acquire the ground truth for the unwrapped phase map. The unwrapped phase maps were obtained using the conventional three-step phase-shift method, the crosstalk matrix method, the Hilbert transform method, and the proposed method. The deviation maps between the ground truth map and the unwrapped phase maps of the four methods are depicted in Fig. 9(b) to (e). The RMSE of the four methods were 0.9101 rad, 0.8104 rad, 0.1382 rad and 0.0931 rad, respectively. By contrast, the RMSE of our method is reduced by 89.8%, 88.5%, and 32.63%, respectively.

 figure: Fig. 9.

Fig. 9. (a) Collected deformed fringe pattern, (b)∼(e) phase error of traditional method, crosstalk matrix method, Hilbert transform method and our method respectively.

Download Full Size | PDF

Figure  10(a) to (d) shows the 3D reconstruction results of a mask by using the four methods. The ground truth was also obtained by using the 20-step phase-shift method. Figure  10(e) compares one profile line on the 3D shapes of the four methods to the ground truth. The profile line of our method was closest to the ground truth. The experimental results proved that the proposed method is effective in compensating the errors from the color crosstalk.

 figure: Fig. 10.

Fig. 10. 3D reconstruction of (a) original crosstalk fringe, (b)crosstalk matrix method, (c) Hilbert transform method,(d) the method proposed in this paper, (e) is a cross section of (a) ∼ (d)

Download Full Size | PDF

6. Conclusion

A color crosstalk compensation method based on phase correction matrix has been proposed for color PSP. In this method, only one gray fringe pattern and one color pattern are utilized to obtain the phase correction matrix. The fundamental frequency components can be easily obtained from the captured gray fringe pattern. We establish the quadratic equations for accurately calculating the actual phase-shift values of the fringes in color pattern. The actual phase-shift values and the corresponding standard ones are employed to form the equilibrium equations for computing the coefficients in the phase correction matrix. Experiments verify the feasibility of the proposed method and its good performance on 3D reconstruction. The normalization process employed in this paper involves filtering the frequency components, which reduces the ability of our method to reconstruct complex structures. This is a key issue that we need to further investigate in our subsequent research.

Appendix A

The crosstalk matrix C is:

$$\textrm{C} = \left[ {\begin{array}{{ccc}} {{C_{11}}}&{{C_{12}}}&{C{}_{13}}\\ {{C_{21}}}&{{C_{22}}}&{{C_{23}}}\\ {{C_{31}}}&{{C_{32}}}&{{C_{33}}} \end{array}} \right]$$

As Eq. (7) in the main text, the actual phase-shift θn in the three channels caused by color crosstalk can be expressed as:

$$\tan ({\theta _n}) = \frac{-{\sqrt 3 ({C_{n,1}} - {C_{n,3}})}}{{2{C_{n,2}} - {C_{n,1}} - {C_{n,3}}}}$$
θn= δn + ɛn denotes the actual phase-shift values in the three channels; δn are −2π/3, 0 and 2π/3, respectively.

In Fig.  11, the crosstalk between red and blue channels is smaller than that between red and green channels or between blue and green channels, blue and green channels is bigger than that between red and green channels, the relationship between the elements of the crosstalk matrix is shown in Table 1 as follows.

 figure: Fig. 11.

Fig. 11. MV-CS050-60UC Quantum Efficiency

Download Full Size | PDF

Tables Icon

Table 1. Relationship of Crosstalk Matrix Elements.

The phase shift ${\varepsilon _r}$ caused by color crosstalk in the red channel is:

$$\begin{aligned} \tan ({\varepsilon _r}) &= \tan ({\theta _r} - {\delta _r})\\ &= \frac{{\tan ({\theta _r}) - \tan ({\delta _r})}}{{1 + \tan ({\theta _r})\tan ({\delta _r})}}\\ &= \frac{{\frac{{ - \sqrt 3 ({{C_{11}} - {C_{13}}} )}}{{2{C_{12}} - {C_{11}} - {C_{13}}}} - \sqrt 3 }}{{1 + \frac{{ - 3({{C_{11}} - {C_{13}}} )}}{{2{C_{12}} - {C_{11}} - {C_{13}}}}}}\\ &= \frac{{ - 2\sqrt 3 {C_{12}} + 2\sqrt 3 {C_{13}}}}{{ - 4{C_{11}} + 2{C_{12}} + 2{C_{13}}}} \end{aligned}$$
 By substituting the data in the Table 1 into the Eq. (25), the numerator on the right side of the formula is negative, and the denominator is negative too, hence $\tan {\varepsilon _r} > 0$, which proves that ${\varepsilon _r} > 0$.

The phase shift ${\varepsilon _g}$ caused by color crosstalk in the red channel is:

$$\begin{aligned} \tan ({\varepsilon _g}) &= \tan ({\theta _g} - {\delta _g})\\ &= \frac{{\tan ({\theta _g}) - \tan ({\delta _g})}}{{1 + \tan ({\theta _g})\tan ({\delta _g})}}\\ &= \frac{{ - \sqrt 3 ({{C_{21}} - {C_{23}}} )}}{{2{C_{22}} - {C_{21}} - {C_{23}}}} \end{aligned}$$
By substituting the data in the Table 1 into the Eq. (26), the numerator on the right side of the formula is positive, and the denominator is positive too, hence $\tan {\varepsilon _g} > 0$, which proves that ${\varepsilon _g} > 0$.

The phase shift ${\varepsilon _b}$ caused by color crosstalk in the red channel is:

$$\begin{aligned} \tan ({\varepsilon _b}) &= \tan ({\theta _b} - {\delta _b})\\ &= \frac{{\tan ({\theta _b}) - \tan ({\delta _b})}}{{1 + \tan ({\theta _b})\tan ({\delta _b})}}\\ &= \frac{{\frac{{ - \sqrt 3 ({{C_{31}} - {C_{33}}} )}}{{2{C_{32}} - {C_{31}} - {C_{33}}}} + \sqrt 3 }}{{1 + \frac{{3({{C_{31}} - {C_{33}}} )}}{{2{C_{32}} - {C_{31}} - {C_{33}}}}}}\\ &= \frac{{ - 2\sqrt 3 {C_{31}} + 2\sqrt 3 {C_{32}}}}{{2{C_{31}} + 2{C_{32}} - 4{C_{33}}}} \end{aligned}$$

By substituting the data in the Table 1 into the Eq.  (27), the numerator on the right side of the formula is positive, and the denominator is negative, hence $\tan {\varepsilon _b} < 0$, which proves that ${\varepsilon _b} < 0$.

Funding

National Natural Science Foundation of China (51835007); Natural Science Foundation of Tianjin City (21JCZDJC00760); Tianjin "Project+Team" Key Training Project (XC202054); Tianjin Graduate Research and Innovation Project (2022SKY191, 2022SKYZ004); Graduate Research and Innovation Project in Tianjin University of Technology (YJ2227, YJ2228, YJ2230).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. R. Yu, H. Yu, W. Sun, et al., “Color phase order coding and interleaved phase unwrapping for three-dimensional shape measurement with few projected pattern,” Opt. Laser Technol. 168, 109842 (2024). [CrossRef]  

2. H. Wu, Y. Cao, H. An, et al., “A novel phase-shifting profilometry to realize temporal phase unwrapping simultaneously with the least fringe patterns,” Optics and Lasers in Engineering 153, 107004 (2022). [CrossRef]  

3. Y. Lee, Y. Mao, and Z. Chen, “Fast combined-frequency phase extraction for phase shifting profilometry,” Opt. Express 30(25), 45288–45300 (2022). [CrossRef]  

4. A. G. Marrugo, F. Gao, and S. Zhang, “State-of-the-art active optical techniques for three-dimensional surface metrology: a review [Invited],” J. Opt. Soc. Am. A 37(9), B60–B77 (2020). [CrossRef]  

5. R. Juarez-Salazar, J. Martinez-Laguna, and V. H. Diaz-Ramirez, “Multi-demodulation phase-shifting and intensity pattern projection profilometry,” Optics and Lasers in Engineering 129, 106085 (2020). [CrossRef]  

6. S. Feng, C. Zuo, Y. Hu, et al., “Deep-learning-based fringe-pattern analysis with uncertainty estimation,” Optica 8(12), 1507–1510 (2021). [CrossRef]  

7. J. Qian, T. Tao, S. Feng, et al., “Motion-artifact-free dynamic 3D shape measurement with hybrid Fourier-transform phase-shifting profilometry,” Opt. Express 27(3), 2713–2731 (2019). [CrossRef]  

8. T. Li, Y. Dong, and X. Wang, “Fourier transform profilometry using single-pixel detection based on two-dimensional discrete cosine transform,” Opt. Laser Technol. 156, 108570 (2022). [CrossRef]  

9. Q. Zhu, H. Zhao, and Z. Zhao, “Research on iterative decoupling algorithm in color fringe projection profilometry,” Opt. Laser Technol. 164, 109541 (2023). [CrossRef]  

10. Y. Wan, Y. Cao, C. Chen, et al., “Active phase error suppression for color phase-shifting fringe projection based on hue pre-correction,” Opt. Laser Technol. 118, 102–108 (2019). [CrossRef]  

11. P. Omidi, M. Najiminaini, M. Diop, et al., “Single-shot 4-step phase-shifting multispectral fringe projection profilometry,” Opt. Express 29(18), 27975–27988 (2021). [CrossRef]  

12. C. Chen, Z. Zhang, F. Gao, et al., “A flexible lateral chromatic aberration correction method for a color-encoded fringe projection system,” Optics and Lasers in Engineering 162, 107425 (2023). [CrossRef]  

13. P. S. Huang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng 38(6), 1065–1071 (1999). [CrossRef]  

14. J. S. Pan and P. S. Huang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng 45(1), 013602 (2006). [CrossRef]  

15. Z. H. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in colour fringe projection,” J. Opt. A: Pure Appl. Opt. 9(6), S81–S86 (2007). [CrossRef]  

16. Y. Wang, L. Liu, J. Wu, et al., “Hilbert transform-based crosstalk compensation for color fringe projection profilometry,” Opt. Lett. 45(8), 2199–2202 (2020). [CrossRef]  

17. T. Hoang, B. Pan, D. Nguyen, et al., “Generic gamma correction for accuracy enhancement in fringe-projection profilometry,” Opt. Lett. 35(12), 1992–1994 (2010). [CrossRef]  

18. X. Yu, Y. Liu, N. Liu, et al., “Flexible gamma calculation algorithm based on probability distribution function in digital fringe projection system,” Opt. Express 27(22), 32047–320057 (2019). [CrossRef]  

19. Y. Hu, J. Xi, J. Chicharo, et al., “Blind color isolation for color-channel-based fringe pattern profilometry using digital projection,” J. Opt. Soc. Am. A 24(8), 2372–2382 (2007). [CrossRef]  

20. J. Antonio Quiroga and M. Servin, “Isotropic n-dimensional fringe pattern normalization,” Opt. Commun. 224(4-6), 221–227 (2003). [CrossRef]  

21. S. Feng, C. Zuo, L. Zhang, et al., “Calibration of fringe projection profilometry: A comparative review,” Optics and Lasers in Engineering 143, 106622 (2021). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1. Schematic diagram of color phase-shifting fringe projection profilometry.
Fig. 2.
Fig. 2. Flowchart of color crosstalk compensation method based on phase correction matrix.
Fig. 3.
Fig. 3. Phase errors of the simulated crosstalk pattern. (a) phase errors of the simulated crosstalk pattern, (b) phase errors after intensity normalization, (c) phase errors by using the proposed phase correction matrix and (d) one cross-section of the phase errors.
Fig. 4.
Fig. 4. Phase errors of the deformed crosstalk pattern without noise and with Gaussian noise (σ=0.0001 and 0.001). (a)(e)(i) phase errors of the deformed crosstalk pattern without correction. (b)(f)(j) phase errors after intensity normalization. (c)(g)(k) phase errors by using the proposed phase correction matrix. (d)(h)(l) one cross-section of the phase errors.
Fig. 5.
Fig. 5. Calibrated CPSP 3D Measurement System.
Fig. 6.
Fig. 6. Captured color pattern on the white plate and the one cross-section of the phase errors (a) captured color pattern, (b) the cross-section of the phase errors before compensation, (c) the cross-section of the phase errors after compensation.
Fig. 7.
Fig. 7. Accuracy analysis experiments of the 3D measurement system. (a) two ceramic standard spheres to be measured. (b) a stepped workpiece. (c-e) 3D reconstruction results of spheres A, B and stepped workpiece. (f-h) plots of the difference distribution between the reconstructed and fitted data of spheres A, B and stepped surface 3. (i-j) quantitative histograms of error plots (f-g). (k) error box line plots of surface 1, surface 2, surface 3 and surface 4.
Fig. 8.
Fig. 8. Unwrapped phase maps (a) original crosstalk fringe, (b) crosstalk matrix method, (c) Hilbert transform method, (d) our method, and (e) ∼(h) are phase error maps of (a) ∼ (d).
Fig. 9.
Fig. 9. (a) Collected deformed fringe pattern, (b)∼(e) phase error of traditional method, crosstalk matrix method, Hilbert transform method and our method respectively.
Fig. 10.
Fig. 10. 3D reconstruction of (a) original crosstalk fringe, (b)crosstalk matrix method, (c) Hilbert transform method,(d) the method proposed in this paper, (e) is a cross section of (a) ∼ (d)
Fig. 11.
Fig. 11. MV-CS050-60UC Quantum Efficiency

Tables (1)

Tables Icon

Table 1. Relationship of Crosstalk Matrix Elements.

Equations (27)

Equations on this page are rendered with MathJax. Learn more.

I n ( x , y ) = A 0 ( x , y ) + B 0 ( x , y ) cos [ ϕ ( x , y ) + δ n ] ,
ϕ ( x , y ) = tan 1 [ 3 ( I r ( x , y ) I b ( x , y ) ) 2 I g ( x , y ) I r ( x , y ) I b ( x , y ) ] ,
[ I r ( x , y ) I g ( x , y ) I b ( x , y ) ] = C [ [ I o r ( x , y ) ] γ R [ I o g ( x , y ) ] γ G [ I o b ( x , y ) ] γ B ] = [ C 11 C 12 C 13 C 21 C 22 C 23 C 31 C 32 C 33 ] [ [ I o r ( x , y ) ] γ R [ I o g ( x , y ) ] γ G [ I o b ( x , y ) ] γ B ] ,
I n ( x , y ) = A n ( x , y ) + B n ( x , y ) cos [ ϕ ( x , y ) + δ n + ε n ] ,
A n = A 0 ( C n , 1 + C n , 2 + C n , 3 ) ,
B n = B 0 1 2 [ ( C n , 1 C n , 2 ) 2 + ( C n , 2 C n , 3 ) 2 + ( C n , 1 C n , 3 ) 2 ] ,
tan ( θ n ) = 3 ( C n , 1 C n , 3 ) 2 C n , 2 C n , 1 C n , 3 ,
I n H P ( x , y ) = B n ( x , y ) cos [ ϕ ( x , y ) + θ n ] ,
I n H T ( x , y ) = B n ( x , y ) sin [ ϕ ( x , y ) + θ n ] ,
I ^ n ( x , y ) = I n H P ( x , y ) ( I n H P ( x , y ) ) 2 + ( I n H T ( x , y ) ) 2 = cos [ ϕ ( x , y ) + θ n ] ,
co s 2 θ n 2 I ^ n ( x , y ) cos ϕ ( x , y ) cos θ n + I ^ n 2 ( x , y ) sin 2 ϕ ( x , y ) = 0 ,
cos θ n = I ^ n ( x , y ) cos ϕ ( x , y ) ± ( 1 cos 2 ϕ ( x , y ) ) ( 1 I ^ n 2 ( x , y ) ) ,
I ( x , y ) = A 0 + B 0 cos ϕ ( x , y ) ,
I N ( x , y ) = cos ϕ ( x , y ) ,
[ I ~ r I ~ g I ~ b ] = A [ I ^ r I ^ g I ^ b ] = [ 1 a 12 a 13 a 21 1 a 23 a 31 a 32 1 ] [ I ^ r I ^ g I ^ b ]
I ~ r ( x , y ) = I ^ r ( x , y ) + a 12 I ^ g ( x , y ) + a 13 I ^ b ( x , y ) ,
cos [ ϕ ( x , y ) + δ r ) ] = cos [ ϕ ( x , y ) + θ r ] + a 12 cos [ ϕ ( x , y ) + θ g ] + a 13 cos [ ϕ ( x , y ) + θ b ] ,
h ( a 12 , a 13 ) : { cos θ r + a 12 cos θ g + a 13 cos θ b = 1 2 sin θ r + a 12 sin θ g + a 13 sin θ b = 3 2
C = [ 1.00 0.36 0.08 0.15 1.00 0.20 0.06 0.25 1.00 ] ,
A = [ 1.00 0.2381 0.1289 0.0301 1.00 0.0246 0.0731 0.1589 1.00 ] ,
R ( x , y ) = 0.5 + 0.5 × G ( x , y ) / max [ G ( x , y ) ] ,
A 1 = [ 1 0.1631 0.1100 0.0481 1 0.0641 0.0684 0.1639 1 ] ,
C = [ C 11 C 12 C 13 C 21 C 22 C 23 C 31 C 32 C 33 ]
tan ( θ n ) = 3 ( C n , 1 C n , 3 ) 2 C n , 2 C n , 1 C n , 3
tan ( ε r ) = tan ( θ r δ r ) = tan ( θ r ) tan ( δ r ) 1 + tan ( θ r ) tan ( δ r ) = 3 ( C 11 C 13 ) 2 C 12 C 11 C 13 3 1 + 3 ( C 11 C 13 ) 2 C 12 C 11 C 13 = 2 3 C 12 + 2 3 C 13 4 C 11 + 2 C 12 + 2 C 13
tan ( ε g ) = tan ( θ g δ g ) = tan ( θ g ) tan ( δ g ) 1 + tan ( θ g ) tan ( δ g ) = 3 ( C 21 C 23 ) 2 C 22 C 21 C 23
tan ( ε b ) = tan ( θ b δ b ) = tan ( θ b ) tan ( δ b ) 1 + tan ( θ b ) tan ( δ b ) = 3 ( C 31 C 33 ) 2 C 32 C 31 C 33 + 3 1 + 3 ( C 31 C 33 ) 2 C 32 C 31 C 33 = 2 3 C 31 + 2 3 C 32 2 C 31 + 2 C 32 4 C 33
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.