Object motion can introduce phase error and thus measurement error for phase-shifting profilometry. This paper proposes a generic motion error compensation method based on our finding that the dominant motion-introduced phase error doubles the frequency of the projected fringe frequency, and the Hilbert transform shifts the phase of a fringe pattern by π/2. We apply a Hilbert transform to phase-shifted fringe patterns to generate another set of fringe patterns, calculate one phase map using the original fringe patterns and another phase map using Hilbert transformed fringe patterns, and then use the average of these two phase maps for three-dimensional reconstruction. Both simulation and experiments demonstrated that the proposed method can substantially reduce motion-introduced measurement error.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Three-dimensional (3D) shape measurement using digital fringe projection (DFP) technique has been exhaustively studied and widely applied due to its simple setup, high-speed and high-resolution measurement capabilities [1–3].
Conventional DFP technique typically projects 8-bit sinusoidal fringe patterns, and its measurement speed is limited to the maximum refresh rate of the projector, typically 120 Hz. Therefore, it is challenging for the conventional DFP technique to measure rapidly moving objects if the object moves too much while acquiring the required phase-shifted fringe patterns for phase retrieval. To overcome this limitation, the digital binary defocusing techniques have been developed to generate quasi-sinusoidal fringes with 1-bit binary patterns through projector lens defocusing [4,5], and the advanced digital-light-processing (DLP) projection platform allows researchers to achieve speed breakthroughs [6–8]. However, the binary defocusing method still has measurement error if the object moves too quickly. Phase-shifting profilometry works well with the assumption that the object stays quasi-static when the required phase-shifted fringe patterns are captured. Therefore, any motion of the object could introduce phase error and thus measurement error, and we refer this type of error as motion-introduced phase error.
The motion-introduced phase error has the similarity to the phase-shift error introduced by the interferometry systems when the phase shift cannot be precisely generated. However, the motion-introduced phase error is more complex because the motion-introduced phase error might not be homogeneous but the phase-shift error in the interferometry is homogeneous. This is because for the interferometry system, the major phase-shift error is introduced by the displacement error of the mirror driven by a piezoelectric device, and the phase-shift error is overall homogeneous (e.g., remains the same across the entire measurement surface). However, the phase error introduced by motion could be nonhomogeneous because the motion of a dynamically deformable object varies from one point to another.
To eliminate the motion introduced error, Lu et al.  proposed a method to reduce error caused by the planar motion parallel to the imaging plane. By placing a few markers on the object and analyzing the movement of markers, the rigid-body motion of an object can be estimated. Later, they improved their method to handle error induced by translation in the direction of object height . The motion is estimated using the arbitrary phase shift estimation method developed by Wang and Han . However, this method is limited to homogeneous background intensity and modulation amplitude. Feng et al.  proposed to solve the nonhomogeneous motion artifact problem by segmenting objects with different rigid shifts. However, this method still assumes that the phase-shift error within a single segmented object is homogeneous, thus it may not work well for dynamically deformable objects where the phase-shift error varies from one pixel to another. Cong et al.  proposed a Fourier-assisted approach to correct the phase-shift error by differentiating the phase maps of two successive fringe images. However, the accuracy is limited due to the use of Fourier Transform Profilometry (FTP) for phase shift estimation. Liu et al.  developed a method to find the phase shift more precisely with an iterative process based on the assumption that the motion is uniform between two adjacent 3D frames. However, such a method requires the acquisition of another set of fringe patterns for motion estimation, and thus such a method may not work well if the object moves too rapidly when the motion cannot be precisely estimated.
We propose a motion-induced phase error compensation method based on our finding that the dominant motion introduced phase error doubles the frequency of the projected fringe frequency, and Hilbert transform shifts the phase of a fringe pattern by π/2. Our proposed method includes four major steps: 1) apply Hilbert transform to the phase-shifted fringe patterns to generate another set of fringe patterns; 2) calculate one phase map ϕ using the original fringe patterns and another phase map ϕH using Hilbert transformed fringe patterns; 3) generate the final phase map ϕf by averaging ϕ and ϕH, i.e., ϕf = (ϕ + ϕH)/2; and 4) reconstruct 3D shape using the averaged phase map ϕf. This method can substantially reduce motion-introduced phase error for objects with rigid uniform motion as well as for dynamically deformable objects with non-uniform motion.
Section 2 explains the principle of the proposed method. Section 3 shows experimental results to verify the performance of the proposed method; and Sec. 4 summarizes the paper.
2.1. Multi-step phase-shifting algorithm
Phase-shifting methods are widely used in optical metrology because of their speed and accuracy . The intensity distributions of the n–th fringe for an N–step phase-shifting algorithm with a phase shift of δn can be described as,
2.2. Motion-introduced phase error analysis
The phase-shifting algorithm could determine accurate phase information ϕ(x, y) if and only if the phase shift δn is precisely known. For DFP systems, the phase-shift can be accurately controlled with digital projectors. However, due to the existence of motion, the actual phase shift δ′n could be different from the projected value δn,
Figure 1 illustrates the phase-shift error caused by the motion of a deformable object. For an image point C1, the corresponding surface point changes from S1 to S̄1 if the object moves, thus the phase-shift error for the adjacent fringes will be ΔΦ(P̄1P1). While for another image point C2, similarly we can find the phase-shift error will be ΔΦ(P̄2P2). Apparently, these two values could be different for different image points, and thus the phase-shift errors caused by object motion is non-homogeneous.
For the N-step phase-shifting algorithm, the phase shift error ∊n(x, y) will result in pattern distortion and final phase calculation as follows,
Therefore, we can obtain the motion-introduced phase error by
From Eq. 10, it can be seen that the motion-introduced phase error Δϕ highly correlates to the real phase ϕ. Since three-step and four-step phase-shifting algorithms are more extensively used for high-speed applications, we provide detailed analysis of phase error for these two special cases.
For a three-step phase-shifting algorithm, the fringe patterns, wrapped phase and phase error can be respectively described as,Eq. 13 can be further approximated as,
Similarly, we deduced the phase error model for the four-step phase-shifting algorithm as follows. The fringe patterns, wrapped phase and phase error can be respectively described as,
Once again, because ∊ is very small, we will have,
Therefore, Equation 22 becomes
To better understand the characteristics of the motion-introduced phase error, simulations were carried out to test the phase-shift errors caused by both uniform and non-uniform motion. First, we tested three-step phase-shifting algorithm under uniform motion with the phase-shift errors ∊ ∈ [−0.1, 0.1] rad. And the resultant phase error plots for one period of pattern are shown in Fig. 2(a). The simulation results indicate that the dominant phase error introduced by uniform-motion approximately doubles the frequency of the projected fringe patterns. Similarly, we simulated the non-uniform motion by setting the phase-shift errors as ∊0 = 0, ∊1 = ∊, ∊2 = 3×∊. This means the acceleration rate is set to be the same with the moving speed. Figure 2(b) shows the results. It can be found that the phase error also shares the same characteristics: primarily doubling the frequency of the fringe pattern, albeit the amplitude is relatively larger. Similar simulations were also conducted for four-step phase-shifting algorithm, with the results shown in Fig. 3, from which we can notice that the phase error also doubles the fringe frequency.
Based on this finding, we came to the idea that if we shift the original phase map by 1/4 period (or π/2), and then we can use the shifted phase map to compensate the motion-induced error by averaging it with the original phase map. This method can simultaneously tackle error caused by the uniform and non-uniform motions. Meanwhile, this method does not require additional pattern acquisition, nor motion estimation, which is especially important for high-speed applications.
2.3. Phase error compensation using Hilbert transform
Hilbert transform is widely used in signal processing, and mathematically it is a linear operator that takes a function, μ(t) of a real variable and produces another function of a real variable H(μ)(t) by convolution as,
Hilbert transform actually is a multiplier operator in frequency domain,
This indicates that Hilbert transform has the effect of shifting the phase of negative frequency components by π/2 and the phase of positive frequency components by −π/2. For a real signal cos(ωt), Hilbert transform actually results in cos(ωt − π/2) = sin(ωt). Therefore, Hilbert transform provides a means to shift the phase by π/2, which can address motion-introduced phase error problems.
Applying Hilbert transform to Eq. 1 leads to
To test the performance of this proposed method, we first evaluated the phase error for both uniform and non-uniform motion cases for three-step phase-shifting algorithm. In the simulations, we set the uniform motion induced phase-shift errors as ∊0 = 0 rad, ∊1 = 0.1 rad, ∊2 = 0.2 rad. The phase error plots of one period are shown in Fig. 4(a). It can be seen that the phase error ϕe obtained from the original phase-shifted patterns, and the phase error obtained from Hilbert transformed fringe patterns indeed have different tendencies. The averaged phase error is significantly reduced: from root-mean-square(rms) of 0.042 rad to 0.0012 rad. Figure 4(c) shows the phase rms errors when the phase-shift error ∊1 varies from −0.1 rad to 0.1 rad and ∊2 = 2∊1. It can be seen that the phase rms error increases approximately linearly with the phase-shift error, and the proposed method can effectively reduce the phase error caused by uniform motion. Figure 4(b) shows simulation results for non-uniform motion when setting the phase-shift error as ∊0 = 0 rad, ∊1 = 0.1 rad, ∊2 = 3∊1. It can be found the proposed method can effectively reduce the phase rms error from 0.066 rad to 0.003 rad. Figure 4(d) plots the results when ∊1 varies from −0.1 rad to 0.1 rad and ∊2 = 3∊1. Once again, the proposed method can drastically reduce phase error caused by non-uniform motion.
Besides, similar simulations were also conducted to test the proposed method for four-step phase-shifting algorithm with the results shown in Fig. 5. As shown in Fig. 5(a), the proposed method can reduce the phase rms error from 0.035 rad to 0.0009 rad for uniform motion. And for nonuniform motion as shown in Fig. 5(b), the phase rms error decreases from 0.072 rad to 0.0036 rad. Figures 5(c) and 5(d) show the compensation results with different phase-shifting errors for uniform and nonuniform motions, which obviously indicate the effectiveness of the proposed compensation method.
We further evaluated the performance of our proposed method through experiments. The system includes a CMOS camera (model: PointGrey Grasshopper3 GS3-U3-23S6M) that is attached with a 8 mm focal length lens (model: Computar M0814-MP2), and a DLP projection developmental kit (model: DLP Lightcrafter 4500). The camera resolution was set as 640 × 480 pixels while the projector resolution was 912 × 1140 pixels. The system was calibrated using the method described by Li et al. . The projector and the camera were synchronized with a speed of 120 Hz, and a three-frequency temporal phase-unwrapping algorithm was adopted for absolute phase retrieval.
First, a moving sphere with a diameter of 79.2 mm at the speed of approximately 80 mm/s was used to quantitatively evaluate the performance of the proposed method for both three-step and four-step phase-shifting algorithms. Figure 6 shows the results for three-step phase-shifting algorithm. Figure 6(a) shows the 3D result obtained from original fringe patterns where the motion-introduced measurement error (i.e. vertical stripes) is obvious. Figure 6(b) shows the 3D result obtained from Hilbert-transformed fringe patterns, and the motion-introduced measurement error is also obvious. Figure 6(c) shows the 3D result obtained from our proposed method, clearly demonstrating that measurement error is greatly reduced (i.e., smoother surface). To quantitatively evaluate the improvements, we compared the measurement results with an ideal sphere, and the corresponding error maps are obtained as shown in Figs. 6(d)–6(f). For the result obtained from the original fringe patterns, the mean measurement error is 0.172 mm and the standard deviation is 0.124 mm. Similarly, Hilbert transformed fringe patterns gives nearly the same error: the mean error of 0.160 mm and the standard deviation of 0.116 mm. In contrast, the result from our proposed method reduces the mean error to 0.038 mm and the standard deviation to 0.032 mm.
We then did the same experiments using four-step phase-shifting algorithm with the results shown in Fig. 7. Figures 7(a)–7(c) respectively show the 3D reconstructed results for original fringe patterns, Hilbert transformed patterns and the proposed method, while Figs. 7(d)–7(f) show the corresponding error maps. For the result obtained from the original fringe patterns, the mean measurement error is 0.118 mm and the standard deviation is 0.099 mm. And Hilbert transformed fringe patterns gives the close error: the mean error of 0.104 mm and the standard deviation of 0.090 mm. In contrast, the result from our proposed method reduces the mean error to 0.031 mm and the standard deviation to 0.027 mm.
Besides, a moving vase with more complex structure was also measured using three-step phase-shifting algorithm, with the texture shown in Fig. 8(a). Figure 8(b) shows the 3D result obtained from the original phase-shifted fringe patterns, depicting clear motion-introduced error. Figure 8(c) shows the 3D result obtained from our proposed method, and most vertical stripes caused by motion are no longer obvious. Even though Hilbert transform could theoretically smooth out the complex structure due to the filtering effect, our experiments demonstrate that the filtering effect is not obvious even for complex surface geometry like the one shown here.
Lastly, we evaluated our proposed method by measuring dynamically deformable objects such as human facial expressions. Figure 9(a) shows the 3D result from the original fringe patterns, while Fig. 9(b) shows the 3D result obtained from Hilbert transformed fringe patterns. Again both 3D results shows severe motion introduced error. Figure 9(c) shows the result obtained from our proposed method, demonstrating that our proposed method can effectively alleviate the motion introduced error and greatly improve measurement quality.
In addition, we noticed that our proposed method can also significantly improve the quality of texture (e.g., A (x, y) in Eq. 1). Figure 10(a) shows the recovered texture using the original fringe patterns. Due to motion, obvious vertical stripes present on the image. After correcting the phase with the proposed method, the texture information can also be greatly improved, as shown in Fig. 10(b). These experimental data clearly demonstrated the effectiveness of our proposed method even for dynamically deformable objects with complex surface geometry.
In our research, we have an assumption that the camera speed is not significantly slower than the motion, thus the mismatch problem between adjacent frames is not serious. Actually, our proposed method also works for high-speed motions if we adopt the binary defocusing technique with a high-speed structure light system.
This paper has presented a motion-induced error compensation algorithm based on Hilbert transform, without requiring the acquisition of additional images or estimation of unknown phase shifts. We successfully demonstrated that the proposed method can drastically reduce measurement error introduced by homogeneous and non-homogeneous motions. Since this method requires no extra fringe pattern, it is suitable for high-speed applications.
National Institute of Justice (NIJ) (2016-DN-BX-0189); National Science Foundation (NSF) (IIS-1637961); National Natural Science Foundation of China (NSFC) (61603360).
We thank the National Institute of Justice (NIJ), and the National Science Foundation (NSF), and the National Natural Science Foundation of China (NSFC). Views expressed in this paper are those of the authors and not necessarily those of the NIJ, NSF, or NSFC.
1. S. Gorthi and P. Rastogi, “Fringe projection techniques: Whither we are?” Opt. Laser. Eng. 48, 133–140 (2010). [CrossRef]
2. S. Zhang, “Recent progresses on real-time 3-d shape measurement using digital fringe projection techniques,” Opt. Laser Eng. 48, 149–158 (2010). [CrossRef]
3. X. Su and Q. Zhang, “Dynamic 3-d shape measurement method: A review,” Opt. Laser. Eng 48, 191–204 (2010). [CrossRef]
5. C. Zuo, Q. Chen, S. Feng, F. Feng, G. Gu, and X. Sui, “Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing,” Appl. Opt. 51, 4477–4490 (2012). [CrossRef] [PubMed]
6. B. Li, Y. Wang, J. Dai, W. Lohry, and S. Zhang, “Some recent advances on superfast 3d shape measurement with digital binary defocusing techniques,” Opt. Laser Eng. 54, 236–246 (2014). [CrossRef]
7. Y. Wang and S. Zhang, “Superfast multifrequency phase-shifting technique with optimal pulse width modulation,” Opt. Express 19, 5143–5148 (2011).
8. J. Zhu, P. Zhou, X. Su, and Z. You, “Accurate and fast 3d surface measurement with temporal-spatial binary encoding structured illumination,” Opt. Express 24, 28549–28560 (2016). [CrossRef] [PubMed]
9. L. Lu, J. Xi, Y. Yu, and Q. Guo, “New approach to improve the accuracy of 3-d shape measurement of moving object using phase shifting profilometry,” Opt. Express 21, 30610–30622 (2013). [CrossRef]
12. S. Feng, C. Zuo, T. Tao, Y. Hu, M. Zhang, Q. Chen, and G. Gu, “Robust dynamic 3-d measurements with motion-compensated phase-shifting profilometry,” Opt. Laser. Eng. 103, 127–138 (2018). [CrossRef]
13. P. Cong, Z. Xiong, Y. Zhang, S. Zhao, and F. Wu, “Accurate dynamic 3d sensing with fourier-assisted phase shifting,” IEEE J. Sel. Top. Signal Process. 9, 396–408 (2015). [CrossRef]
15. D. Malacara, ed., Optical Shop Testing, 3rd ed. (John Wiley and Sons, New York, 2007). [CrossRef]