Phase-shifting profilometry combined with Gray-code patterns projection has been widely used for 3D measurement. In this technique, a phase-shifting algorithm is used to calculate the wrapped phase, and a set of Gray-code binary patterns is used to determine the unwrapped phase. In the real measurement, the captured Gray-code patterns are no longer binary, resulting in phase unwrapping errors at a large number of erroneous pixels. Although this problem has been attended and well resolved by a few methods, it remains challenging when a measured object has step-heights and the captured patterns contain invalid pixels. To effectively remove unwrapping errors and simultaneously preserve step-heights, in this paper, an effective method using an adaptive median filter is proposed. Both simulations and experiments can demonstrate its effectiveness.
© 2017 Optical Society of America
Phase-shifting profilometry (PSP) combined with Gray-code patterns projection has been widely used for 3D measurement due to its merits of flexibility, high-speed and high-resolution . Traditionally, phase-shifting sinusoidal patterns are projected by a projector, and a phase-shifting algorithm is used to calculate the phase . The gamma distortion of the projector, more or less deviates the sinusoidal patterns from its ideal distributions . To eliminate the influence of this distortion and simultaneously obtain high-speed 3D measurement, binary patterns are often used to generate sinusoidal patterns with projector defocusing [3, 4]. The phase calculated from fringe patterns is wrapped in the interval and contains phase jumps [5, 6]. A phase unwrapping process is necessary to determine the unwrapped phase . For optical methods such as interferometry, tomography and ESPI, a wrapped phase filtering is essential before phase unwrapping because of the difficulty brought by the noise [7, 8]. Since incoherent white light is used, influence of noise is a trivial problem in phase unwrapping [1, 7]. Phase unwrapping algorithms can be classified into spatial and temporal categories. The spatial category often fails when a complex surface introduces more than phase changes between two neighbouring pixels . The temporal category mainly includes spatial-code, multi-frequency, and Gray-code methods . The spatial-code method relies on the neighborhood pixel information, and only works for the surface locally smooth . The multi-frequency method using multiple-wavelengths is effective but needs many intermediate temporal phase maps [12, 13]. In [14, 15], Servin et al. describe a method using two-wavelengths to keep within the limits imposed by the Nyquist temporal sampling rate. The multi-frequency method has been used for optical interferometry [16, 17] and fringe projection profilometry . Because phase unwrapping methods using multiple wavelengths require all the fringe patterns to be sinusoidal, it is difficult for binary patterns with projector defocusing . A more practical temporal phase unwrapping method uses the so called Gray-code patterns, which are projected by the projector and then captured by the camera [19–23]. A unique codeword is later calculated pixel-wise to determine the unwrapped phase. To assist the phase unwrapping, the unique codeword is designed that its binary changes align with the phase jumps . In the real measurement, the Gray-code patterns captured by a camera are no longer binary, and they have to be binarized before phase unwrapping . Because of the defocused projector , the discrete sampling camera  and the blooming effect [24, 25], the binary changes of the binarized Gray-code pattern may shift left or right by one or two pixels to the phase jumps [3, 20]. This binarization corruption of Gray-code patterns destroys the perfect alignment between the codeword changes and the phase jumps. Consequently, the codeword could be wrong near the phase jumps, resulting in erroneous pixels with unwrapping errors. Such errors should be removed before the 3D reconstruction .
This unwrapping error problem has been attended and well resolved by a few methods. These methods become less successful when the measured object has step-heights [3, 20–23]. The self-correction method is difficult to remove unwrapping errors in the step-heights . The surface smoothness method may pollute the step-heights . Furthermore, the modified Gray-code patterns are also used [21, 22]. The median filter is applied to remove unwrapping errors [26, 27]. Besides the step-heights, invalid pixels caused by systems noise or shadows should also be considered [28, 29]. When the captured patterns contain no invalid pixels, a small-size of 1D median filter works perfectly . However, it does not work for erroneous pixels adjacent to invalid pixels. A large-size of 2D median filter is then introduced , which works well but also may pollute step-heights. To effectively remove unwrapping errors with the existence of both step-heights and invalid pixels, an effective method using an adaptive median filter is proposed, and its performance is demonstrated by both simulations and experiments.
The rest of the paper is organized as follows. Section 2 introduces the principle of PSP combined with Gray-code patterns projection. Section 3 illustrates the unwrapping error removal by using the median filter. Section 4 presents our adaptive median filter. Section 5 shows the 3D reconstruction. Section 6 summarizes the paper.
2. PSP combined with Gray-code patterns
The PSP uses the following sinusoidal patterns
The phase is calculated by using a least-squares algorithm ,
In the projector defocusing technique , the sinusoidal patterns in Eq. (1) are generated by projecting squared binary patterns with a period of pixels ( pixels are white and the other pixels are black). The phase shifts is realized by shifting the binary pattern with the amount of
In order to determine the absolute phase, a set of Gray-code patterns are used to generate code words for phase unwrapping as follows ,
In Fig. 1, is selected for illustration. From top to bottom of Fig. 1, three Gray-code, , and , the corresponding codeword , the wrapped and unwrapped phases are shown. Note that to keep the unique relationship between and , the coding patterns perfectly align with the phase jump.
As mentioned in Section 1, the binarization corruption of Gray-code patterns leads to erroneous pixels with unwrapping errors . In the unwrapped phase, the erroneous pixels are sparsely distributed near the phase jumps, and can be deemed as impulsive noise, which can be effectively removed by a simple median filter [30, 31] as follows26] and large  filter sizes have been used. The median filter takes the median value of neighboring pixels as the output of the current pixel, which means that the output of the current pixel is actually very likely another pixel’s value. In other words, the unwrapping result is not congruent: the difference between the wrapped phase and unwrapped phase should be just multiple of , but it is not. In the context of precision measurement, the output of the current pixel is expected to be indeed its own value. Interestingly, with the median filtering result, the congruent result can be easily obtained as follows ,
3. Unwrapping error removal by using traditional median filters
3.1 Analysis of erroneous pixels
An experiment is conducted to expose the unwrapping error problem. A five-step phase-shifting algorithm is used, for which five squared binary patterns are generated and projected with projector defocusing . The measurement system uses a TI DLP Discovery 4100 projector with a resolution of . Because vertical fringe patterns with a fringe period are used, the squared binary patterns should be designed with 45.6 (i.e.,) fringe periods. Accordingly, a set of six Gray-code binary patterns should be designed . A Basler acA2000CMOS-340kc camera combined with one Ricoh FL-CC0614A-2M lens is used to capture patterns with a resolution of . A white flat board as a measured object is placed at the distance of 60 cm from the camera. The inner pixels of the captured patterns without invalid pixels are selected. Figure 2(a) shows one of the captured sinusoidal patterns, which are obtained by defocusing squared binary patterns and used for the wrapped phase calculation. Figure 2(b) shows one of the captured Gray-code patterns where undesired but unavoidable slight defocusing also presents and thus binarization is applied. By using Eq. (3), the wrapped phase is calculated. By using Eqs. (5) and (6), the unwrapped phase is determined. By using Eqs. (7) and (8), the median filter is applied to the unwrapped phase, and the congruent phase is obtained. The unwrapped phase and the congruent phase are shown in Figs. 2(c) and 2(d), respectively. In the unwrapped phase, there are obvious wrinkles related to unwrapping errors, while the obtained congruent phase is smooth and correct without unwrapping errors.
The unwrapping errors can be easily obtained by subtracting Fig. 2(d) from Fig. 2(c). For any erroneous pixel, its unwrapping error is , where is an integer number. In our system, these erroneous pixels can be classified into two kinds: (1) one isolated erroneous pixel; (2) two adjacent erroneous pixels. In the unwrapped phase along the horizontal direction, when the binary change shifts one pixel from its nearest phase jump, one isolated erroneous pixel is generated. The unwrapping error of one isolated erroneous pixel of is illustrated in Fig. 3(a). Among five pixels, only is erroneous while others are error-free. The median filter can remove the unwrapping error of . Similarly, when the binary change shifts two pixels from its nearest phase jump, two adjacent erroneous pixels are generated as illustrated in Fig. 3(b). At this time, among six pixels, and are erroneous while others are error-free. The median filter is necessary to remove the unwrapping errors of and . In the experiment mentioned above, at the distance of 60 cm, there are 5321 isolated erroneous pixels and 22 adjacent erroneous pixels. The flat white board is placed on a translation stage, which is moved in 10 cm increment from the distance of 40 cm to that of 90 cm. The numbers of erroneous pixels at different distances are provided in Table 1. The number of isolated erroneous pixels is much larger than that of the two adjacent erroneous pixels. Both of them can be treated as impulse noises. By using the median filter, all the impulse noises can be removed at different distances. For simplicity, the erroneous pixels are denoted by EPs as follows.
3.2 Performance of traditional median filters with different sizes
In the real measurement, very often the measured object has step-heights and the captured patterns contain invalid pixels [28, 29]. The co-existence of step-heights and invalid pixels makes the median filter size determination awkward: a small-size is not effective in removing the impulse noise, while a large-size may pollute the step-heights. This awkwardness can be clearly shown through the following simulation.
A continuous phase is first generated and shown in Fig. 4(a). The phase value is increased linearly from to along the horizontal direction. That is to say, each row includes 32 fringe periods, and each period includes 16 pixels. Four circular step-phases with values of , , and , respectively, are generated to simulate four objects , which have a radius of 40 pixels and centered at (150, 150), (150, 350), (350, 150) and (350, 350), respectively. Invalid pixels are generated in four circles with a radius of 40 pixels and centered at (250, 150), (250, 350), (350, 250) and (150, 250), respectively. The phase value of the invalid pixels is assigned with (i.e., not in the range of ). The resulting synthesized phase, with both step-phases and invalid pixels, is shown in Fig. 4(b), which will be used as the ideal phase. To simulate EPs with unwrapping errors, some pixels close to phase jumps, step-heights and invalid pixels are randomly selected to give unwrapping errors of and shown in Fig. 4(c). Both one isolated and two adjacent erroneous pixels are generated. By adding phase errors, the final phase used to simulate the unwrapped phase is shown in Fig. 4(d), which contains 2529 EPs with unwrapping errors.
By using the way described in Section 2, the median filters with the sizes of , , , , …, are applied to the simulated phase one by one, and only the results of , , and are shown in Fig. 5. After median filtering, the congruent phases are obtained and shown in Figs. 5(a)-5(d), respectively. By subtracting the ideal phase from these congruent phases, the unwrapping errors are obtained for different sizes as shown in Figs. 5(e)-5(h), respectively. For clarity, the number of the remaining EPs for different median filter sizes is also provided in Table 2. It can be observed that, the small-size of , although effective, only reduces unwrapping errors partially. The small-size of remove all the unwrapping errors near phase jumps, but some unwrapping errors close to step-heights and invalid pixels are still existing. The and give obvious pollution to step-heights, which cannot be directly used in the real measurement.
4. The proposed adaptive median filter
4.1 Principle of the adaptive median filter
According to the observations and analysis presented in Section 3, the small-size of the median filter can remove unwrapping errors near phase jumps, but some unwrapping errors close to step-heights and invalid pixels are still existing and annoying. An adaptive median filter for unwrapping error removal is proposed, consisting of two key steps, erroneous pixel detection and median filtering. These two steps will be iterated for automatic filter size determination until a stopping criterion is satisfied. The algorithm is illustrated in Fig. 6.
The first step of the erroneous pixel detection is proposed as follows. In the unwrapped phase map, by taking the current pixel as the center, a window is selected as its local area. For an erroneous pixel, its unwrapping error should be multiples of . Thus, in a local area, the phase values of an erroneous pixel and a good pixel are very different, while those of two good pixels are similar. By utilizing this property, we first classify any pixel within the window as either a kin pixel or an unrelated pixel. If , is a kin pixel of ; otherwise is unrelated pixel of . Next, in this window, if has more kin pixels than unrelated pixels, then is considered as a “good” pixel; otherwise, it is erroneous. The threshold value is selected as , which is the Nyquist sampling frequency.
In the second step, a median filter is applied only to these detected EPs. In order to automatically determine the filter size, a size scanning mechanism is proposed, i.e., the filter size is sequentially taken as , , , , , and so on. The advantage of this scanning mechanism is that, it prioritizes a small filter size and maximizes the chance of removing the noise while preventing the step-height pollution.
The stopping criteria are designed based on the number of detected EPs. We denote the number of detected EPs before filtering as , while the number of those after median filtering with a size of as . Our stopping criteria are as follows:
- (i) , indicating that there are no EPs;
- (ii) , indicating that the number of EPs does not reduce after three consecutive filtering attempts. If this happens, we consider that further median filtering is no longer effective.
4.2 Performance of the adaptive median filter
To evaluate the proposed adaptive median filter, we first apply it to the example shown in Fig. 2. The EPs in Fig. 2(c) are detected and . Because the measured flat board does not contain step-heights and invalid pixels, all the EPs in the unwrapped phase can be correctly detected. All these detected EPs are removed after two iterations in the adaptive median filter.
In the example shown in Fig. 4, EPs are detected according to Fig. 4(c), and , which is a little more than the actual number of 2529 because a small amount of good pixels close to step-heights are wrongly detected as EPs. However, all the true EPs can be correctly detected. For clarity, the congruent phase and its detected EPs using the median filter are reproduced in Figs. 7(a) and 7(b), respectively. The proposed adaptive median filter is then applied to the detected EPs. For each size scanning, the number of the detected EPs is reduced and recorded in Table 3.
When the size is scanned to , unwrapping errors of all the detected EPs are removed and the filtering is completed. The final congruent phase is shown in Fig. 7(c). By subtracting the ideal phase from the congruent phase, the remaining 37 EPs are obtained and shown in Fig. 7(d). Comparing with the true phase, these 37 EPs are not correct, but our EP detector is not able to find them as they are consistent with neighboring pixels. Despite of the imperfection, the adaptive median filter can remove most of true EPs, and dramatically improve the result from traditional median filters.
To study the effect of the noise level, another simulation is carried out with more EPs around the phase jumps. It can be noticed from Table 1 that our real system produces about 5000 EPs. We double this number to simulate a much worse situation, as shown in Figs. 8(a) and 8(b). As can be seen from these figures, almost all the phase jumping pixels are polluted. By applying the median filter, there are 186 remaing EPs, as shown in Fig. 8(c). By applying the adaptive median filter, there are 76 remaining EPs, as shown Fig. 8(d). Both the median filter and the adaptive filter are seen robust to a high noise level, and the latter outperforms the former. Note that we also simulated random noises in all fringe patterns for phase extraction, and they do not affect the unwrapping error suppression.
As a real example, a complicated David plaster is placed in front of the white flat board within the measurement range. The inner pixels in the captured area are selected. Figures 9(a)-9(c) show the David plaster, one of the captured sinusoidal patterns, and one of the captured Gray-code patterns, respectively. The unwrapped phase is obtained by using the traditional Gray-code method and shown in Fig. 10(a). We first mask the pixels with very low intensity or fringe modulation as invalid pixels and show them as black points . The adaptive median filtering is then applied, with the number of the detected EPs recorded in Table 4 and shown as red points in Fig. 10. When the size is scanned to , unwrapping errors of all the detected EPs are removed. The final congruent phase is shown in Fig. 10(b).
For comparison, the , , and median filters are also applied, and their congruent phases are shown in Figs. 10(c)-10(f), respectively. To see the filter performances more clearly, zoom-in views for a small blue box with detected EPs and a small red box with step-heights are shown in Figs. 11 and 12, respectively. The processing results are consistent with our previous analysis. The proposed adaptive median filter works better than the and ones in Fig. 11 and better than the and ones in Fig. 12.
5. The 3D reconstruction
As our PSP system has been calibrated by using the method in , all six phases in Fig. 10 are reconstructed into 3D profiles, which are shown correspondingly in Fig. 13. The profile in Fig. 13(a) corresponds to the phase map without any filtering, which presents many wrinkles reflecting EPs. Figure 13(b) shows the profile from the adaptive median filter, which contains no obvious wrong 3D points. Its superiority over and median filters in Figs. 13(e) and 13(f) is obvious. In contrast, its superiority over and is much less obvious, but still observable. Several examples are highlighted by red circles.
To further evaluate the performance of the adaptive median filter, two isolated objects including one step-ladder and one blade are measured. Figures 14(a)-14(c) show these two isolated objects, one sinusoidal pattern, and one Gray-code pattern, respectively. Figure 15 shows the reconstructed 3D profiles. The filtering results are presented in Fig. 15 in the same order and format as the previous example, from which the same conclusion can be drawn, showing the robustness of the proposed adaptive median filter.
An adaptive median filter is proposed to remove unwrapping errors in phase-shifting profilometry combined with Gray-code patterns projection. It is adaptive in the sense that, (i) the erroneous pixels are detected. Our median filter only applies to these erroneous pixels instead of the whole image; (ii) the filter size is gradually enlarged until a good result is obtained to maximize the chance of preventing edge-pollution. From the filtered result, a congruent congruent phase is computed and then converted to a 3D profile. Our simulations and experiments show the success of erroneous pixel removal and the superiority over traditional median filters. It should be noted the erroneous pixel detection process is a little time-consuming. Generally, the process time of the adaptive median filter is similar to the large-size of 2D median filter.
Natural Science Foundation of China (51475092) ; Natural Science Foundation of Jiangsu Province (BK20160693); National Research Foundation, Prime Minister’s Office, Singapore under its IDM Futures Funding Initiative.
References and links
1. S. Gorthi and P. Rastogi, “Fringe projection techniques: Whither we are?” Opt. Lasers Eng. 48(2), 133–140 (2010). [CrossRef]
2. B. Pan, Q. Kemao, L. Huang, and A. Asundi, “Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry,” Opt. Lett. 34(4), 416–418 (2009). [CrossRef] [PubMed]
4. D. Zheng, F. Da, Q. Kemao, and H. S. Seah, “Phase error analysis and compensation for phase shifting profilometry with projector defocusing,” Appl. Opt. 55(21), 5721–5728 (2016). [CrossRef] [PubMed]
6. M. Servin, J. A. Quiroga, and J. M. Padilla, Fringe Pattern Analysis for Optical Metrology (Wiley-Vch, 2014).
7. Q. Kemao, “A simple phase unwrapping approach based on filtering by windowed Fourier transform: A note on the threshold selection,” Opt. Laser Technol. 39(7), 1364–1369 (2007). [CrossRef]
8. Y. Xu, D. Darga, J. Smid, A. M. Zysk, D. Teh, S. A. Boppart, and P. S. Carney, “Filtering and Unwrapping Doppler Optical Coherence Tomography Velocity Maps,” in Imaging and Applied Optics 2016, OSA Technical Digest (online) (Optical Society of America, 2016), paper CW1C.4.
9. D. C. Ghiglia, and M. D. Pritt, eds., Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (Wiley, 1998).
11. J. A. Quiroga, D. Crespo, J. Vargas, and J. A. Gomez-Pedrero, “Adaptive spatiotemporal structured light method for fast three-dimensional measurement,” Opt. Eng. 47(10), 107203 (2006). [CrossRef]
14. M. Servin, J. M. Padilla, A. Gonzalez, and G. Garnica, “Temporal phase-unwrapping of static surfaces with 2-sensitivity fringe-patterns,” Opt. Express 23(12), 15806–15815 (2015). [CrossRef] [PubMed]
15. M. Servin, M. Padilla, G. Garnica, and A. Gonzalez, “Peofilometry of three-dimensional discontinuous solids by combiing two-steps temporal phase unwrapping, co-phased profilometry and phase-shifting interferometry,” Opt. Lasers Eng. 87(76), 75–82 (2016). [CrossRef]
19. G. Sansoni, M. Carocci, and R. Rodella, “Three-dimensional vision based on a combination of gray-code and phase-shift light projection: analysis and compensation of the systematic errors,” Appl. Opt. 38(31), 6565–6573 (1999). [CrossRef] [PubMed]
20. D. Zheng and F. Da, “Self-correction phase unwrapping method based on Gray-code light,” Opt. Lasers Eng. 50(8), 1130–1139 (2012). [CrossRef]
21. Q. Zhang, X. Su, L. Xiang, and X. Sun, “3-D shape measurement based on complementary Gray-code light,” Opt. Lasers Eng. 50(4), 574–579 (2012). [CrossRef]
22. S. Yu, J. Zhang, X. Yu, X. Sun, and H. Wu, “Unequal-period combination approach of gray code and phase-shifting for 3-D visual measurement,” Opt. Commun. 374(1), 97–106 (2016). [CrossRef]
23. L. Q. Bui and S. Lee, “Boundary Inheritance Codec for high-accuracy structured light three-dimensional reconstruction with comparative performance evaluation,” Appl. Opt. 52(22), 5355–5370 (2013). [CrossRef] [PubMed]
24. J. Vargas, T. Koninckx, J. A. Quiroga, and L. V. Gool, “Three-dimensional measurement of microchips using structured light techniques,” Opt. Eng. 47(5), 053602 (2008). [CrossRef]
25. J. Vargas, R. Restrepo, J. A. Quiroga, and T. Belenguer, “High dynamic range imaging method for interferometry,” Opt. Commun. 284(18), 4141–4145 (2011). [CrossRef]
26. N. Karpinsky, M. Hoke, V. Chen, and S. Zhang, “High-resolution, real-time three-dimensional shape measurement on graphics processing unit,” Opt. Eng. 53(2), 024105 (2014). [CrossRef]
30. T. Huang, G. Yang, and G. Tang, “A fast two-dimensional median filtering algorithm,” IEEE Trans. Acoust. Speech Signal Process. 27(1), 13–18 (1979). [CrossRef]
31. E. Arias-Castro and D. L. Donoho, “Does median filtering truly preserve edges better than linear filtering?” Ann. Stat. 37(3), 1172–1206 (2009). [CrossRef]
32. Q. Kemao, Windowed Fringe Pattern Analysis (SPIE, 2013).
33. L. Huang, P. S. K. Chua, and A. Asundi, “Least-squares calibration method for fringe projection profilometry considering camera lens distortion,” Appl. Opt. 49(9), 1539–1548 (2010). [CrossRef] [PubMed]
34. S. Zhang, “Phase unwrapping error reduction framework for a multiple-wavelength phase-shifting algorithm,” Opt. Eng. 48(10), 105601 (2009). [CrossRef]