The profile measurement system is widely used in industrial quality control, and phase unwrapping (PU) is a key technique. An algorithm-driven PU is often used to reduce the impact of noise-induced residues to retrieve the most reliable solution. However, measuring speed is lowered due to the searching of optimal integration paths or correcting of phase gradients. From the viewpoint of the rapidity of the system, this paper characterizes the noise-induced residues, and it proposes a clustering-driven residue filter based on a set of directional windows. The proposed procedure makes the wrapped phases included in the filtering window have more similar values, and it groups the correct and noisy phases into individual clusters along the local fringe direction adaptively. It is effective for the tightly packed fringes, and it converts the algorithm-driven PU to the residue-filtering-driven one. This improves the operating speed of the 3D reconstruction significantly. The tests performed on simulated and real projected fringes confirm the validity of our approach.
© 2011 Optical Society of America
Surface profile measurement by noncontact optical methods has been extensively applied to automated optical inspection, defect detection of surface shape, and solid modeling. In such a context, active methods based on the projection of structured light are very popular and commercially successful. A typical fringe projection profilometry (FPP) system consists of a projection unit, an image acquisition module, and a processing module [1, 2]. The implementation, as illustrated in Fig. 1, involves (i) projecting a set of structured patterns (e.g., sinusoidal fringes) onto the observed object surface, (ii) recording the fringe images that are phase modulated by the object’s height distribution, (iii) resolving the phase modulation using a fringe analysis technique and a proper phase unwrapping (PU) algorithm to acquire continuous phase distribution, and (iv) calibrating the system for mapping the continuous phase distribution to real-world 3D coordinates.
Common to most applications of FPP is the fact that the interesting physical information, e.g., shape, deformation, and displacement, is related to the absolute phase ϕ. However, only phase ψ of modulo-, the so-called wrapped phase, can be measured limited to the interval . Thus, in order to recover the continuous phase distribution, a 2D PU algorithm has to be used. Formally, PU is defined as given the wrapped phase , find the absolute phase , and the relationship between these two kinds of phases is as follows:3]. However, PU is still a problem due to the existence of residues, which are the potential source of phase-error propagation . Thus, the problem for PU is to minimize the impact of those residue-induced false jumps on the estimated unwrapped phase image. There are two ways to realize this purpose: algorithm driven and residue filtering driven. The existing algorithm-driven PUs for 3D reconstruction can be classified into four major categories: (i) path-following [5, 6], (ii) minimum-norm [7, 8, 9], (iii) model-based , and (iv) Bayesian-based [11, 12]. The advantage of these algorithm-driven approaches is that they minimize the number of phase jumps larger than π between neighboring points of the estimated unwrapped phases through searching optimal integration paths or correcting the phase gradients to retrieve the most reliable solution. The disadvantage is the computational burden of the optimization procedure. For example, the computational complexity of the minimum spanning tree method is , the computational complexity of the branch-cut method is , and the computational complexity of the network-flow method reaches .
In comparison, the motivation of the residue-filtering- driven approach is to identify residues and eliminate them in order to provide an irrotational field that can be integrated along any path. This makes 3D reconstruction faster with similar accuracy. Capanni et al.  proposed an adaptive median filter based on some histogram properties of the filtering window to identify noisy pixels in a wrapped phase image. However, this method introduces a smoothing impact on noise-free phases, and noisy phases cannot be located in areas with relatively dense noise. Lee et al.  presented a sigma filter as an adaptive extension of the averaging filter. It averages only selected phase values within the neighborhood along the directions of phase fringes. This filter, however, results in an undesired phase smoothing in residue-free areas. Nico  was confined to the description of configurations of residues, and designed a special filter. This method is based on the observation of the noisy phases inducing adjacent phase inconsistencies. A combinatorial analysis, however, shows that the filter designed is very complicated, owing to various configurations of the adjacent residues. Fornaro et al.  addressed a maximum-likelihood PU based on the phase difference, and the noise is postprocessed in the unwrapped phase image using a median filter. Recently, Qian et al.  proposed a windowed filter in the Fourier domain, and the filtered amplitude is used as a quality map. Hitherto, no new method can remove the residues thoroughly within the minimum operating time.
In this paper, we analyze the configurations of noise- induced residues, introduce clustering into the problem of residue filtering, design a set of directional windows to involve more similar phases, and propose a feasible filter to remove residues without the need of searching optimal integration paths or correcting the phase gradients. The remainder of the paper is organized as follows. Section 2 briefly introduces the principle of the FPP system. Section 3 analyzes the characteristics of residues. Section 4 addresses our motivations and the proposed filter. Section 5 demonstrates the validity of our approach in the profile measurement system with arbitrarily arranged devices. Section 6 presents the conclusion.
2. PRINCIPLE OF FRINGE PROJECTION PROFILOMETRY
In the FPP system, a set of structured fringe patterns is projected on the object surface. The captured images are phase modulated by the object height distribution. Thus, the task is to analyze the deformed fringes to determine the profile of the measured object. The main steps consist of fringe analysis, PU, and system calibration.
2A. Fringe Analysis
In the fringe projection technique, an object shape is evaluated through a phase distribution extracted from the captured fringes. In a general FPP system, the illumination is divergent and a nonlinear carrier will be introduced. Figure 2 shows the optical geometry of the FPP system. When a sinusoidal fringe is projected onto a 3D diffusing object, its deformed image can be expressed as
2B. Phase Unwrapping
In practice, the presence of shadows, low modulations, fringe discontinuities, and noise makes PU difficult. In the next sections, we will analyze how to eliminate the impact of noise-induced residues on PU to improve the running speed of the system. Given that all the residues are removed, PU can be operated in the wrapped phase image using Eqs. (4, 5) to acquire a continuous phase image:
2C. System Calibration
In Fig. 2, the unwrapped phase at point B must equate to the phases at points Q and A, respectively, which is the essence of calibration for an arbitrarily arranged FPP system. Du and Wang  derived a mathematical description of the out-of-plane height distribution. The analytic equation is as follows:
To calculate the out-of-plane height h, the unknown coefficients must be determined first. A proper way is to use the nonlinear least-squares algorithm, e.g., the Levenberg–Marquardt method. The least-squares error can be expressed as
3. CHARACTERISTICS OF NOISE-INDUCED RESIDUES
3A. Localization of Residues
We assume that the projected fringes are sampled sufficiently, in that, Nyquist sampling theorem is satisfied, and no abrupt shape change occurs on the object. A general method utilizes the irrotational property to check phase inconsistency , and a residue is located by evaluating the sum of the phase gradients counterclockwise (or clockwise) around each set of four adjacent points:5). For a given pixel neighborhood, if is different from zero, the area is called a residue in this paper. If the summation takes the value , it is referred to as positive residue (positive polarity); if the summation takes the value , it is referred to as negative residue (negative polarity). Figure 3 shows a phase image and its corresponding residue image.
3B. Configurations of Residues
The noise-induced residue is caused by the random fluctuation of phases due to noise in the fringe images. Of the four phase gradients in Eq. (8), only three are independent. For a given pixel neighborhood, it can be shown that the summation defined in Eq. (8) can be different from zero only if an odd number, one or three, of its four phase gradients round to a nonzero value. Some triads cannot occur, and only triads satisfying the following rules are observed: (i) two consecutive phase gradients greater than π in modulus must be of an opposite sign and (ii) two nonconsecutive phase gradients greater than π in modulus must share the same sign, and the third one must have the opposite sign. As Jiang and Cheng  depicted, most of the residues have only one phase gradient greater than π in modulus. From a quantitative viewpoint, the adjacent residues are the majority (more than 70% ). In this paper, we classify the noise-induced residues into four configurations: (i) a couple of adjacent opposite-sign residues in the form of left–right or up–down, (ii) a couple of adjacent opposite-sign residues in the form of diagonal, (iii) a couple of disjoint opposite-sign residues, and (iv) more than two residues joined together. Figure 4 depicts the four categories of residues. Besides, Karout et al.  indicate that a single residue only occurs close to the borders of the phase image—the simple fact of their border location causing their opposite-sign residue to lie outside the field of measurement.
4. CLUSTERING-DRIVEN RESIDUE FILTER
- The operating speed of the PU algorithm determines the running speed of a FPP system. Although the algorithm-driven PUs can minimize the adverse impact of residues, it is very time consuming to search integration paths or correct the phase gradients. Thus, the performance of the system might be reduced drastically. From the viewpoint of rapidity, the path-independent PU based on residue filtering is superior to the algorithm-driven one. Consequently, the main task reduces to filtering the residues effectively.
- Because residues are the potential source of phase-error propagation in the process of PU, we only have to filter the phases within a residue area rather than all the phases in the wrapped phase image. In an actual FPP system, residues are the minority relative to correct phases, and it is reasonable to regard these noisy phases inducing residues as outliers or anomalies locally. From the viewpoint of anomaly detection, the correct phases belong to large and dense clusters, while the noisy phases (or anomalies) belong to small or sparse clusters. Thus, we formulate the problem of residue filtering as a simple problem of 1D data clustering, as illustrated in Fig. 5.
- For steep slope areas, the fringe is tightly packed. Squared windows, such as the one for the boxcar filter, will contain more dissimilar phases. This will destroy the continuity of fringes and make the PU incorrect. Thus, we want to design directional windows to make the wrapped phases within the filtering window have more similar values. This makes filtering more effective and preserves the details of the fringes.
4B. Identification of the Filtering Window
To filter the noise-induced residues, the local wrapped phases in the operating window have to be unwrapped, because these discontinuous phases are improper to measure their similarity, and we are not able to identify the local fringe orientation as Lee et al.  depicted. We should choose an effective PU to unwrap the pixels within the window to continuous phases reliably.
We use 16 directional windows, as shown in Fig. 6. The window size is set to empirically, and the windows enclose the four categories of residues as shown in Fig. 4. Only the white pixels in windows are included for computation. The window to be selected should be approximately parallel to the local fringe. In the implementation, these windows are convolved with the phase image. The center of the adjacent residue area serves as the center of the windows. We take advantage of local statistics to analyze the similarities of local unwrapped phases, and corresponding variances are computed in all 16 windows individually using Eqs. (9, 10):
4C. Unsupervised Clustering
Because we are discussing a clustering problem, the k-means and the single-linkage (SL) algorithms are selected as candidates for their excellent performance. For the problem of 1D data clustering, the computational complexity of the SL algorithm is with the n samples, and the complexity of the k-means algorithm is with c clusters and T iterations . Because the number of clusters of an arbitrary residue area cannot be definite, SL, with unsupervised attribute, is selected as the basis of the proposed noise-residue-filtering algorithm rather than k-means clustering.
4D. Proposed Filtering Algorithm
- Locate the residues in the wrapped phase image. Residues are tested using Eq. (8). An accompanying flag matrix, the same size as the corresponding wrapped phase image, is introduced to mark the locations of the residues. The pixel neighborhood forming a residue is marked “1” (white), and the other pixels are marked “0” (black).
- Identify the filtering window. The window size is set to empirically. We choose Costantini’s approach  to unwrap the pixels locally within the windows to continuous phases. The sixteen directional windows in Fig. 6 convolve with a residue area, and the corresponding variances are computed individually. The window, with the minimum variance, is selected as the filtering window, as described in Subsection 4B.
- Locate the noisy phases and remove them. The wrapped correspondences of the local unwrapped phases in the filtering window serve as the input of the unsupervised classifier. The SL algorithm is used to classify the selected wrapped phases in the filtering window into different groups . The phases within the minimal cluster  are regarded as the noisy phases based on the assumption of an unsupervised anomaly detection. The assumption is that normal data instances belong to large and dense clusters, while anomalies either belong to small or sparse clusters . Thus, the noisy phases are replaced by the median phase within the maximal cluster .
- Judge whether there is still any residue. If any residue occurs, go back to step 1 until all the residues are removed.
After all the residues have been removed, and in Eq. (5) can be estimated by and reliably. Then we can unwrap the filtered wrapped phase image using Eq. (4) to acquire consistent unwrapped phase image.
In this section, the filter is used as a preprocessing step of PU to remove noise-induced residues. The filtering performance is evaluated by applying the filter to synthetic and real phase signals. We demonstrate the capability of the proposed filter in three aspects:
- processing tightly packed fringes with dense noise- induced residues effectively,
- removing noisy phases inducing residues without reducing the detailed information, and
- substituting the residue-filtering-driven PU for the algorithm-driven one reliably.
We test our algorithm on simulated and real phase images. The simulated data allow a quantitative validation of the method. We compare the proposed filter with the complex average filter  and the local histogram-based filter  from the viewpoints of residue reduction and fringe preservation, while we use real data to compare the path-independent PU based on the proposed filter with the algorithm-driven methods [6, 7, 8] from the viewpoint of rapidity. The accuracy is also demonstrated after calibrating the FPP system in a less controlled situation. Besides, the reason why we choose these comparative algorithms is that they are representative and widely used in practice.
We simulate four phase-shifted fringes as the object to be constructed. The fringe images are contaminated by dense random noise, and the corresponding phase image is shown in Fig. 7a. In the proposed algorithm, the residues are filtered iteratively, because new residues may occur when previous residues are removed. This means that a wrapped phase, not inducing a residue, is not always a noise-free one. Figure 7b is the rewrapped phase reconstruction using the proposed filter. The fringes are clear and unbroken, indicating that the proposed filter removes the noise-induced residues thoroughly and preserves the fringe boundaries effectively.
Figure 7c illustrates the rewrapped phase fringe based on the complex average filter. Obviously, area A is blurred, indicating that there is residue-induced erroneous unwrapping. Using a local histogram-based filter, there are similar phenomena at B and C, as shown in Fig. 7d. Each of the three discussed algorithms applies a filtering window; however, the results are quite different. The fundamental reason is that the selected window used in the proposed algorithm varies adaptively according to the local orientation of the fringe. The phases within this window are classified into correct phase groups and noisy phase groups accurately. The noisy phases are removed according to the assumption of unsupervised anomaly detection. On the contrary, although the complex average filter is feasible to filter random noise in fringe images, the selection of window size may impact the preservation of the fringe edges and the accuracy of the PU, particularly for the case of narrow spacing fringes. On the other hand, if the noise is dense, the shape of the local histogram may be distorted and a clear peak may not be distinguished, so that it becomes difficult to decide whether there is a phase jump crossing the window, and the rate of false detection increases consequently. Besides, the selection of threshold σ (see ) may impact the filtering effect significantly. If σ is too small, the window will smooth the noise-free phases, and if σ is too big, the noisy phases inducing residues cannot be identified. Thus, the criterion to determine optimal σ must be constructed beforehand.
We also verify the rapidity of path-independent PU based on the proposed filter using real phase signals. The surface of a container is selected as the object, because the corresponding measurement is relatively complex in natural environment. This can verify our filter convictively. Our FPP system is shown in Fig. 8. Four phase-shifted sinusoidal fringe images are projected on the surface of a container, and the corresponding deformed fringes are captured by the camera successively. All the captured images share the same size , and the corresponding measured area is . The actual size of the observed surface is shown in Fig. 9. Figure 10 illustrates the process of 3D reconstruction of a partial surface of the container. The number of residues reduces from 589 to 0 within three iterations. The PU result using our filter is shown in Fig. 10d. Obviously, no error propagation occurs, and the time consumed is only . Although the algorithm-driven methods can acquire the same result, the operation for PU is time consuming. From the viewpoint of rapidity, the sequence in ascending order is PUs based on minimum-cost-flow, branch-cut, least-squares, and SL, as shown in Table 1. Table 2 lists the computational complexity of the major procedures for separate approaches. The problem of minimum cost flow is equivalent to the problem of linear programming, and this procedure is the most time consuming. For the branch-cut algorithm, flood filling is used to prevent the integration path from crossing the branch cuts. The least-squares-driven PU is a computationally efficient algorithm. However, the least-squares procedure tends to spread the errors rather than containing them within a limited set of points. In comparison with the above three methods, the most time-consuming procedure is the locating residues for an SL-driven PU. In Table 2, N is the pixel number of an image; T is the iteration number of residue filtering, ; is the number of residues in each iteration, ; and n is the point number within the filtering window, .
To confirm further that the estimated unwrapped phase image is almost correct, we will calibrate the profile measurement system for mapping the unwrapped phases to real world 3D coordinates. In order to avoid an undetermined solution for the problem of nonlinear least squares, two gauge blocks, with uniform top heights of 9.115 and , are employed. Figure 11 shows one projected fringe image on the two gauge blocks. When the Levenberg–Marquard algorithm is used, we should set a suitable initial value to prevent the solution from converging to the local minimum. In Eq. (7), let the content within the brackets equal zero, for , , and then we get11) can be described as 12), we have 13). Substituting into the Levenberg–Marquardt algorithm, we obtain . Finally, we retrieve the height map using Eq. (6). The height map is shown in Fig. 12. After repeated measurements, we list the accuracy analysis of the height measurement of the top surface in Table 3: is the actual value, is the maximum measured value, is the minimum measured value, is the average measured value, is the maximum deviation, and is the average deviation. Consequently, the error of repeated measurements is less than 2%.
In terms of the characteristics of noise-induced residues, a fast clustering-driven residue filter is proposed, and it is embedded in the arbitrarily arranged FPP system. It uses a set of directional windows to include more similar phases, and it classifies the noisy phases inducing residues and the other correct phases within the filtering window into separate clusters accurately. The noisy phases are removed based on the assumption of anomaly detection. The filter is designed to make the measurement system substitute the residue-filtering-driven PU for the algorithm-driven one. Experimental results demonstrated the filter’s effectiveness in residue reduction, fringe preservation, and time savings. After all the residues are removed, the error of repeated measurements is less than 2%.
The authors would like to thank Dacheng Tao and Wei Bian from Nanyang Technological University for their constructive comments, and Bruno Luong from FOGALE Nanotech for his help with compiling the program of “2D phase unwrapping algorithm based on network flow.”
The work described in this paper is supported by the National Natural Science Foundation of China (NSFC) (grant 60806050), the Knowledge Innovation Program of the Chinese Academy of Sciences (CAS) (KGCX2-YW-156, KGCX2-YW-154), the Key Laboratory of Robotics and Intelligent System of Guangdong Province (2009A060800016), the Shenzhen Technology Project (JC200903160416A), and the Shenzhen Nanshan Research Project (2009016).
1. F. Chen, G. M. Brown, and M. Song, “Overview of three- dimensional shape measurement using optical methods,” Opt. Eng. 39, 10–22 (2000). [CrossRef]
2. S. S. Gorthi and P. Rastogi, “Fringe projection techniques: whither we are?” Opt. Lasers Eng. 48, 133–140 (2010). [CrossRef]
5. H. A. Zebker and Y. P. Lu, “Phase unwrapping algorithms for radar interferometry: residue-cut, least-squares, and synthesis algorithms,” J. Opt. Soc. Am. A 15, 586–598 (1998). [CrossRef]
7. D. C. Ghiglia and L. A. Romero, “Minimum -norm two- dimensional phase unwrapping,” J. Opt. Soc. Am. A 13, 1999–2013 (1996). [CrossRef]
8. M. Costantini, “A novel phase unwrapping method based on network programming,” IEEE Trans. Geosci. Remote Sensing 36, 813–821 (1998). [CrossRef]
9. M. Hubig, S. Suchandt, and N. Adam, “A class of solution- invariant transformations of cost functions for minimum cost flow phase unwrapping,” J. Opt. Soc. Am. A 21, 1975–1987 (2004). [CrossRef]
11. J. M. N. Leitão and M. A. T. Figueiredo, “Absolute phase image reconstruction: a stochastic nonlinear filtering approach,” IEEE Trans. Image Process. 7, 868–882 (1998). [CrossRef]
12. G. Nico, G. Palubinskas, and M. Datcu, “Bayesian approaches to phase unwrapping: theoretical study,” IEEE Trans. Signal Process. 48, 2545–2556 (2000). [CrossRef]
14. A. Capanni, L. Pezzati, D. Bertani, M. Cetica, and F. Francini, “Phase-shifting speckle interferometry: a noise reduction filter for phase unwrapping,” Opt. Eng. 36, 2466–2472 (1997). [CrossRef]
15. J. S. Lee, K. P. Papathanassiou, T. L. Ainsworth, M. R. Grunes, and A. Reigber, “A new technique for noise filtering of SAR interferometric phase images,” IEEE Trans. Geosci. Remote Sensing 36, 1173 (1998). [CrossRef]
16. G. Nico, “Noise-residue filtering of interferometric phase images,” J. Opt. Soc. Am. A. 17, 1962–1974 (2000). [CrossRef]
18. K. M. Qian, W. Gao, and H. Wang, “Windowed Fourier-filtered and quality-guided phase-unwrapping algorithm,” Appl. Opt. 47, 5408 (2008). [CrossRef]
20. J. Jiang and J. Cheng, “Noise-residue filtering based on unsupervised clustering for phase unwrapping,” in Proceedings of 5th International Symposium on Visual Computing, M. Z. Nashed, ed. (Springer, 2009), pp. 719–727.
21. S. A. Karout, M. A. Gdeisat, D. R. Burton, and M. J. Lalor, “Residue vector, an approach to branch-cut placement in phase unwrapping: theoretical study,” Appl. Opt. 46, 4712–4727 (2007). [CrossRef] [PubMed]
22. P. Berkhin, Survey of Clustering Data Mining Techniques (Springer, 2002).
23. is the cluster in which the number of members is smallest.
24. V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: a survey,” ACM Comput. Surv. 41, 1–58 (2009). [CrossRef]
25. is the cluster in which the number of members is biggest.
26. D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (Wiley, 1998).