Spatial-temporal phase unwrapping algorithm for fringe projection profilometry

Haihua An; Yiping Cao; Haitao Wu; Na Yang; Cai Xu; Hongmei Li

doi:10.1364/OE.430305

1. Introduction

Fringe projection profilometry (FPP) [1–5] is widely applied in many fields of virtual reality, industrial production and biomedical sciences due to its high speed and accuracy [6–9]. FPP can be typically described as two primary categories: Fourier transform profilometry (FTP) [10,11] and phase-shifting profilometry (PSP) [12–15]. Generally, the former only requires to project single fringe pattern, but the later by projecting multiple fringe patterns is more robust for the complex measured objects. Whether it’s FTP or PSP, the extracted phase is wrapped from -π to π with 2π discontinuities [16,17]. Therefore, phase unwrapping process has always played an important role to retrieve the absolute phase map.

Spatial phase unwrapping algorithms (SPUAs) [18–21] have been deeply researched and widely used in FPP, which can be classified into four types: branch-cut algorithms, the minimal norm algorithms, the regional growth algorithms and quality-guided phase unwrapping algorithms (QGPUAs) [18,21]. In these SPUAs, QGPUAs can automatically select the direction of the unwrapping path to avoid spatial discontinuities, so usually reach higher measuring accuracy [16,20,22]. Unfortunately, the optimal unwrapping path could not be found due to the discontinuous or isolated objects.

To retrieve the absolute phase map, many related researchers proposed various temporal phase unwrapping algorithms (TPUAs) [2,3,23–32] working by point-to-point calculation. Multifrequency approaches [2,3,25,26] utilize the assistance of more wrapped phase maps differing in fringe pattern periods to resolve the phase ambiguity of the discontinuous surface, while suffering phase-unwrapping difficulty when the selected frequencies are far higher than the low frequency. Phase-code approaches [27–29] determine the fringe order by embedding codeword into the corresponding phase rather than the intensity of patterns, so that have higher robustness. Gray-code approaches [30–32] project a serial binary Gray-code patterns to remove the phase ambiguity, and M patterns can code 2^M fringe orders. Compared with SPUAs, each phase point is unwrapped independently so that TPUAs can ignore the discontinuities of the measured objects. However, due to the truncation property of the stair fringe order, the existing TPUAs have pseudo-periodic jump errors, which has always been one of the main troubles [28,32]. Median filter or projecting more additional patterns [29,31] are applied to correct and compensate the absolute phase, but these techniques usually income new errors because of the intensity instability, the nonlinearity of equipment and the reflectivity difference of object’s surfaces.

This paper presents a spatial-temporal phase unwrapping algorithm (STPUA) combining SPUAs and TPUAs for the first time. The wrapped phase, which is extracted from phase-shifting deformed patterns, is separated into a series of isolated regional wrapped phases, which can be unwrapped to regional unwrapped phases using QGPUAs. Meanwhile, the corresponding regional fringe order maps can be obtained from the code deformed patterns. By selecting fringe orders of the reliable points from regional order maps, all regional unwrapped phases can be successively mapped into an absolute phase. The selection of the reliable points has enough radii in phase-shifting direction, which can provide the high robustness for eliminating the phase ambiguity. Therefore, the proposed STPUA can resolve order jumps in the conventional TPUAs, while inheriting the high measuring accuracy of QGPUAs. In addition, no more additional fringe patterns are projected by DLP to compensate the absolute phase map compared with the corresponding TPUAs, so it can be applied in real-time 3D measurement.

2. Principle

2.1 Description of phase-shifting profilometry

Phase-shifting profilometry is widely employed in optical 3D metrology due to its concision and accuracy. The conventional N-step phase-shifting algorithms can be classically expressed as

(1)$${I_n}(x,y) = {A^{\prime}}(x,y) + {A^{^{\prime\prime}}}(x,y)\cos [\mathrm{\varphi} (x,y) + \frac{{2n\pi }}{N}]{\kern 1pt} {\kern 1pt} {\kern 1pt} , $$

where n is the index of the designed sinusoidal fringe pattern ($n = 1,2, \cdots ,N$), ${A^{\prime}}(x,y)$ and ${A^{^{\prime\prime}}}(x,y)$ represent the ambient lights and the intensity modulation, respectively. $\mathrm{\varphi} (x,y)$ is the measured phase, it can be typically calculated by

(2)$$\mathrm{\varphi} (x,y) = {\tan ^{ - 1}}\left( {\frac{{\sum\limits_{n = \textrm{1}}^N {{I_n}(x,y)\sin (n\delta )} }}{{\sum\limits_{n = \textrm{1}}^N {{I_n}(x,y)\cos (n\delta )} }}} \right){\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

In Eq. (2), $\mathrm{\varphi} (x,y)$ is wrapped in ($- \pi ,\pi $] because of the arctangent operation. To unwrap thoroughly the wrapped phase with $2\pi $ discontinuities, QGPUAs, one of SPUAs, are successfully applied in the single connected object but has unsolvable problems when multiple isolated objects with discontinuous surfaces are simultaneously measured [17,22]. Compared with SPUAs, TPUAs can recover the absolute phase of each pixel separately by projecting a series of additional coded fringe patterns. Among of TPUAs, phase-code methods and Gray-code methods are typical two ways to obtain fringe order map, and have been maturely used.

2.2 Principle of phase-code method

In phase-code method, a stair phase codeword ${\mathrm{\varphi} ^p}(x,y)$ with the fringe order information is directly embedded into the N-step phase-shifting fringe patterns, which can be described as

(3)$$I_n^p(x,y) = {A^{\prime}}(x,y) + {A^{^{\prime\prime}}}(x,y)\cos [({\mathrm{\varphi} ^p}(x,y) + \frac{{2n\pi }}{N}]{\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

The stair phase ${\mathrm{\varphi} ^p}(x,y)$ can be expressed as

(4)$${\mathrm{\varphi} ^p}(x,y) ={-} \pi + {k^p}(x,y) \times \frac{{2\pi }}{T}{\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where ${k^p}(x,y)$ is fringe order, which can be expressed as ${k^p}(x,y) = {\mathop{\rm int}} (x/P)$. P is the period of the sinusoidal fringe patterns in Eq. (1), and T is the total number of fringe period. As shown in Fig. 1, $\mathrm{\varphi} (x,y)$ and ${\mathrm{\varphi} ^p}(x,y)$ can be correspondingly calculated from two groups of phase-shifting fringe patterns, then ${k^p}(x,y)$ can be determined by

(5)$${k^p}(x,y) = round\{ N[{\mathrm{\varphi} ^p}(x,y) + \mathrm{\pi} ]/(2\mathrm{\pi} )\} {\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where $round({\kern 1pt} {\kern 1pt} )$ computes the closest integer in point $(x,y)$. Combine the wrapped phase $\mathrm{\varphi} (x,y)$ and the fringe order ${k^p}(x,y)$, the absolute phase can be easily obtained by

(6)$$\Phi (x,y) = \mathrm{\varphi} (x,y) \pm 2{k^p}(x,y)\mathrm{\pi} {\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

Fig. 1. The principle of phase-code method.

Download Full Size | PDF

Specifically, ${\mathrm{\varphi} ^p}(x,y)$ can be directly embedded into the traditional N-step phase-shifting fringe patterns successively when $N = 4$, the process can be expressed as

(7)$$\begin{array}{l} \left\{ \begin{array}{l} I_\textrm{1}^p(x,y) = {A^{\prime}}(x,y) + {A^{^{\prime\prime}}}(x,y)\sin [\mathrm{\varphi} (x,y) + {\mathrm{\varphi}^p}(x,y)]\\ I_2^p(x,y) = {A^{\prime}}(x,y) - {A^{^{\prime\prime}}}(x,y)\sin [\mathrm{\varphi} (x,y) - {\mathrm{\varphi}^p}(x,y)]\\ I_3^p(x,y) = {A^{\prime}}(x,y) - {A^{^{\prime\prime}}}(x,y)\cos [\mathrm{\varphi} (x,y) + {\mathrm{\varphi}^p}(x,y)]\\ I_4^p(x,y) = {A^{\prime}}(x,y) + {A^{^{\prime\prime}}}(x,y)\cos [\mathrm{\varphi} (x,y) - {\mathrm{\varphi}^p}(x,y)] \end{array} \right.{\kern 1pt} {\kern 1pt} {\kern 1pt} ,\\ {\kern 1pt} {\kern 1pt} {\kern 1pt} \end{array}.$$

Accordingly, $\mathrm{\varphi} (x,y)$ can be still obtained by Eq. (2) and ${\mathrm{\varphi} ^p}(x,y)$ can be calculated by

(8)$${\mathrm{\varphi} ^p}(x,y) = {\tan ^{ - 1}}[\frac{{I_1^p(x,y) + I_2^p(x,y) - I_3^p(x,y) - I_4^p(x,y)}}{{ - I_1^p(x,y) + I_2^p(x,y) - I_3^p(x,y) + I_4^p(x,y)}}]{\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

By only projecting four phase-code fringe patterns, both the wrapped phase and the fringe order map can be retrieved, so this coding method can realize the real-time measurement for the isolated objects [29].

2.3 Principle of Gray-code method

Gray-code TPUA is perhaps the simplest method by projecting a series of pre-designed binary patterns. Currently, M Gray-code fringe patterns (e.g., M=4 in Fig. 2) can determine a maximum 2^M fringe orders. In this paper, a general model is developed for generating M Gray-code patterns of size $X \times Y$ pixels, which can be described as

\begin{array}{l} {G_m} = zeors(X,Y);\\ for{\kern 1pt} {\kern 1pt} y = 1:Y\\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} for{\kern 1pt} {\kern 1pt} {\kern 1pt} i = 1:{2^m}:{\textrm{2}^M}\\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {G_m}\{ (i + {2^{m - 1}} - 1)P + 1:(i + {2^{m - 1}} - 1)P + {2^{m - 1}}P,y\} = 1;\\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} end\\ end \end{array}

where ${G_m}$ represents the m^th binary Gray-code patten ${G_m}(x,y)$, $m = 1,2, \cdots ,M$ . P is the is the period of the corresponding sinusoidal fringe patterns. As shown in Fig. 2, fringe order map ${k^G}(x,y)$ can be determined by

(9)$${k^G}(x,y) = \sum\limits_{m = 1}^{M - 1} {{2^{M - m - 1}}} {G_m}(x,y){\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

Fig. 2. The principle of Gray-code method.

Download Full Size | PDF

Similarly, the absolute phase map $\Phi (x,y)$ in Fig. 2 can be calculated by Eq. (6).

2.4 Phase-to-height algorithm

After obtaining the absolute phase, the phase-to-height mapping relationship [33,34] is applied to reconstruct the geometry information of the measured objects, it can be expressed as

(10)$$\frac{1}{{h(x,y)}} = \mathrm{\alpha} (x,y) + {\mathrm{\beta} _1}(x,y)\frac{1}{{\Delta \Phi (x,y)}} + {\mathrm{\beta} _2}(x,y)\frac{{{\mathrm{\varphi} _c}(x,y)}}{{\Delta \Phi (x,y)}} + \gamma (x,y)\frac{1}{{\Delta {\Phi ^2}(x,y)}}{\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where $\mathrm{\alpha} (x,y){\kern 1pt} ,{\kern 1pt} {\kern 1pt} {\mathrm{\beta} _\textrm{1}}(x,y),{\mathrm{\beta} _2}(x,y)$ and $\gamma (x,y)$ are the FPP system constants that can be calculated by measuring a series of standard planes and pre-saved in computer for further measurement. $\Delta \Phi (x,y)$ is the absolute phase map relative to the phase map of the reference plane ${\mathrm{\varphi} _c}(x,y)$.

2.5 Discussion

In theory, TPUAs, such as phase-code method and Gray-code method, are essential algorithms to overcome the phase ambiguity and error propagation when measuring isolated objects with discontinuous surfaces. However, for the existing TPUAs, the truncation property between two adjacent fringe orders will easily lead to order jump errors at the order edge. As shown in Fig. 1 and Fig. 2, there are still obvious order jumps in the absolute phase because of the inevitable digital errors, while guaranteeing the period of the sinusoidal fringe patterns $P = {2^s}$ (s is an integer, e.g., s=4 in Fig. 1 and Fig. 2) and unwrapped phase without simulating objects. In actual measurement, the intensity instability, the nonlinearity of equipment and the reflectivity difference of object’s surfaces will seriously exacerbate order jump problem, so that fringe orders map from TPUAs even has the $2\mathrm{\pi} $ discontinuities in simple surfaces but SPUAs do not. To overcome this mainly challenge in TPUAs, median filter or projecting more additional patterns are applied to correct or compensate the absolute phase. Meanwhile, these techniques income new errors, and they are difficult for the phase from high frequency fringe pattern [31]. On the contrary, QGPUAs based on path programming perform high measuring accuracy for the single connected phase map. Combine strengths of SPUAs and TPUAs, STPUA proposed in this paper breaks up the whole unwrapped phase into a series of regional connected phase. QGPUA [18] is applied to unwrap all regional connected phase, then the corresponding reliable points selected from the fringe order ${k^p}(x,y)$ or ${k^G}(x,y)$ map all regional unwrapped phase into the absolute phase successively. The proposed algorithm can avoid the order jump problem of TPUAs, while inheriting high measuring accuracy of QGPUAs.

3. Proposed spatial-temporal phase unwrapping algorithm (STPUA)

In this section, the complete process and principle of the proposed STPUA will be expounded in detail by measuring two isolated objects. As shown in Fig. 3, the wrapped phase $\mathrm{\varphi} (x,y)$ and modulation of the captured deformed patterns $Mo(x,y)$ can be obtained first from the conventional N-step phase-shifting deformed patterns, in which $Mo(x,y)$ can be directly calculated by

(11)$$Mo(x,y) = \sqrt {[\sum\limits_{n = 1}^N {I_n^{\prime}(x,y)\sin (2n\mathrm{\pi} /N){]^2}} \textrm{ + }[\sum\limits_{n = 1}^N {I_n^{\prime}(x,y)\cos (2n\mathrm{\pi} /N){]^2}} } {\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where $I_n^{\prime}(x,y)$ represents the phase-shifting deformed pattern captured by camera. Combining Eq. (1) and Eq. (11), $Mo(x,y)$ can be simplified to

(12)$$Mo(x,y) = \frac{1}{2}NA_c^{^{\prime\prime}}(x,y){\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where $A_c^{^{\prime\prime}}(x,y)$ is the intensity modulation of $I_n^{\prime}(x,y)$. Obviously, $Mo(x,y)$ contains the complete information about the area to be measured due to the intensity complementarity of all phase-shifting deformed pattern. To extract the area illuminated by the structured light, $Mo(x,y)$ is binarized into a mask image $B(x,y)$ by OTSU algorithm [35,36].

Fig. 3. The principle of the proposed STPUA. (a) reconstructed result by TPUA, (b) reconstructed result by the proposed algorithm.

Download Full Size | PDF

In $B(x,y)$, there are u isolated connected domains, u is extremely related to the number of the isolated objects and closed shadows (e.g., u=2 in Fig. 3). In particular, the areas of all isolated connected domains are calculated and marked as ${B_u}(x,y)$ correspondingly. To obtain more complete phase and eliminate invalid information, ${B_u}(x,y)$ whose area is larger than a certain value can be successively extracted according to their area’s size. Concurrently, fringe order $K(x,y)$ could be obtained by capturing a group of phase-code deformed pattern $I_n^p(x,y)$.

After obtaining ${B_u}(x,y),{\kern 1pt} {\kern 1pt} \mathrm{\varphi} (x,y)$ and $K(x,y)$, the regionalization will be performed. Firstly, the regional wrapped phases ${\mathrm{\varphi} _u}(x,y)$ can be extracted from the whole wrapped phase $\mathrm{\varphi} (x,y)$ utilizing ${B_u}(x,y)$ and described as

(13)$${\mathrm{\varphi} _u}(x,y) = \mathrm{\varphi} (x,y) \times {B_u}(x,y){\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

Similarly, the regional fringe orders ${K_u}(x,y)$ extracted from $K(x,y)$ can be expressed as

(14)$${K_u}(x,y) = K(x,y) \times {B_u}(x,y){\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

Subsequently, QGPUA is applied to unwrap all ${\mathrm{\varphi} _u}(x,y)$ and the unwrapping initial point $(x_u^s,y_u^s)$ of ${\mathrm{\varphi} _u}(x,y)$ can be described as

(15)$$\left\{ \begin{array}{l} x_u^s = \sum\limits_{(x,y) \in {\Omega _u}} {x{\mathrm{\varphi}_u}(x,y)/\sum\limits_{(x,y) \in {\Omega _u}} {{\mathrm{\varphi}_u}(x,y)} } \\ y_u^s = \sum\limits_{(x,y) \in {\Omega _u}} {y{\mathrm{\varphi}_u}(x,y)/\sum\limits_{(x,y) \in {\Omega _u}} {{\mathrm{\varphi}_u}(x,y)} } \end{array} \right.{\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where ${\Omega _u}$ represents the effective area of the corresponding ${\mathrm{\varphi} _u}(x,y)$. The regional unwrapped phases ${\Phi _u}(x,y)$ could be calculated by

(16)$${\Phi _u}(x,y) = {\mathrm{\varphi} _u}(x,y) + 2{k_u}(x,y)\mathrm{\pi} {\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where ${k_u}(x,y)$ is fringe order relative to point $(x_u^s,y_u^s)$ in the corresponding ${\mathrm{\varphi} _u}(x,y)$. Meanwhile, a series of the reliable fringe orders ${K_u}({x_u},{y_u})$ are selected from the corresponding ${K_u}(x,y)$. To avoid the points at the stair edge of ${K_u}(x,y)$, the following constraint is satisfied in phase-shifting direction for ${K_u}({x_u},{y_u})$:

(17)$${K_u}({x_u} - d + 1,y) = {K_u}({x_u} - d + 2,{y_u}) = \cdots = {K_u}({x_u},{y_u}) = \cdots {K_u}({x_u} + d - 1,{y_u}) = {K_u}({x_u} + d,{y_u}),$$

where $d = \frac{{{P^{\prime}}}}{4}$, which is called the radius of reliable point. $P^{\prime}$ is the approximate period of $I_n^{\prime}(x,y)$. The absolute regional phases $\Phi _u^{ST}(x,y)$ could be calculated by

(18)$$\Phi _u^{ST}(x,y) = [{\Phi _u}(x,y) - 2{k_u}({x_u},{y_u})\mathrm{\pi} ] + 2{K_u}({x_u},{y_u})\mathrm{\pi} {\kern 1pt} {\kern 1pt} {\kern 1pt} ,$$

where ${k_u}({x_u},{y_u})$ represents fringe order of the reliable point $({x_u},{y_u})$ in ${\Phi _u}(x,y)$, which can be expressed as

(19)$${k_u}({x_u},{y_u}) = round[\frac{{{\Phi _u}({x_u},{y_u}) - {\mathrm{\varphi} _u}({x_u},{y_u})}}{{2\mathrm{\pi} }}]{\kern 1pt} {\kern 1pt} {\kern 1pt} .$$

Finally, all $\Phi _u^{ST}(x,y)$ are summed together to generate the absolute phase ${\Phi ^{ST}}(x,y)$, which can achieve the reconstruction of the isolated objects.

For conventional TPUA based on phase-code algorithm, the measured objects are directly reconstructed by $\mathrm{\varphi} (x,y)$ and $K(x,y)$, and then visualization is performed. As shown in Fig. 3(a), reconstructed result has obvious pseudo-periodic errors due to order jumps in $K(x,y)$. By contrast, the proposed STPUA can overcome the order jump problem effectively, reconstructed result shown in Fig. 3(b) inherits high measuring accuracy of QGPUA. Furthermore, no additional patterns are projected compared with the corresponding TPUAs.

4. Experiments and results

In this paper, as shown in Fig. 4(a), a typical FPP system is developed, which mainly contains two major devices: an HDLP (Light crafter 4500 with the frame rate of 120 fps for 8-bits Grayscale) and a high frame CMOS monochrome camera (HCX20 with the resolution of 1624 × 1326 pixel and the highest frame rate 337 fps). As shown in Fig. 4(b), to achieve real-time acquisition, the HDLP projects the pre-designed fringe patterns onto the measured objects individually and sequentially at a frame rate that is four times higher than the object dynamic rate [37], and the corresponding deformed patterns can be captured by the synchronized monochrome camera with the projecting frame rate of HDLP. Furthermore, the calculating condition of the computer includes: a NVIDIA GeForce GTX1660ti graphic card, an Intel i7-10750H processor, and 16G memory.

Fig. 4. The experimental system. (a) the schematic diagram of the typical FPP, (b) the experimental devices.

Download Full Size | PDF

4.1 Analysis of measuring accuracy

Firstly, the multiple objects with the complex surfaces and color difference shown in Fig. 5 are measured to indicate the high measuring accuracy of QGPUA compared with the traditional phase-code and Gray-code TPUAs, in which the reference plane provides a single connected wrapped phase. Figures 6(a)–6(c) show four conventional phase-shifting deformed patterns, four phase-code phase-shifting deformed patterns, five Gray-code deformed patterns, respectively. Figures 6(d)–6(f) show the whole wrapped phase $\mathrm{\varphi} (x,y)$, the phase-code fringe order ${K^p}(x,y)$ and Gray-code fringe order ${K^G}(x,y)$ extracted from the corresponding deformed patterns, respectively. Whether it’s ${K^p}(x,y)$ and ${K^G}(x,y)$, the range of fringe order is [11,26] based on the captured area of camera. Notably, the phase-code approach is more robust than the Gray-code approach based on the intensity of patterns, ${K^G}(x,y)$ is even out the objective range value because of the color difference and sharp surfaces.

Fig. 5. Measured object with the reference plane.

Download Full Size | PDF

Fig. 6. The captured deformed patterns and extracted results. (a) four conventional phase-shifting deformed patterns, (b) four phase-code phase-shifting deformed patterns, (c) five Gray-code deformed patterns, (d) wrapped phase extracted from Fig. 5(a), (e) fringe order extracted from Fig. 5(b), (f) fringe order extracted from Fig. 5(c).

Download Full Size | PDF

Figure 7 shows the section views of the 1140^th column of Figs. 6(d)–6(f) and the partial enlarged comparison. It can be clearly seen that there are two main error sources. On the one hand, the jumps of $\mathrm{\varphi} (x,y)$ and ${K^p}(x,y)$ (or ${K^G}(x,y)$) at the 2π discontinuities are not synchronized. On the other hand, ${K^p}(x,y)$ appears the order jumps in the area where $\mathrm{\varphi} (x,y)$ changes dramatically and ${K^G}(x,y)$ has serious jump errors in the area where the surface’s reflectivity is too high or low. Therefore, as shown in Figs. 8(a)–8(c), the reconstructed surfaces utilizing the mentioned TPUAs are coarse due to order jumps, but QGPUA based on the optimal path programming can obtain considerable measuring accuracy. Further, Fig. 8(d) shows the section views of the 600^th column of Fig. 8(a)–8(c), which indicates QGPUA can avoid the order jump problem in TPUAs for a single connected phase.

Fig. 7. The section views of $\mathrm{\varphi} (x,y),{\kern 1pt} {\kern 1pt} {\kern 1pt} {K^p}(x,y)$ and ${K^G}(x,y)$.

Download Full Size | PDF

Fig. 8. Reconstructed results for a single connected wrapped phase (height:mm). (a)-(c) reconstructed results from the phase-code TPUA, the Gray-code TPUA and QGPUA, (d) the section views of the 600^th column in Fig. 7(a)-(c).

Download Full Size | PDF

Then, the isolated objects containing three standard geometries shown in Fig. 9(a) are measured to stress the advantages of the proposed STPUA and its measuring accuracy, in which the radius of the ball $R = \textrm{17}.00 \pm 0.20$ mm. Figure 9(b) shows three isolated regional wrapped phase ${\mathrm{\varphi} _u}(x,y)$, Fig. 9(c) show the reconstructed result ${h^p}(x,y)$ from fringe order ${K^p}(x,y)$ and wrapped phase $\mathrm{\varphi} (x,y)$, Fig. 9(d) shows the reconstructed result ${h^G}(x,y)$ from the fringe order ${K^G}(x,y)$ and the whole wrapped phase $\mathrm{\varphi} (x,y)$. It can be clearly seen that there are still inevitable jump errors on the smooth surfaces of all geometries. Figure 9(e) show the reconstructed result ${h^{Sp}}(x,y)$ combining QGPUA and the phase-code TPUA, Fig. 9(f) show the reconstructed result ${h^{SG}}(x,y)$ combining QGPUA and the Gray-code TPUA, in which both fitting radii of the reconstructed balls are 17.14 mm. The reconstructed results can indicate that ${h^{Sp}}(x,y)$ and ${h^{SG}}(x,y)$ inherit the high measuring accuracy of QGPUA, while overcoming order jumps. Furthermore, the corresponding reliable points for mapping regional unwrapped phases into the absolute phase are (490,1300), (560,450) and (336,914), Table 1 shows the measuring results of three reliable points and two other points with the coordinates (219,784) and (520,283). All measuring results are obtained from the same phase-shifting deformed patterns, once fringe orders of reliable points are determined correctly, all height’s values in Table 1 are exactly the same for each reliable point.

Fig. 9. The analysis of measuring accuracy. (a) measured object, (b) regional wrapped phase, (c)-(d) reconstructed results using the phase-code TPUA and the Gray-code TPUA, (e)-(f) reconstructed results using STPUA based on the phase-code method and Gray-code method.

Download Full Size | PDF

Table 1. Comparison of measuring results for five points

View Table

4.2 Measurement of isolated objects with intricate surfaces

In order to further prove the capability of the proposed STPUA, as shown in Fig. 10(a), the isolated objects containing three plasters with complex surfaces and plastic cap with high reflectivity are measured simultaneously. Figure 10(b) shows the corresponding regional wrapped phase, which can be separately unwrapped to a series of regional unwrapped phase by QGPUA. Then, by selecting the corresponding reliable points, all regional unwrapped phase can be mapped into an absolute phase $\Phi (x,y)$ rather than the point-to-point unwrapping for the wrapped phase. Figures 10(c)–10(d) show the absolute phase maps using the phase-code approach and the Gray-code approach respectively, and the corresponding absolute phases based on the proposed algorithm are shown in Figs. 10(e)–10(f). By comparing these absolute phase maps, it can be clearly seen that the proposed STPUA has high robustness compared with the used TPUAs for measuring the isolated objects with complex surfaces, and it is a general theory for removing the order jumps in various TPUAs. Moreover, Figs. 10(g)–10(h) show the regional reconstructed results marked by the box A₁ and A₂ in Fig. 10(a), which are extracted from Fig. 10(e) and Fig. 10(f). Obviously, the thin character “3M” on a highly reflective bottle cap in the box A₁ and the complex necktie in the box A₂ can be reconstructed well, which reflects the excellent ability of the proposed STPUA to measure the detailed information of the isolated objects.

Fig. 10. The measurement of the intricate objects. (a) measured objects with discontinuous and sharp surfaces, (b) regional wrapped phase, (c)-(d) the absolute phases using the phase-code TPUA and the Gray-code TPUA, (e)-(f) the corresponding absolute phases using STPUA, (g)-(h) the detailed information marked by the box A₁ and A₂ in Fig. 9(a).

Download Full Size | PDF

4.3 Real-time measurement

Because of only projecting four fringe patterns, the phase-code TPUA is applied to measure the dynamic isolated objects containing a rotating fan, as shown in Fig. 11. To minimize the errors generated from motion blur, the four computing deformed patterns are replaced successively to obtain a series of reconstructed results rather than one height can be reconstructed after projecting every four pre-designed phase-shifting patterns. For example, {$I_t^p,I_{t + 1}^p,I_{t + 2}^p,I_{t + 3}^p$} and {$I_{t + 1}^p,I_{t + 2}^p,I_{t + 3}^p,I_{t + 4}^p$} can obtain two adjacent heights, where $I_t^p$ represents the t^th deformed pattern projected by HDLP. The deformed patterns, which are captured in real time by camera at 72 fps, are shown in Visualization 1, Figs. 12(a)–12(c) show the reconstructed results at three different times using the phase-code TPUA, and Figs. 12(d)–12(f) show the reconstructed results at the corresponding times using the proposed STPUA. Due to the radii of reliable points, the proposed STPUA performs higher robustness compared with the conventional phase-code TPUA for the isolated objects with motion blur and noise. Correspondingly, Visualization 2 shows all reconstructed results, and the display rate is 25 fps.

Fig. 11. The isolated objects containing a rotating fan and a fixed plaster.

Download Full Size | PDF

Fig. 12. Measurement of the objects containing a rotating fan. (a)-(c) reconstructed results using the conventional phase-code TPUA at three different times, (d)-(f) reconstructed results using the proposed STPUA at the corresponding times.

Download Full Size | PDF

5. Conclusion

SPUAs utilize neighborhood operation to provide the relative phase information, TPUAs usually use more fringe patterns to recover the absolute phase map. This paper presents a generalized spatial-temporal phase unwrapping algorithm (STPUA), which fuses the advantages of SPUAs and TPUAs. To eliminate order jump errors in various TPUAs, the regional wrapped phases are unwrapped independently using QGPUA, so it inherits the high measuring accuracy of QGPUA. Compared with the conventional point-to-point TPUAs, the regional unwrapped phases can be mapped into an absolute phase map by selecting the corresponding reliable points from fringe order map. Because of existing the radii of reliable points, the proposed STPUA has higher robustness for the objects with complex surfaces and motion blur, while keeping the same number of patterns. A series of experimental results indicate the effectiveness of the proposed algorithm for recovering the absolute phase. Similar to the conventional TPUAs, how to reduce the number of fringe pattern is another interesting direction for further investigation.

Funding

National Major Science and Technology Projects of China (2009ZX02204-008).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Appl. Opt. 22(24), 3977–3982 (1983). [CrossRef]

2. C. Lin, D. Zheng, K. Qian, J. Han, and L. Bai, “Spatial pattern-shifting method for complete two-wavelength fringe projection profilometry,” Opt. Lett. 45(11), 3115–3118 (2020). [CrossRef]

3. X. Liu, T. Tao, Y. Wan, and J. Kofman, “Real-time motion-induced-error compensation in 3D surface-shape measurement,” Opt. Express 27(18), 25265–25279 (2019). [CrossRef]

4. C. Zuo, L. Huang, M. Zhang, Q. Chen, and A. Asundi, “Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review,” Opt. Lasers Eng. 85, 84–103 (2016). [CrossRef]

5. Y. Wang, S. Zhang, and J. Oliver, “3D shape measurement technique for multiple rapidly moving objects,” Opt. Express 19(9), 8539–8545 (2011). [CrossRef]

6. H. An, Y. Cao, H. Li, N. Yang, and J. Chen, “An accuracy improving method for composite grating phase measuring profilometry,” Opt. Commun. 477, 126343 (2020). [CrossRef]

7. S. Yang, G. Wu, Y. Wu, J. Yan, H. Luo, Y. Zhang, and F. Liu, “High-accuracy high-speed unconstrained fringe projection profilometry of 3D measurement,” Opt. Laser Technol. 125, 106063 (2020). [CrossRef]

8. Y. Fu, Z. Wang, B. Li, and J. Zhang, “Three-dimensional shape measurement of complex surfaces based on defocusing and phase-shifting coding,” J. Mod. Optic 62(12), 1015–1026 (2015). [CrossRef]

9. Z. Zhang, S. Huang, S. Meng, F. Gao, and X. Jiang, “A simple, flexible and automatic 3D calibration method for a phase calculation-based fringe projection imaging system,” Opt. Express 21(10), 12218–12227 (2013). [CrossRef]

10. J. Yu, N. Gao, Z. Meng, and Z. Zhang, “A three-dimensional measurement system calibration method based on red/blue orthogonal fringe projection,” Opt. Lasers Eng. 139, 106506 (2021). [CrossRef]

11. B. Li, Z. Liu, and S. Zhang, “Motion-induced error reduction by combining Fourier transform profilometry with phase-shifting profilometry,” Opt. Express 24(20), 23289–23303 (2016). [CrossRef]

12. L. Lu, V. Suresh, Y. Zheng, Y. Wang, J. Xi, and B. Li, “Motion induced error reduction methods for phase shifting profilometry: A review,” Opt. Lasers Eng. 141, 106573 (2021). [CrossRef]

13. C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: A review,” Opt. Lasers Eng. 109, 23–59 (2018). [CrossRef]

14. L. Lu, Y. Yin, Z. Su, X. Ren, Y. Luan, and J. Xi, “General model for phase shifting profilometry with an object in motion,” Appl. Opt. 57(36), 10364–10369 (2018). [CrossRef]

15. S. Zhang, “High-speed 3D shape measurement with structured light methods: A review,” Opt. Lasers Eng. 106, 119–131 (2018). [CrossRef]

16. S. Zhang, “High-resolution 3D profilometry with binary phase-shifting methods,” Appl. Opt. 50(12), 1753–1757 (2011). [CrossRef]

17. M. Dai, K. Peng, J. Zhao, M. Wan, W. Wang, and Y. Cao, “Fast 3D measurement based on improved optical flow for dynamic objects,” Opt. Express 28(13), 18969–18985 (2020). [CrossRef]

18. H. Zhang, H. Su, and X. Su, “Automatic phase unwrapping algorithm for reconstruction of three-dimensional objects,” Opt. Lett. 32(21), 3119–3121 (2007). [CrossRef]

19. L. Meng, S. Fang, P. Yang, L. Wang, M. Komori, and A. Kubo, “Image-inpainting and quality-guided phase unwrapping algorithm,” Appl. Opt. 51(13), 2457–2462 (2012). [CrossRef]

20. Q. Li, C. Bao, J. Zhao, and Z. Jiang, “A New Fast Quality-Guided Flood-Fill Phase Unwrapping Algorithm,” Journal of Physics: Conf. Series.1069, 012182 (2018).

21. K. Chen, J. Xi, and Y. Yu, “Quality-guided spatial phase unwrapping algorithm for fast three-dimensional measurement,” Opt. Commun. 294, 139–147 (2013). [CrossRef]

22. S. Zhang, X. Li, and S. Yau, “Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruction,” Appl. Opt. 46(1), 50–57 (2007). [CrossRef]

23. Y. Wan, Y. Cao, X. Liu, T. Tao, and J. Kofman, “High-frequency color-encoded fringe-projection profilometry based on geometry constraint for large depth range,” Opt. Express 28(9), 13043–13058 (2020). [CrossRef]

24. Y. Wang, L. Liu, J. Wu, X. Chen, and Y. Wang, “Spatial binary coding method for stripe-wise phase unwrapping,” Appl. Opt. 59(14), 4279–4285 (2020). [CrossRef]

25. M. Takeda, Q. Gu, M. Kinoshita, H. Takai, and Y. Takahashi, “Frequency-multiplex Fourier-transform profilometry: a single-shot three-dimensional shape measurement of objects with large height discontinuities and/or surface isolations,” Appl. Opt. 36(22), 5347–5354 (1997). [CrossRef]

26. W. Guo, Z. Wu, R. Xu, Q. Zhang, and M. Fujigaki, “A fast reconstruction method for three-dimensional shape measurement using dual-frequency grating projection and phase-to-height lookup table,” Opt. Laser Technol. 112, 269–277 (2019). [CrossRef]

27. Y. Wang and S. Zhang, “Novel phase-coding method for absolute phase retrieval,” Opt. Lett. 37(11), 2067–2069 (2012). [CrossRef]

28. Y. Wang, L. Liu, J. Wu, X. Song, X. Chen, and Y. Wang, “Dynamic three-dimensional shape measurement with a complementary phase-coding method,” Opt. Lasers Eng. 127, 105982 (2020). [CrossRef]

29. G. Wu, Y. Wu, L. Li, and F. Liu, “High-resolution few-pattern method for 3D optical measurement,” Opt. Lett. 44(14), 3602–3605 (2019). [CrossRef]

30. G. Sansoni, M. Carocci, and R. Rodella, “Three-dimensional vision based on a combination of gray-code and phase-shift light projection: analysis and compensation of the systematic errors,” Appl. Opt. 38(31), 6565–6573 (1999). [CrossRef]

31. Q. Zhang, X. Su, L. Xiang, and X. Sun, “3-D shape measurement based on complementary Gray-code light,” Opt. Lasers Eng. 50(4), 574–579 (2012). [CrossRef]

32. Z. Wu, W. Guo, Y. Li, Y. Liu, and Q. Zhang, “High-speed and high-efficiency three-dimensional shape measurement based on Gray-coded light,” Photon. Res. 8(6), 819–829 (2020). [CrossRef]

33. Q. Ma, Y. Cao, C. Chen, Y. Wan, G. Fu, and Y. Wang, “Intrinsic feature revelation of phase-to-height mapping in phase measuring profilometry,” Opt. Laser Technol. 108, 46–52 (2018). [CrossRef]

34. H. An, Y. Cao, H. Wu, H. Zhang, and H. Li, “The spatial phase-shifting measuring profilometry based on dual-frequency grating,” Opt. Lasers Eng. 143, 106638 (2021). [CrossRef]

35. N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern. 9(1), 62–66 (1979). [CrossRef]

36. L. Xiao, C. Fan, H. Ouyang, A. F. Abate, and S. Wan, “Adaptive trapezoid region intercept histogram based Otsu method for brain MR image segmentation,” J Ambient Intell Human Comput, (2021) https://doi.org/10.1007/s12652-021-02976-6.

37. G. Fu, Y. Cao, Y. Wang, Y. Wan, L. Wang, and C. Li, “Dynamic Phase Measuring Profilometry Based on Tricolor Binary Fringe Encoding Combined Time-Division Multiplexing,” Appl. Sci. 9(4), 813 (2019). [CrossRef]

Points	$K^{p}$	$h^{p}$ /mm	$h^{S p}$ /mm	$K^{G}$	$h^{G}$ /mm	$h^{S G}$ /mm
(490,1300)	20	54.64	54.64	20	54.64	54.64
(560,450)	19	52.28	52.28	19	52.28	52.28
(336,914)	20	33.82	33.82	20	33.82	33.82
(219,784)	21	30.55	30.55	21	30.55	30.55
(520,283)	19	44.84	44.84	19	44.84	44.84

Spatial-temporal phase unwrapping algorithm for fringe projection profilometry

Abstract

1. Introduction

2. Principle

2.1 Description of phase-shifting profilometry

2.2 Principle of phase-code method

2.3 Principle of Gray-code method

2.4 Phase-to-height algorithm

2.5 Discussion

3. Proposed spatial-temporal phase unwrapping algorithm (STPUA)

4. Experiments and results

4.1 Analysis of measuring accuracy

4.2 Measurement of isolated objects with intricate surfaces

4.3 Real-time measurement

5. Conclusion

Funding

Disclosures

Data availability

References

Supplementary Material (2)

Data availability

Cited By

Figures (12)

Tables (1)

Equations (20)

Optics Express

Name	Description
Visualization 1	the captured deformed pattern by camera
Visualization 2	the reconstructed results using the phase-code methed and the proposed STPUA