Fourier analysis of RGB fringe-projection profilometry and robust phase-demodulation methods against crosstalk distortion

Moises Padilla; Manuel Servin; Guillermo Garnica

doi:10.1364/OE.24.015417

1. Introduction

Multichannel red, green and blue (RGB) color encoding is a very useful approach in fringe projection profilometry, which allows simultaneous acquisition of three linearly independent phase-shifted fringe patterns for single-shot measurements [1–6]. Pioneer applications of RGB fringe-projection are due to Wust and Capson [1], who in turn used the phase-demodulation methods by Bruning et al. [7] and Takeda et al. [8]. In RGB fringe-projection profilometry, a digital projector casts simultaneously up-to three sinusoidal phase-shifted fringe patterns over the tridimensional (3D) object under test, and these fringe patterns became phase-modulated proportionally to the height profile of the object [1–6]. Under ideal conditions, we can estimate the searched modulating phase from the RGB encoded data using Bruning’s 3-step phase-shifting algorithm (PSA) [1,7]. However, most color cameras and digital projectors have overlapping spectral sensitivities, which produce RGB-crosstalk distortion [9]. Phase-shifted fringe patterns distorted by RGB crosstalk have non-sinusoidal profiles and their phase-steps differ from the nominal value; this is particularly problematic for few-step algorithms [10–12]. In order to improve the quality of the measured phase proportional to the object’s profile under study, crosstalk compensation is required before feeding the data into the 3-step PSA [1–6]. Other common practices in fringe projection include gamma correction and projection defocusing in order to obtain more sinusoidal profiles [1–6,13–15].

In this paper, we employ the frequency transfer function (FTF) formalism to analyze the robustness of Bruning’s 3-step PSA when applied to fringe patterns distorted by RGB crosstalk. From our analysis, we deduce an equivalent but computationally more efficient method to perform the quadrature filtering in fringe patterns with mild to moderate RGB crosstalk. We also show that Bruning’s 3-step PSA is unreliable for severe RGB crosstalk, since it introduces pseudo-detuning distortion and preserves too many distorting harmonics. When working under these conditions we strongly recommend working with the squeezing interferometry technique (which we also analyze briefly in this paper) for robust quadrature filtering. Finally, we support our conclusions with numerical simulations and by digitizing a static 3D surface (for repetitively, in order to compare all methods discussed in this paper).

2. Theoretical background

2.1 The FTF formalism and Bruning’s 3-step PSAs

For ease of exposition and the reader’s convenience, let us begin with a brief review of the FTF formalism applied to phase-shifting algorithms [12]. Our standard mathematical model for a temporal-carrier fringe pattern is given by:

I (x, y, t) = a (x, y) + b (x, y) \cos [φ (x, y) + ω_{0} t],

where

a (x, y)

represents the background signal,

b (x, y)

is the fringes’ contrast function, and

φ (x, y)

is the searched modulating phase. Finally, the temporal carrier frequency

ω_{0} \in (0, π)

is assumed to be known. Taking the temporal Fourier transform of Eq. (1), we have:

ℱ {I (x, y, t)} = I (x, y, ω) = a (x, y) δ (ω) + \frac{1}{2} b (x, y) [e^{i φ (x, y)} δ (ω - ω_{0}) + e^{- i φ (x, y)} δ (ω + ω_{0})] .

We need to isolate one of the spectrally displaced analytic signals (without loss of generality we choose the one at

ω = ω_{0}

) in order to demodulate the searched phase. We do this using an

N

-step frequency transfer function

H (ω)

that fulfills the quadrature conditions:

H (ω) = \sum_{n = 0}^{N - 1} c_{n} \exp (- i n ω); H (- ω_{0}) = H (0) = 0, H (ω_{0}) \neq 0.

Note that the coefficients

c_{n}

are complex numbers. From Eqs. (2) and (3), we have:

I (x, y, ω) H (ω) = \frac{1}{2} b (x, y) \exp [i φ (x, y)] H (ω_{0}) δ (ω - ω_{0}) .

Back on the temporal domain, we need to evaluate at

t = N - 1

in order to use all available data [12], and using the compact notation

I_{n} (x, y) = I (x, y, t) δ (t - n)

results:

A_{0} (x, y) \exp [i \hat{φ} (x, y)] = {I (x, y, t) * h (t) |}_{t = N - 1} = \sum_{n = 0}^{N - 1} c_{n} I_{n} (x, y),

where

A_{0} (x, y) = (1 / 2) H (ω_{0}) b (x, y)

, and we use a “hat” to label estimated values. Once we isolate this analytic signal, it is easy to solve for all unknowns in Eq. (1) as:

\begin{array}{l} \hat{a} (x, y) = \frac{1}{N} \sum_{n = 0}^{N - 1} I_{n} (x, y); \hat{b} (x, y) = \frac{2}{| H (ω_{0}) |} | \sum_{n = 0}^{N - 1} c_{n} I_{n} (x, y) |; \\ \hat{φ} (x, y) \mod 2 π = \tan^{- 1} {\frac{Im [\sum c_{n} I_{n} (x, y)]}{Re [\sum c_{n} I_{n} (x, y)]}} = \tan^{- 1} {\frac{\sum b_{n} I_{n} (x, y)}{\sum a_{n} I_{n} (x, y)}}, c_{n} = a_{n} + i b_{n} . \end{array}

Since the estimated phase is wrapped within

(- π, π]

, we generally apply a phase unwrapper as last step of the demodulation process [12,16].

Particularly, in this paper we will work with the 3-step PSA by Bruning et al. [7,12], for which $ω_{0} = 2 π / 3$ and $c_{n} = {1, e^{- i 2 π / 3}, e^{- i 4 π / 3}}$ , resulting:

A_{0} (x, y) \exp [i \hat{φ} (x, y)] = \sum c_{n} I_{n} (x, y) = I_{0} (x, y) + e^{- i 2 π / 3} I_{1} (x, y) + e^{- i 4 π / 3} I_{2} (x, y) .

The coefficients

c_{n}

also define the FTF, so we have after some algebraic manipulation:

H (ω) = \sum c_{n} e^{- i n ω} = [1 - e^{- i ω}] [1 - e^{- i (ω + 2 π / 3)}] .

As illustrated in Fig. 1, this FTF fulfills the quadrature conditions (see Eq. (3)), thus rejecting the spectral components at

ω = {0, - 2 π / 3}

and amplifying the searched analytic signal at

ω = 2 π / 3

. This is the ideally expected result.

Fig. 1 Fourier spectra of the temporal-carrier fringe pattern before (on the left side) and after (on the right side) the quadrature linear filtering given by Bruning’s 3-step PSA [7,12].

Download Full Size | PDF

2.2 RGB fringe projection profilometry with 3-step phase shifting

Three-step phase-shifted fringe projection with a single-frequency spatial carrier is arguably the most commonly used approach in RGB fringe-projection profilometry (for alternative approaches see, for instance, Zhang’s review [5] and its references). Our mathematical model for these spatial-carrier open fringes is given by:

\begin{array}{l} f_{0} (x, y) = 255 \times L P F {[^{0.5 + 0.5 \cos (u_{0} x + 0)] 1 / γ}}, \\ f_{1} (x, y) = 255 \times L P F {[^{0.5 + 0.5 \cos (u_{0} x + 2 π / 3)] 1 / γ}}, \\ f_{2} (x, y) = 255 \times L P F {[^{0.5 + 0.5 \cos (u_{0} x + 4 π / 3)] 1 / γ}} . \end{array}

Here

L P F {\cdot}

represents an analog low-pass filter (usually defocusing),

u_{0}

is the frequency of the spatial carrier (in radians per pixel), and the numerical values for

γ

are within the range

(1.6, 2.4)

. The low-pass filtering and the non-linear gamma encoding allow us to attenuate nonlinear intensity distortions during the fringe-projection process [14,15].

Considering the profilometer setup in Fig. 2, one-at-a-time projection of the open fringes in Eq. (9) produces the following phase-shifted fringe-patterns [1]:

\begin{matrix} I_{0} (x, y) = a (x, y) + b (x, y) \cos [φ (x, y) + 0], \\ I_{1} (x, y) = a (x, y) + b (x, y) \cos [φ (x, y) + 2 π / 3]; φ (x, y) = u_{0} \tan (θ) [p (x, y) + x], \\ I_{2} (x, y) = a (x, y) + b (x, y) \cos [φ (x, y) + 4 π / 3] . \end{matrix}

Here

a (x, y)

and

b (x, y)

represent the background illumination signal and the fringes' contrast function, respectively. The searched modulating phase is proportional to the profile of the 3D object under study plus a known term; therefore, it is trivial to solve for

p (x, y)

once we find

φ (x, y)

using Eqs. (6) and (7).

Fig. 2 Schematic of a single-camera single-projector setup for fringe projection profilometry.

Download Full Size | PDF

Using RGB color encoding (see Fig. 3), we can project a single full-color image and generate simultaneously the three phase-shifted fringe patterns in Eq. (10). Without loss of generality we assume $f_{0} (x, y)$ , $f_{1} (x, y)$ , and $f_{2} (x, y)$ being projected in the red, green and blue channel, respectively. In absence of crosstalk distortion (which is feasible using dichromic filters and multiple sensors such as the ones inside of a 3-CCD camera), we can recover the searched fringe patterns from the recorded data in the RGB channels as [1,9]:

Fig. 3 RGB multichannel operation allows us to encode/decode up-to three phase-shifted open-carrier fringes into a single full-color image. Here we show low-frequency fringes for illustrative purposes; a much higher spatial frequency is used on the actual experiments.

Download Full Size | PDF

{\hat{I}}_{0} (x, y) = I_{R} (x, y), {\hat{I}}_{1} (x, y) = I_{G} (x, y), {\hat{I}}_{2} (x, y) = I_{B} (x, y) .

It is a well-known fact that most single-sensor color cameras have overlapping sensibilities [9]. Therefore, when using this kind of cameras for fringe projection profilometry, RGB crosstalk distorts the recorded data and some compensation algorithm is required [2–6]. Assuming the crosstalk phenomenon to be linear, we have:

[\begin{matrix} I_{R} (x, y) \\ I_{G} (x, y) \\ I_{B} (x, y) \end{matrix}] = [\begin{matrix} A_{11} & A_{12} & A_{13} \\ A_{21} & A_{22} & A_{23} \\ A_{31} & A_{32} & A_{33} \end{matrix}] [\begin{matrix} I_{0} (x, y) \\ I_{1} (x, y) \\ I_{2} (x, y) \end{matrix}] .

The real-valued matrix elements

A_{m n}

represent the camera's sensibility on the m-th channel (for

m = {R, G, B}

) to a signal projected in the n-th channel (

n = {R, G, B}

).

Huang et al. [2], proposed a calibration method for matrix $A$ , which was later improved by Zhang et al. [3]. This method consists in generating only red, only green, and only blue phase-shifted open fringes and project them sequentially (instead of simultaneously) over a white reference plane. Due to crosstalk, the CCD camera records these single-color projected fringes on all three RGB channels. Analyzing separately each channel of the recorded data, we can easily calibrate all nine crosstalk coefficients $A_{m n}$ proportionally to the average of the fringes’ contrast function [2,3]. Once we calibrate matrix $A$ , we can easily solve Eq. (12) for the crosstalk-compensated phase-shifted fringe-patterns as:

[\begin{matrix} {\hat{I}}_{0} (x, y) \\ {\hat{I}}_{1} (x, y) \\ {\hat{I}}_{2} (x, y) \end{matrix}] = [\begin{matrix} B_{11} & B_{12} & B_{13} \\ B_{21} & B_{22} & B_{23} \\ B_{31} & B_{32} & B_{33} \end{matrix}] [\begin{matrix} I_{R} (x, y) \\ I_{G} (x, y) \\ I_{B} (x, y) \end{matrix}]; where B = {\hat{A}}^{- 1} .

We can introduce these crosstalk-compensated fringe patterns into Bruning’s 3-step PSA:

A_{0} (x, y) \exp [i \hat{φ} (x, y)] = [\begin{matrix} 1 & e^{- i 2 π / 3} & e^{- i 4 π / 3} \end{matrix}] [\begin{matrix} {\hat{I}}_{0} (x, y) \\ {\hat{I}}_{1} (x, y) \\ {\hat{I}}_{2} (x, y) \end{matrix}] = \sum c_{n} {\hat{I}}_{n} (x, y) .

However, from Eqs. (13) and (14), it is easy to see that an equivalent but computationally more efficient approach to demodulate the searched analytic signal is given by:

\begin{matrix} A_{0} (x, y) \exp [i \hat{φ} (x, y)] = d_{0} I_{R} (x, y) + d_{1} I_{G} (x, y) + d_{2} I_{B} (x, y); \\ [\begin{matrix} d_{0} & d_{1} & d_{2} \end{matrix}] = [\begin{matrix} 1 & e^{- i 2 π / 3} & e^{- i 4 π / 3} \end{matrix}] [\begin{matrix} B_{11} & B_{12} & B_{13} \\ B_{21} & B_{22} & B_{23} \\ B_{31} & B_{32} & B_{33} \end{matrix}] . \end{matrix}

The complex-valued coefficients

d_{n}

need to be computed only once for fixed experimental conditions. The above equation requires 4 operations per pixel and it works directly over the raw data

{I_{R}, I_{G}, I_{B}}

. On the other hand, Eq. (14) required 16 operations per pixel and memory allocation for the intermediate results

{{\hat{I}}_{0}, {\hat{I}}_{1}, {\hat{I}}_{2}}

. Since RGB fringe-projection profilometry is meant for dynamic measurements [1–6], this represents a substantial reduction on the computational workload. As far as we know, Eq. (15) has never been reported before so this is our first contribution on this paper.

3. Influence of systematic errors on the phase-demodulation algorithm

Here we illustrate how systematic errors produce low-quality phase estimations when applying Bruning’s 3-step PSA under non-ideal conditions, such as residual errors during the calibration of the crosstalk coefficients and distorting harmonics.

We start by modeling our estimation for the crosstalk matrix as $\hat{A} = A + δ$ , where $A$ describes the actual crosstalk values (see Eq. (12)), $δ$ is unknown, and for small errors we assume $| δ_{m n} | < < 1$ . From Eqs. (13)-(15), taking $B I_{R G B} = B (A I_{n}) = B (\hat{A} - δ) I_{n}$ , we have:

A_{0} (x, y) \exp (i \hat{φ}) = \sum c'_{n} I_{n} (x, y); {[\begin{matrix} c'_{0} \\ c'_{1} \\ c'_{2} \end{matrix}]}^{T} = {[\begin{matrix} 1 \\ e^{- i 2 π / 3} \\ e^{- i 4 π / 3} \end{matrix}]}^{T} [\begin{matrix} 1 + ε_{11} & ε_{12} & ε_{13} \\ ε_{21} & 1 + ε_{22} & ε_{23} \\ ε_{31} & ε_{32} & 1 + ε_{33} \end{matrix}],

where

ε = - B δ

; therefore

ε_{m n}

are unknown, and

| ε_{m n} | < 1

because

| δ_{m n} | < < 1

.

As we know from Eq. (5), the coefficients $c'_{n}$ define univocally not only the phase-demodulation formula but also the FTF: $H (ω) = \sum c'_{n} \exp (- i n ω)$ . In Fig. 4 we show several computer-simulated realizations of these FTFs, where $ε_{m n}$ are uniformly distributed random numbers within $(- 0.15, 0.15)$ . We must remark that these plots still correspond to Bruning’s 3-step PSA but being applied to fringe patterns distorted by RGB crosstalk.

Fig. 4 Left side shows three FTFs given by $H (ω) = \sum c'_{n} \exp (- i n ω)$ using uniformly distributed random values $ε_{m n} \in (- 0.15, 0.15)$ ; each color represents a different realization. Right side shows the Fourier spectrum of the analytic signal estimated using one of these FTFs.

Download Full Size | PDF

From Fig. 4 we see that $H (ω) = \sum c'_{n} \exp (- i n ω)$ fails to fulfill the quadrature conditions $H (0) = 0$ and $H (- ω_{0}) = 0$ . Thus, the estimated analytic signal is given by:

A_{0} (x, y) e^{i \hat{φ} (x, y)} = H (0) a (x, y) + \frac{1}{2} b (x, y) {H (2 π / 3) e^{i φ (x, y)} + H (- 2 π / 3) e^{- i φ (x, y)}} .

Carefully controlled experimental conditions and/or fringe pattern normalization allow us to eliminate the background signal term [13]. Thus, taking

a (x, y) \to 0

, we have:

A_{0} (x, y) e^{i \hat{φ} (x, y)} = \frac{1}{2} H (2 π / 3) b (x, y) e^{i φ (x, y)} {1 + \frac{H (- 2 π / 3)}{H (2 π / 3)} e^{i 2 φ (x, y)}} .

For the trained eye, Eq. (18) predicts that the wrapped phase will be distorted by a double-frequency spurious signal, which is identical to the well-known distortion due to detuning error in phase-shifting interferometry [12]. We performed a numeric simulation to assess this pseudo-detuning distortion (see Fig. 5). First, we modulated three phase-shifted fringe patterns with a known phase. Next, we estimated the analytic signal according to Eq. (16) using random values for the crosstalk calibration uncertainty such as

| ε_{m n} | \leq {0.05, 0.10, 0.15}

. Finally, we computed the wrapped difference between the actual phase and the estimated one, revealing the predicted double-frequency spurious signal. In this simulation, we used a low-frequency spatial carrier for ease of observation.

Fig. 5 The first row shows the phase map (given by MATLAB’s peaks function) modulating three phase-shifted patterns of open fringes. The second row shows the estimated phase as obtained from the 3-step PSA for different amounts of crosstalk-calibration error. The third row shows the wrapped difference $Δ φ = (φ - \hat{φ}) \mod 2 π$ , multiplied by a factor 10 for ease of observation, revealing the predicted double-frequency ripple pattern. The intensity scale for the wrapped phases follows the standard linear mapping from $- π$ (black) to $π$ (white).

Download Full Size | PDF

Up to this point, we have modeled perfectly sinusoidal fringe patterns for ease of exposition but, in practice, these fringe patterns may be distorted by harmonics due to gain saturation, poorly estimated gamma, and/or non-lineal RGB crosstalk [12]:

\begin{array}{l} I (x, y, t) = \sum_{n = - N}^{N} b_{n} (x, y) \exp {i n [φ (x, y) + (2 π / 3) t]}, \\ I (x, y, ω) = \sum_{n = - N}^{N} b_{n} (x, y) \exp [i n φ (x, y)] δ (ω - ω_{n}); ω_{n} = \arg [\exp (i 2 n π / 3)] . \end{array}

Here

N

represents the order of the highest harmonic with still-significant distorting energy,

b_{n} (x, y)

is the contrast function of the

n

-th harmonic, and the wrapped location of the Dirac deltas is due the

2 π -

periodicity of the discrete-time Fourier transform [12]. When the fringe pattern is distorted by harmonics, we want to isolate

b_{1} \exp (i φ)

in order to find the best-quality estimation of the modulating phase. However, since

H (ω) = \sum c'_{n} \exp (- i n ω)

fails to fulfill the quadrature conditions

H (- 2 π / 3) = H (0) = 0

, we have:

\begin{matrix} I (ω) H (ω) = H (2 π / 3) {b_{1} e^{i φ} + b_{2} e^{- i 2 φ} + b_{4} e^{i 4 φ} + b_{5} e^{- i 5 φ} + ...} δ (ω - 2 π / 3) \\ + H (0) {b_{0} + b_{3} e^{i 3 φ} + b_{3} e^{- i 3 φ} + b_{6} e^{i 6 φ} + b_{6} e^{- i 6 φ} + ...} δ (ω) \\ + H (- 2 π / 3) {b_{1} e^{- i φ} + b_{2} e^{i 2 φ} + b_{4} e^{- i 4 φ} + b_{5} e^{i 5 φ} + ...} δ (ω + 2 π / 3) . \end{matrix}

It is easy to see from the above equation that back on the temporal domain we will have an analytic signal distorted by pseudo-detuning (because

b_{1} \exp (- i φ)

remains unfiltered), and corrupted by the distorting harmonics with the highest energy (both components of the second one, both components of the third one, and so on). Therefore, it is clear that we require a more robust approach to cope with non-sinusoidal fringes and severe RGB crosstalk.

4. Robust quadrature filtering for crosstalk-distorted fringe patterns

As we know from the FTF formalism (see Appendix A in [12]), the PSAs with detuning error robustness require four or more phase-shifted interferograms. Nevertheless, the squeezing interferometry method allows us to achieve detuning robust quadrature filtering and decrease the harmonics distortion using just three phase-shifted samples [10].

The first step in squeezing interferometry is to rearrange the phase-shifted fringe patterns ( ${{\hat{I}}_{0}, {\hat{I}}_{1}, {\hat{I}}_{2}}$ in our case), in order to produce a spatial-carrier fringe pattern with very high spatial-carrier frequency as follows:

\begin{matrix} I_{S} (3 x - 2, y) = {\hat{I}}_{0} (x, y) \\ I_{S} (3 x - 1, y) = {\hat{I}}_{1} (x, y) \\ I_{S} (3 x - 0, y) = {\hat{I}}_{2} (x, y) \end{matrix}} (0, 0) \leq (x, y) \leq (X, Y),

where

(X, Y)

represent the horizontal and vertical sizes of

{\hat{I}}_{n} (x, y)

. Rewriting the resulting spatial-carrier fringe pattern in terms of

x' = 3 x

, we have:

I_{S} (x', y) = \sum_{n = - N}^{N} b_{n} (x', y) \exp {i n [φ (x', y) + 2 π x' / 3]}; (0, 0) \leq (x', y) \leq (3 X, Y) .

And computing its discrete Fourier transform results:

ℱ {I_{S} (x', y)} = I_{S} (u, v)

,

I_{S} (u, v) = \sum_{n = - N}^{N} ℱ {b_{n} (x', y) \exp [i n φ (x', y)]} * δ (u - u_{n}, v); u_{n} = \arg [\exp (i 2 n π / 3)] .

The asterisk represents the convolution operation, and we are applying the translation property of Dirac delta:

f (u, v) * δ (u - Δ, v) = f (u - Δ, v)

. Factorizing the spectral lobes by their spectral displacement, we can rewrite the above equation as:

\begin{matrix} I_{S} (u, v) = ℱ {b_{1} e^{i φ} + b_{2} e^{- i 2 φ} + b_{4} e^{i 4 φ} + b_{5} e^{- i 5 φ} + b_{7} e^{i 7 φ} + ...} * δ (u - 2 π / 3, v) \\ + ℱ {b_{0} + b_{3} e^{i 3 φ} + b_{3} e^{- i 3 φ} + b_{5} e^{i 6 φ} + b_{5} e^{- i 6 φ} + ...} * δ (u, v) \\ + ℱ {b_{1} e^{- i φ} + b_{2} e^{i 2 φ} + b_{4} e^{- i 4 φ} + b_{5} e^{i 5 φ} + b_{7} e^{- i 7 φ} + ...} * δ (u + 2 π / 3, v) . \end{matrix}

Comparing Eqs. (20) and (24), it is clear that the temporal and spatial carriers produce similar spectral distributions, but there are significant differences. First, the energy of

ℱ {b_{n} e^{\pm i n φ}}

spreads over a spectral region

n^{2}

times as large as the bandwidth of

ℱ {b_{1} e^{i φ}}

; thus, the

n

-th harmonic has much lower spectral density than the searched analytic signal. Also, processing in the spatial Fourier domain is immune to detuning error (and/or pseudo-detuning) because we can filter-out all signals outside the area of interest, instead of just the components exactly at

(- 2 π / 3, 0)

and

(0, 0)

. As illustrated in Fig. 6, this means we can achieve robust quadrature filtering with the following synchronous demodulation formula [10,11]:

A_{0} (x', y) \exp [i \hat{φ} (x', y)] = \exp (- i 2 π x' / 3) \times Q F {I_{S} (x', y)} .

Here

Q F {\cdot}

represents a narrow quadrature filter in the Fourier domain (usually a binary mask) which preserves only the first harmonic of the analytic signal. Once we estimate

A_{0} (x', y) \exp [i \hat{φ} (x', y)]

, we squeeze it back to the original size of

{\hat{I}}_{n} (x, y)

; finally, we compute its argument and unwrap the modulating phase, as usual [10,11].

Fig. 6 Schematic illustration of Fourier demodulation for the spatial-carrier fringe pattern $I_{S} (u, v)$ in Eq. (24). The black circle represents a binary mask band-passing the searched analytic signal (red pentagon) and rejecting all unwanted distorting harmonic components.

Download Full Size | PDF

5. Experimental results

In our experiment, we worked with a static test subject for repeatability: digitizing a human-face model made of expanded polystyrene (see Fig. 7). This allows us to take a crosstalk-free measurement using sequential grayscale-based fringe projection as null test. We used this null test to assess the quality of the recovered phases from single-shot RGB fringe projection using Bruning’s 3-step PSA and squeezing interferometry. The open fringes were projected using a $1024 \times 768$ digital light projector, with defocusing and pre-encoding gamma [14,15]. The fringe patterns were captured using a single-sensor digital color camera that performs RGB color separation using a Bayer filter mosaic with a $1280 \times 768$ resolution and 8-bit color depth per channel. We calibrated the crosstalk matrix using 6-step least-squares phase-shifting measurements of the reference plane for increased reliability (to generalize from [2,3] is straightforward). Our numerical values for the crosstalk calibration matrix $\hat{A}$ (see Eqs. (12) and (13)) and for the corresponding phase demodulation coefficients $d_{n}$ , required to work directly with raw data (see Eq. (15)), are given by:

\hat{A} = (\begin{matrix} 0.4334 & 0.4041 & 0.0749 \\ 0.0791 & 0.9092 & 0.3316 \\ 0.0007 & 0.3679 & 0.9536 \end{matrix}), (\begin{matrix} d_{0} \\ d_{1} \\ d_{2} \end{matrix}) = (\begin{matrix} 2.2252 - i 0.3585 \\ 0.4581 + i 1.9615 \\ - 0.8586 + i 0.2542 \end{matrix}) .

According to these numerical values, we faced severe RGB-crosstalk distortion in comparison to other calibration matrices reported in the literature [2,3,6].

Fig. 7 Photograph of our test subject being illuminated with RGB phase-shifted open fringes. The decoded RGB images are shown in gray levels.

Download Full Size | PDF

Figure 8 shows the first step of the squeezing interferometry method [10], where we rearrange the information from the phase-shifted fringe patterns ${{\hat{I}}_{0} (x, y), {\hat{I}}_{1} (x, y), {\hat{I}}_{2} (x, y)}$ in order to generate the very high-frequency spatial-carrier fringe pattern $I_{S} (x', y)$ . The beating distortion observed in $I_{S} (x', y)$ is a presentation artifact (nonexistent in the actual data) as consequence of the down sampling from the original size ( $3840 \times 768$ pixels).

Fig. 8 First stage of the squeezing interferometry method as described in Eq. (21).

Download Full Size | PDF

The next step is to compute $ℱ {I_{S} (x', y)}$ , from where we isolate the searched analytic signal by means of a narrow one-sided band-pass filter. Back on the spatial domain, we squeezed the analytic signal back to the original size of $I_{n} (x, y)$ , and then we estimated the wrapped phase from its argument. For ease of visualization, at this point we removed the low-frequency modulation $u_{0} x \tan (θ)$ , previously calibrated from the reference plane. See Fig. 9.

Fig. 9 Last steps of the squeezing interferometry method: quadrature filtering of $ℱ {I_{S} (x', y)}$ ; wrapped phase estimated as the argument of the analytic signal inside the black circle; and the unwrapped phase proportional to the profile of our test subject.

Download Full Size | PDF

Figure 10 shows the recovered profiles from the RGB raw data, the crosstalk compensated fringe patterns, squeezing interferometry, and our null-test for illustrative purposes. These plots use the amplitude of the analytic signal as texture in order to visualize fine details.

Fig. 10 Tridimensional render of the digitized object with RGB fringe projection profilometry. These panels are described in the main text. Units of all axes are in millimeters.

Download Full Size | PDF

Figure 10(a) shows the 3D profile recovered from the raw camera data, where crosstalk-distortion was so severe that the phase unwrapper failed to recover even the overall shape of the test subject. Figure 10(b) shows the result of processing the same raw data using our proposed algorithm (Eq. (15)); this result shows a drastic improvement respect to Fig. 10(a) but still exhibits the ripple pattern due to pseudo-detuning error and distorting harmonics. Figure 10(c) shows our best quality result, which we obtained with the combination of crosstalk-compensation and squeezing interferometry. Finally, Fig. 10(d) shows the result of our crosstalk-free null test for illustrative purposes; this result was obtained by means of sequential gray-scale based fringe projection.

Figure 11 shows horizontal slices of the unwrapped phase for qualitative comparison of the results estimated with Bruning’s 3-step PSA and with squeezing interferometry. This figure also demonstrates that the ripple pattern due to pseudo-detuning and distorting harmonics is observed on the modulated phase.

Fig. 11 Horizontal slices of the unwrapped phase estimated with Burning’s 3-step PSA, the squeezing interferometry method, and the crosstalk-free null test. A global piston was included between each plot for ease of visualization.

Download Full Size | PDF

6. Conclusions

We analyzed the most commonly used phase-demodulation approaches for RGB fringe-projection profilometry by means of the FTF formalism. We proposed a computationally more efficient algorithm applicable for mild to moderate linear RGB-crosstalk conditions, which combines crosstalk compensation and phase demodulation, and that works directly with the raw data (Eq. (15)). We show that relying on Bruning’s 3-step PSA for severe crosstalk conditions (when modeling this phenomenon using linear combinations no longer applies) translates into a pseudo-detuning distortion and high-energy distorting harmonics. The pseudo-detuning error was somehow unexpected because the phase-step is a controlled parameter on fringe projection techniques. We also analyzed the squeezing interferometry phase-demodulation method to show that it allows us to recover high-quality results even with non-sinusoidal fringes and under severe crosstalk distortion. Finally, we supported our claims with numerical simulations and experimental results.

Acknowledgments

The authors want to thank the financial support from the Mexican National Council for Science and Technology (CONACYT), grant CB2010-157044. The authors also acknowledge Cornell University for supporting the e-print repository arXiv.org and the Optical Society of America for allowing the contributors to post their manuscript at arXiv.

References and links

1. C. Wust and D. W. Capson, “Surface profile measurement using color fringe projection,” Mach. Vis. Appl. 4(3), 193–203 (1991). [CrossRef]

2. P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng. 38(6), 1065–1071 (1999). [CrossRef]

3. Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency Selection,” Opt. Express 14(14), 6444–6455 (2006). [CrossRef] [PubMed]

4. S. Ma, R. Zhu, C. Quan, B. Li, C. J. Tay, and L. Chen, “Blind phase error suppression for color-encoded digital fringe projection profilometry,” Opt. Commun. 285(7), 1662–1668 (2012). [CrossRef]

5. Z. H. Zhang, “Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques,” Opt. Lasers Eng. 50(8), 1097–1106 (2012). [CrossRef]

6. J. L. Flores, J. A. Ferrari, G. García Torales, R. Legarda-Saenz, and A. Silva, “Color-fringe pattern profilometry using a generalized phase-shifting algorithm,” Appl. Opt. 54(30), 8827–8834 (2015). [CrossRef] [PubMed]

7. J. H. Bruning, D. R. Herriott, J. E. Gallagher, D. P. Rosenfeld, A. D. White, and D. J. Brangaccio, “Digital wavefront measuring interferometer for testing optical surfaces and lenses,” Appl. Opt. 13(11), 2693–2703 (1974). [CrossRef] [PubMed]

8. M. Takeda, H. Ina, and S. Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. 72(1), 156–160 (1982). [CrossRef]

9. T. Katsuta, F. Ando, M. Kida, and M. Kawasaki, “Optical system for tricolor separation,” U.S. Patent: 3 602 637, issued date August 31, 1971.

10. M. Servin, M. Cywiak, D. Malacara-Hernandez, J. C. Estrada, and J. A. Quiroga, “Spatial carrier interferometry from M temporal phase shifted interferograms: Squeezing Interferometry,” Opt. Express 16(13), 9276–9283 (2008). [CrossRef] [PubMed]

11. B. Li, L. Chen, W. Tuya, S. Ma, and R. Zhu, “Carrier squeezing interferometry: suppressing phase errors from the inaccurate phase shift,” Opt. Lett. 36(6), 996–998 (2011). [CrossRef] [PubMed]

12. M. Servin, J. A. Quiroga, and J. M. Padilla, Fringe Pattern Analysis for Optical Metrology: Theory, Algorithms, and Applications (Wiley-VCH, Weinheim, 2014).

13. J. A. Quiroga and M. Servin, “Isotropic n-dimensional fringe pattern normalization,” Opt. Commun. 224(4-6), 221–227 (2003). [CrossRef]

14. H. Guo, H. He, and M. Chen, “Gamma correction for digital fringe projection profilometry,” Appl. Opt. 43(14), 2906–2914 (2004). [CrossRef] [PubMed]

15. S. Lei and S. Zhang, “Flexible 3-D shape measurement using projector defocusing,” Opt. Lett. 34(20), 3080–3082 (2009). [CrossRef] [PubMed]

16. D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (Wiley, 1998).

Fourier analysis of RGB fringe-projection profilometry and robust phase-demodulation methods against crosstalk distortion

Abstract

1. Introduction

2. Theoretical background

2.1 The FTF formalism and Bruning’s 3-step PSAs

2.2 RGB fringe projection profilometry with 3-step phase shifting

3. Influence of systematic errors on the phase-demodulation algorithm

4. Robust quadrature filtering for crosstalk-distorted fringe patterns

5. Experimental results

6. Conclusions

Acknowledgments

References and links

Cited By

Figures (11)

Equations (26)

Optics Express