## Abstract

In this paper, we apply the frequency transfer function formalism to analyze the red, green and blue (RGB) phase-shifting fringe-projection profilometry technique. The phase-shifted patterns in RGB fringe projection are typically corrupted by crosstalk because the sensitivity curves of most projection-recording systems overlap. This crosstalk distortion needs to be compensated in order to obtain high quality measurements. We study phase-demodulation methods for null/mild, moderate, and severe levels of RGB crosstalk. For null/mild crosstalk distortion, we can estimate the searched phase-map using Bruning’s 3-step phase-shifting algorithm (PSA). For moderate crosstalk, the recorded data is usually preprocessed before feeding it into the PSA; alternatively, in this paper we propose a computationally more efficient approach, which combines linear crosstalk compensation with the phase-demodulation algorithm. For severe RGB crosstalk, we expect non-sinusoidal fringes’ profiles (distorting harmonics) and a significant uncertainty on the linear crosstalk calibration (which produces pseudo-detuning error). Analyzing these distorting phenomena, we conclude that squeezing interferometry is the most robust demodulation method for RGB fringe-projection techniques. Finally, we support our conclusions with numerical simulations and experimental results.

© 2016 Optical Society of America

## 1. Introduction

Multichannel red, green and blue (RGB) color encoding is a very useful approach in fringe projection profilometry, which allows simultaneous acquisition of three linearly independent phase-shifted fringe patterns for single-shot measurements [1–6]. Pioneer applications of RGB fringe-projection are due to Wust and Capson [1], who in turn used the phase-demodulation methods by Bruning et al. [7] and Takeda et al. [8]. In RGB fringe-projection profilometry, a digital projector casts simultaneously up-to three sinusoidal phase-shifted fringe patterns over the tridimensional (3D) object under test, and these fringe patterns became phase-modulated proportionally to the height profile of the object [1–6]. Under ideal conditions, we can estimate the searched modulating phase from the RGB encoded data using Bruning’s 3-step phase-shifting algorithm (PSA) [1,7]. However, most color cameras and digital projectors have overlapping spectral sensitivities, which produce RGB-crosstalk distortion [9]. Phase-shifted fringe patterns distorted by RGB crosstalk have non-sinusoidal profiles and their phase-steps differ from the nominal value; this is particularly problematic for few-step algorithms [10–12]. In order to improve the quality of the measured phase proportional to the object’s profile under study, crosstalk compensation is required before feeding the data into the 3-step PSA [1–6]. Other common practices in fringe projection include gamma correction and projection defocusing in order to obtain more sinusoidal profiles [1–6,13–15].

In this paper, we employ the frequency transfer function (FTF) formalism to analyze the robustness of Bruning’s 3-step PSA when applied to fringe patterns distorted by RGB crosstalk. From our analysis, we deduce an equivalent but computationally more efficient method to perform the quadrature filtering in fringe patterns with mild to moderate RGB crosstalk. We also show that Bruning’s 3-step PSA is unreliable for severe RGB crosstalk, since it introduces pseudo-detuning distortion and preserves too many distorting harmonics. When working under these conditions we strongly recommend working with the squeezing interferometry technique (which we also analyze briefly in this paper) for robust quadrature filtering. Finally, we support our conclusions with numerical simulations and by digitizing a static 3D surface (for repetitively, in order to compare all methods discussed in this paper).

## 2. Theoretical background

#### 2.1 The FTF formalism and Bruning’s 3-step PSAs

For ease of exposition and the reader’s convenience, let us begin with a brief review of the FTF formalism applied to phase-shifting algorithms [12]. Our standard mathematical model for a temporal-carrier fringe pattern is given by:

where $a(x,y)$ represents the background signal, $b(x,y)$ is the fringes’ contrast function, and $\phi (x,y)$ is the searched modulating phase. Finally, the temporal carrier frequency ${\omega}_{0}\in (0,\pi )$ is assumed to be known. Taking the temporal Fourier transform of Eq. (1), we have:Particularly, in this paper we will work with the 3-step PSA by Bruning et al. [7,12], for which ${\omega}_{0}=2\pi /3$ and ${c}_{n}=\{1,\text{\hspace{0.17em}}{e}^{-i2\pi /3},\text{\hspace{0.17em}}{e}^{-i4\pi /3}\}$, resulting:

#### 2.2 RGB fringe projection profilometry with 3-step phase shifting

Three-step phase-shifted fringe projection with a single-frequency spatial carrier is arguably the most commonly used approach in RGB fringe-projection profilometry (for alternative approaches see, for instance, Zhang’s review [5] and its references). Our mathematical model for these spatial-carrier open fringes is given by:

Considering the profilometer setup in Fig. 2, one-at-a-time projection of the open fringes in Eq. (9) produces the following phase-shifted fringe-patterns [1]:

Using RGB color encoding (see Fig. 3), we can project a single full-color image and generate simultaneously the three phase-shifted fringe patterns in Eq. (10). Without loss of generality we assume ${f}_{0}(x,y)$, ${f}_{1}(x,y)$, and ${f}_{2}(x,y)$ being projected in the red, green and blue channel, respectively. In absence of crosstalk distortion (which is feasible using dichromic filters and multiple sensors such as the ones inside of a 3-CCD camera), we can recover the searched fringe patterns from the recorded data in the RGB channels as [1,9]:

It is a well-known fact that most single-sensor color cameras have overlapping sensibilities [9]. Therefore, when using this kind of cameras for fringe projection profilometry, RGB crosstalk distorts the recorded data and some compensation algorithm is required [2–6]. Assuming the crosstalk phenomenon to be linear, we have:

*m*-th channel (for $m=\{R,G,B\}$) to a signal projected in the

*n*-th channel ($n=\{R,G,B\}$).

Huang et al. [2], proposed a calibration method for matrix $A$, which was later improved by Zhang et al. [3]. This method consists in generating only red, only green, and only blue phase-shifted open fringes and project them sequentially (instead of simultaneously) over a white reference plane. Due to crosstalk, the CCD camera records these single-color projected fringes on all three RGB channels. Analyzing separately each channel of the recorded data, we can easily calibrate all nine crosstalk coefficients ${A}_{mn}$ proportionally to the average of the fringes’ contrast function [2,3]. Once we calibrate matrix $A$, we can easily solve Eq. (12) for the crosstalk-compensated phase-shifted fringe-patterns as:

## 3. Influence of systematic errors on the phase-demodulation algorithm

Here we illustrate how systematic errors produce low-quality phase estimations when applying Bruning’s 3-step PSA under non-ideal conditions, such as residual errors during the calibration of the crosstalk coefficients and distorting harmonics.

We start by modeling our estimation for the crosstalk matrix as $\widehat{A}=A+\delta $, where $A$ describes the actual crosstalk values (see Eq. (12)), $\delta $ is unknown, and for small errors we assume $\left|{\delta}_{mn}\right|<<1$. From Eqs. (13)-(15), taking $B{I}_{RGB}=B(A{I}_{n})=B(\widehat{A}-\delta ){I}_{n}$, we have:

As we know from Eq. (5), the coefficients $c{\text{'}}_{n}$ define univocally not only the phase-demodulation formula but also the FTF: $H(\omega )={\displaystyle \sum c{\text{'}}_{n}\mathrm{exp}(-in\omega )}$. In Fig. 4 we show several computer-simulated realizations of these FTFs, where ${\epsilon}_{mn}$ are uniformly distributed random numbers within $(-0.15,0.15)$. We must remark that these plots still correspond to Bruning’s 3-step PSA but being applied to fringe patterns distorted by RGB crosstalk.

From Fig. 4 we see that $H(\omega )={\displaystyle \sum c{\text{'}}_{n}\mathrm{exp}(-in\omega )}$ fails to fulfill the quadrature conditions $H(0)=0$ and $H(-{\omega}_{0})=0$. Thus, the estimated analytic signal is given by:

Up to this point, we have modeled perfectly sinusoidal fringe patterns for ease of exposition but, in practice, these fringe patterns may be distorted by harmonics due to gain saturation, poorly estimated gamma, and/or non-lineal RGB crosstalk [12]:

## 4. Robust quadrature filtering for crosstalk-distorted fringe patterns

As we know from the FTF formalism (see Appendix A in [12]), the PSAs with detuning error robustness require four or more phase-shifted interferograms. Nevertheless, the squeezing interferometry method allows us to achieve detuning robust quadrature filtering and decrease the harmonics distortion using just three phase-shifted samples [10].

The first step in squeezing interferometry is to rearrange the phase-shifted fringe patterns ($\{{\widehat{I}}_{0},{\widehat{I}}_{1},{\widehat{I}}_{2}\}$ in our case), in order to produce a spatial-carrier fringe pattern with very high spatial-carrier frequency as follows:

## 5. Experimental results

In our experiment, we worked with a static test subject for repeatability: digitizing a human-face model made of expanded polystyrene (see Fig. 7). This allows us to take a crosstalk-free measurement using *sequential* grayscale-based fringe projection as null test. We used this null test to assess the quality of the recovered phases from single-shot RGB fringe projection using Bruning’s 3-step PSA and squeezing interferometry. The open fringes were projected using a $1024\times 768$ digital light projector, with defocusing and pre-encoding gamma [14,15]. The fringe patterns were captured using a single-sensor digital color camera that performs RGB color separation using a Bayer filter mosaic with a $1280\times 768$ resolution and 8-bit color depth per channel. We calibrated the crosstalk matrix using 6-step least-squares phase-shifting measurements of the reference plane for increased reliability (to generalize from [2,3] is straightforward). Our numerical values for the crosstalk calibration matrix $\widehat{A}$ (see Eqs. (12) and (13)) and for the corresponding phase demodulation coefficients ${d}_{n}$, required to work directly with raw data (see Eq. (15)), are given by:

Figure 8 shows the first step of the squeezing interferometry method [10], where we rearrange the information from the phase-shifted fringe patterns $\{{\widehat{I}}_{0}(x,y),{\widehat{I}}_{1}(x,y),{\widehat{I}}_{2}(x,y)\}$ in order to generate the very high-frequency spatial-carrier fringe pattern ${I}_{S}(x\text{'},y)$. The beating distortion observed in ${I}_{S}(x\text{'},y)$ is a presentation artifact (nonexistent in the actual data) as consequence of the down sampling from the original size ($3840\times 768$ pixels).

The next step is to compute $\mathcal{F}\left\{{I}_{S}(x\text{'},y)\right\}$, from where we isolate the searched analytic signal by means of a narrow one-sided band-pass filter. Back on the spatial domain, we squeezed the analytic signal back to the original size of ${I}_{n}(x,y)$, and then we estimated the wrapped phase from its argument. For ease of visualization, at this point we removed the low-frequency modulation ${u}_{0}x\mathrm{tan}(\theta )$, previously calibrated from the reference plane. See Fig. 9.

Figure 10 shows the recovered profiles from the RGB raw data, the crosstalk compensated fringe patterns, squeezing interferometry, and our null-test for illustrative purposes. These plots use the amplitude of the analytic signal as texture in order to visualize fine details.

Figure 10(a) shows the 3D profile recovered from the raw camera data, where crosstalk-distortion was so severe that the phase unwrapper failed to recover even the overall shape of the test subject. Figure 10(b) shows the result of processing the same raw data using our proposed algorithm (Eq. (15)); this result shows a drastic improvement respect to Fig. 10(a) but still exhibits the ripple pattern due to pseudo-detuning error and distorting harmonics. Figure 10(c) shows our best quality result, which we obtained with the combination of crosstalk-compensation and squeezing interferometry. Finally, Fig. 10(d) shows the result of our crosstalk-free null test for illustrative purposes; this result was obtained by means of *sequential* gray-scale based fringe projection.

Figure 11 shows horizontal slices of the unwrapped phase for qualitative comparison of the results estimated with Bruning’s 3-step PSA and with squeezing interferometry. This figure also demonstrates that the ripple pattern due to pseudo-detuning and distorting harmonics is observed on the modulated phase.

## 6. Conclusions

We analyzed the most commonly used phase-demodulation approaches for RGB fringe-projection profilometry by means of the FTF formalism. We proposed a computationally more efficient algorithm applicable for mild to moderate linear RGB-crosstalk conditions, which combines crosstalk compensation and phase demodulation, and that works directly with the raw data (Eq. (15)). We show that relying on Bruning’s 3-step PSA for severe crosstalk conditions (when modeling this phenomenon using linear combinations no longer applies) translates into a pseudo-detuning distortion and high-energy distorting harmonics. The pseudo-detuning error was somehow unexpected because the phase-step is a controlled parameter on fringe projection techniques. We also analyzed the squeezing interferometry phase-demodulation method to show that it allows us to recover high-quality results even with non-sinusoidal fringes and under severe crosstalk distortion. Finally, we supported our claims with numerical simulations and experimental results.

## Acknowledgments

The authors want to thank the financial support from the Mexican National Council for Science and Technology (CONACYT), grant CB2010-157044. The authors also acknowledge Cornell University for supporting the e-print repository arXiv.org and the Optical Society of America for allowing the contributors to post their manuscript at arXiv.

## References and links

**1. **C. Wust and D. W. Capson, “Surface profile measurement using color fringe projection,” Mach. Vis. Appl. **4**(3), 193–203 (1991). [CrossRef]

**2. **P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng. **38**(6), 1065–1071 (1999). [CrossRef]

**3. **Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency Selection,” Opt. Express **14**(14), 6444–6455 (2006). [CrossRef] [PubMed]

**4. **S. Ma, R. Zhu, C. Quan, B. Li, C. J. Tay, and L. Chen, “Blind phase error suppression for color-encoded digital fringe projection profilometry,” Opt. Commun. **285**(7), 1662–1668 (2012). [CrossRef]

**5. **Z. H. Zhang, “Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques,” Opt. Lasers Eng. **50**(8), 1097–1106 (2012). [CrossRef]

**6. **J. L. Flores, J. A. Ferrari, G. García Torales, R. Legarda-Saenz, and A. Silva, “Color-fringe pattern profilometry using a generalized phase-shifting algorithm,” Appl. Opt. **54**(30), 8827–8834 (2015). [CrossRef] [PubMed]

**7. **J. H. Bruning, D. R. Herriott, J. E. Gallagher, D. P. Rosenfeld, A. D. White, and D. J. Brangaccio, “Digital wavefront measuring interferometer for testing optical surfaces and lenses,” Appl. Opt. **13**(11), 2693–2703 (1974). [CrossRef] [PubMed]

**8. **M. Takeda, H. Ina, and S. Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. **72**(1), 156–160 (1982). [CrossRef]

**9. **T. Katsuta, F. Ando, M. Kida, and M. Kawasaki, “Optical system for tricolor separation,” U.S. Patent: 3 602 637, issued date August 31, 1971.

**10. **M. Servin, M. Cywiak, D. Malacara-Hernandez, J. C. Estrada, and J. A. Quiroga, “Spatial carrier interferometry from M temporal phase shifted interferograms: Squeezing Interferometry,” Opt. Express **16**(13), 9276–9283 (2008). [CrossRef] [PubMed]

**11. **B. Li, L. Chen, W. Tuya, S. Ma, and R. Zhu, “Carrier squeezing interferometry: suppressing phase errors from the inaccurate phase shift,” Opt. Lett. **36**(6), 996–998 (2011). [CrossRef] [PubMed]

**12. **M. Servin, J. A. Quiroga, and J. M. Padilla, *Fringe Pattern Analysis for Optical Metrology: Theory, Algorithms, and Applications* (Wiley-VCH, Weinheim, 2014).

**13. **J. A. Quiroga and M. Servin, “Isotropic n-dimensional fringe pattern normalization,” Opt. Commun. **224**(4-6), 221–227 (2003). [CrossRef]

**14. **H. Guo, H. He, and M. Chen, “Gamma correction for digital fringe projection profilometry,” Appl. Opt. **43**(14), 2906–2914 (2004). [CrossRef] [PubMed]

**15. **S. Lei and S. Zhang, “Flexible 3-D shape measurement using projector defocusing,” Opt. Lett. **34**(20), 3080–3082 (2009). [CrossRef] [PubMed]

**16. **D. C. Ghiglia and M. D. Pritt, *Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software* (Wiley, 1998).