High-accuracy 3D shape measurement of translucent objects by fringe projection profilometry

Yang Xu; Huijie Zhao; Hongzhi Jiang; Xudong Li

doi:10.1364/OE.27.018421

1. Introduction

Optical 3D shape measurement is widely applied in several areas [1]. Fringe projection profilometry (FPP) [2,3], which is one of the most popular 3D shape measurement methods, can acquire full-field, high-resolution, and high-precision 3D point cloud. However, objects composed of translucent materials, such as marbles, synthetic resins, and biological tissues, are difficult to be accurately measured by FPP because of the subsurface scattering effect. The incident light penetrates the surface, scatters, and then exits at different locations around the incident point for translucent materials. Subsurface scattering degrades the modulation of the fringe pattern, which produces incomplete measurement results and large random errors. In addition, subsurface scattering introduces phase offset, which induces geometric errors.

In recent years, optical 3D shape measurement methods for translucent objects are widely investigated. Polarizers can be used to separate surface reflection from subsurface scattering to eliminate geometric errors [4]. High-frequency illumination can be utilized to separate direct and indirect illumination because low-frequency indirect illumination, such as subsurface scattering, becomes nearly constant under high-frequency illumination [5]. Low-frequency phase-shifting fringe patterns can be modulated by high-frequency patterns to separate surface reflection from subsurface scattering [6]. Phase-shifting fringe patterns can be limited in a narrow high-frequency band to reduce geometric errors [7].

However, high-frequency illumination reduces the contrast of the projected patterns, which leads to higher phase errors for FPP and binarization errors for binary pattern projection method. Therefore, conventional Gray codes can be substituted with minimum stripe-width codes to achieve robust binarization [8], but large width stripe increases geometric errors. Sparse dashed lines can measure translucent objects with high accuracy [9]. The frequency of dashed lines is sufficiently high in the vertical direction to reduce geometric errors. In the horizontal direction, a large stride is used to maintain the image contrast.

Geometric errors can be compensated for by measuring an etalon with the same material as the measured object from different directions [10]. A statistical error model can be fitted by measuring an object with the same material to the measured object before and after coating to compensate for the geometric errors [11]. Another error compensation method requires the bidirectional surface scattering reflectance distribution function of the object and can only handle short-range scattering effects [12]. Epipolar imaging can significantly reduce the impact of subsurface scattering [13–15], but geometric errors cannot be fully eliminated because indirect illumination transports in the epipolar plane remains. The influence of light wavelength on measuring translucent objects is investigated [16,17], and the results indicate that short-wave light can reduce geometric errors.

In this research, a high-accuracy 3D shape measurement method for translucent objects based on phase-shifting FPP is presented. The proposed method copes with measurement errors from different sources. The relationship between fringe period and random errors is established to determine the fringe periods. Random errors induced by modulation degradation are suppressed by temporal noise reduction. The robustness of multi-frequency heterodyne phase unwrapping is improved by increasing the interval of fringe periods along with temporal noise reduction. Geometric errors introduced by the phase offset are compensated for by projecting fringe patterns with multiple fringe periods to fit an empirical error model that describes the relationship between fringe period and depth offset. Experimental results demonstrate that the proposed method can acquire complete 3D shape measurement results of translucent objects and significantly reduce the overall error.

The rest of this paper is organized as follows. Principles of the proposed method, including the relationship between fringe period and phase error, random error suppression, phase unwrapping improvement, and geometric error compensation, are explained in Section 2. Experimental and simulation results are exhibited and discussed in Section 3, and conclusions are presented in Section 4.

2. Principles

2.1 Challenges for measuring translucent objects by FPP

In this research, stereo vision based phase-shifting FPP is utilized [18,19]. The phase-shifting sinusoidal fringe patterns are projected by a projector, and two cameras capture the deformed fringe patterns on the object simultaneously. Phase matching between two cameras is performed to obtain the 3D point cloud [20].

The stereo vison based FPP is resistant to several error sources, such as the nonlinearity of the projector and diffuse interreflections, because the error sources exhibit in the same manner in the two cameras. As exhibited in Fig. 1, the exit point of the light is the same as the incident point when opaque objects are illuminated by the projector. However, when translucent objects are illuminated, the incident light penetrates the surface, scatters in the participating media, and then exits at different locations around the incident point. This phenomenon is called subsurface scattering, which increases the difficulty of accurately measuring translucent objects by FPP. In this research, four-step phase-shifting FPP is utilized to recover the phase. The intensities of sinusoidal fringe patterns I_i (i = 0, 1, 2, 3) for each point are described as

I_{i} = A + B \cos (ϕ + \frac{π}{2} i),

where A is the average intensity and B is the modulation of sinusoidal fringe pattern, which can be obtained by

B = \frac{1}{2} \sqrt{{(I_{3} - I_{1})}^{2} + {(I_{0} - I_{2})}^{2}},

and ϕ is the phase of sinusoidal fringe pattern that can be retrieved by

Fig. 1 Schematics of 3D shape measurement of opaque and translucent objects by FPP. (a) 3D shape measurement of opaque objects by FPP; (b) 3D shape measurement of translucent objects by FPP.

Download Full Size | PDF

ϕ = arc \tan \frac{I_{3} - I_{1}}{I_{0} - I_{2}} .

The intensity noise of the image captured by the camera is assumed to be additive and zero-mean Gaussian distributed with a standard deviation of σ_n. The standard deviation of the phase error σ_ϕ can be defined as [3]

σ_{ϕ} = \frac{σ_{n}}{\sqrt{2} B} .

There are three major challenges to be faced when phase-shifting FPP is utilized to measure translucent objects. An opaque object (ABS resin) and a translucent object (polyamide) are measured for comparison to illustrate the challenges.

(1) Random errors: Phase errors caused by intensity noise lead to random errors in the reconstructed 3D point cloud. The random errors for translucent materials are significantly larger than those for opaque materials because the fringe modulation B is degraded by the subsurface scattering effects as exhibited in Fig. 2(a).
(2) Phase unwrapping failures: Phase unwrapping is likely to fail because the multi-frequency heterodyne phase unwrapping method [21,22], which is commonly utilized in FPP, is sensitive to intensity noise. Phase unwrapping failure causes incomplete 3D shape measurement results for translucent materials.
(3) Geometric errors: Geometric errors are introduced by the phase offset caused by the subsurface scattering effect. Such errors can be regarded as depth offsets that make the 3D point far from the actual position. As shown in Fig. 2(b), the measured depth remains constant across various fringe periods for opaque materials but changes for translucent materials, which indicates that the measured depth for translucent materials has geometric errors. The depth offsets may not affect shape measurement if the depth offsets are constant across the measured surface. However, projection and observation directions are different across the measured surface, and the transparency of the surface may be spatially varying. Therefore, the spatially-varying depth offset changes the shape of the measured object. More details about geometric errors induced by subsurface scattering in stereo vision based FPP can be found in [10,23,24].

Fig. 2 Comparison of fringe intensity profile and measured depth between opaque and translucent materials. The data are from the measurement results of an opaque object (ABS resin) and a translucent object (polyamide). (a) Fringe intensity profile. The modulation for translucent materials is significantly degraded; (b) Measured depth across various fringe periods. The measured depth is constant for opaque materials but variational for translucent materials (the depth is averaged in a 10 × 10 window to suppress random errors).

Download Full Size | PDF

In previous research, high-frequency fringe patterns can be used to reduce geometric errors [5–7], but the degradation of fringe modulation increases random errors. The reduction of random or geometric errors is a pair of contradictions. In addition, the fringe frequency cannot be infinite; hence, geometric errors cannot be completely eliminated. In this research, fringe patterns with relatively low frequencies are combined with a novel geometric error compensation method to reduce random and geometric errors simultaneously.

2.2 Relationship between fringe period and phase error

The relationship between fringe period λ and phase error is investigated to determine the fringe periods. The wrapped phase ϕ, which is computed by Eq. (3), should be unwrapped to remove phase ambiguity. The range of the unwrapped phase $Φ$ is [0, 2πF], where F is defined as the total number of fringe periods in the projected pattern as follows

F = W_{p} f = \frac{W_{p}}{λ},

where W_p is the horizontal resolution of the projector and f is the spatial frequency of the fringe pattern. If the unwrapped phase is scaled to the same range [0, 2π], the standard deviation of the normalized unwrapped phase error

σ_{Φ}

can be expressed as [3]

σ_{Φ} = \frac{σ_{n}}{\sqrt{2} B F} .

The modulation B is not constant and is affected by multiple factors, such as the resolution of the optical system, defocusing, and the subsurface scattering effect of translucent materials. Thus, the modulation B can be regarded as a function b(λ) with a variable fringe period λ. Then, Eq. (6) can be rewritten as

σ_{Φ} (λ) = \frac{σ_{n} λ}{\sqrt{2} b (λ) W_{p}} .

For opaque materials, the modulation B changes slightly with the growth of the fringe period. Thus, random errors can be reduced by decreasing the fringe period. However, decreasing the fringe period also degrades the modulation for translucent materials. Hence, confirming whether random errors are reduced is difficult.

To acquire the modulation function b(λ), the line spread function (LSF) ℒ [25], which can be assumed to be symmetric, can be measured and convolved with the original sinusoidal fringe patterns as follows [26]

g (x) = f (x) \otimes h (x) = [A + B \cos (ω_{0} x)] \otimes L (x),

where g(x) is the convolved fringe pattern, f(x) is the original sinusoidal fringe pattern, h(x) is the convolution kernel, ℒ(x) is the LSF,

\otimes

is the convolution operator, and ω₀ = 2π / λ is the angular frequency of the fringe pattern. According to the convolution theorem, g(x) can be also obtained by

g (x) = F^{- 1} [F (ω) H (ω)],

where ℱ denotes the Fourier transform and ℱ⁻¹ is the inverse Fourier transform, F(ω) and H(ω) are the Fourier spectra of the fringe pattern and the LSF, respectively, and ω is the angular frequency. In the frequency domain, the Fourier spectrum of the fringe pattern F(ω) can be expressed as

F (ω) = 2 π A δ (ω) + π B [δ (ω - ω_{0}) + δ (ω + ω_{0})] .

Therefore, the Fourier spectrum of the convolved fringe pattern G(ω) can be obtained by

\begin{matrix} G (ω) = F (ω) H (ω) \\ = 2 π A δ (ω) H (0) + π B [δ (ω - ω_{0}) + δ (ω + ω_{0})] H (ω_{0}) . \end{matrix}

Consequently, the convolved fringe pattern g(x) is given by

g (x) = F^{- 1} [G (ω)] = H (0) A + H (ω_{0}) B \cos (ω_{0} x) .

Hence, the modulation function b(λ) can be expressed as

b (λ) = H (ω_{0}) B .

However, the analytical expression of H(ω) is difficult to acquire because the LSF is measured from the discrete image data and the LSF of translucent materials cannot be well modelled by common degradation models, such as the Gaussian kernel. Alternatively, the discrete expression of H(ω) can be obtained by

H (k) = DFT[L (n)],

where DFT denotes the discrete Fourier transform, ℒ(n) (n = 0, 1, 2,…, N − 1) is the measured discrete LSF where N is the number of sample points, and H(k) (k = 0, 1, 2,…, N − 1) is the discrete expression of H(ω). Thus, H(ω₀) can be approximated by

H (ω_{0}) \approx | H (\frac{N - 1}{λ_{c}}) |,

where | | denotes the amplitude of the complex spectrum, and λ_c is the fringe period in the camera image, which can be obtained by

λ_{c} = \frac{f_{c}}{f_{p}} λ,

where f_c and f_p are the focal lengths expressed in pixel units (the product of the focal length in world units and the number of pixels per world unit) from the intrinsic parameters of the camera and the projector, respectively. Therefore, Eq. (7) can be rewritten according to Eqs. (13) and (15) as follows

σ_{Φ} (λ) \approx \frac{σ_{n} λ}{\sqrt{2} | H (\frac{N - 1}{λ_{c}}) | B W_{p}},

and the standard deviation of the normalized unwrapped phase error

σ_{Φ} (λ)

across various fringe periods can be simulated by Eq. (17).

2.3 Random error suppression by temporal noise reduction

For translucent materials, one of the most effective approaches to suppress random errors is to reduce intensity noise according to Eq. (6). The intensity noise, which is additive and zero-mean, can be reduced by temporal noise reduction. Each fringe pattern is repetitively captured for K times. Both the measured object and the measurement system should remain stationary. For each point, the intensities of all captured images I_1,2,...,_i are averaged to obtain the mean intensity $\bar{I}$ as follows

\bar{I} = \frac{1}{K} \sum_{i = 1}^{K} I_{i} .

The standard deviation of intensity noise can be reduced as follows

{σ^{'}}_{n} = \frac{σ_{n}}{\sqrt{K}},

where

{σ^{'}}_{n}

denotes the standard deviation of the intensity noise after temporal noise reduction and σ_n denotes the standard deviation of the single image intensity noise. According to Eq. (4), the standard deviation of the phase error σ_ϕ is divided by

\sqrt{K}

as well after temporal noise reduction.

2.4 Robustness improvement for multi-frequency heterodyne phase unwrapping

In this research, multi-frequency heterodyne phase unwrapping [21,22] is utilized for FPP because the measurement results of different fringe periods can be averaged to further reduce random errors. However, multi-frequency heterodyne phase unwrapping is sensitive to intensity noise because the modulation of the fringe pattern is low for translucent materials, which increases the phase error. Suppose that two-frequency fringe patterns with two fringe periods λ₁ and λ₂ (λ₁ < λ₂) are utilized, and ∆λ = λ₂ − λ₁ is the interval of fringe periods. Hence, the equivalent fringe period λ_eq is given by

λ_{eq} = \frac{λ_{1} λ_{2}}{λ_{2} - λ_{1}},

and the equivalent phase ϕ_eq corresponding to λ_eq can be obtained by

ϕ_{eq} = {\begin{matrix} ϕ_{1} - ϕ_{2}, & ϕ_{1} - ϕ_{2} > 0 \\ ϕ_{1} - ϕ_{2} + 2 π, & ϕ_{1} - ϕ_{2} < 0 \end{matrix},

where ϕ₁ is the phase corresponding to the fringe period λ₁, and ϕ₂ is the phase corresponding to λ₂. The equivalent phase ϕ_eq is unambiguous in the entire range of the projector, but it cannot be directly regarded as the unwrapped phase because it is noisy. ϕ_eq is only used to assist phase unwrapping. Hence, the unwrapped phase of λ₁ can be obtained by

Φ_{1} = ϕ_{1} + 2 π round [\frac{1}{2 π} (\frac{λ_{eq}}{λ_{1}} ϕ_{eq} - ϕ_{1})] .

In Eq. (22), the round operation can eliminate errors less than 0.5. To correctly unwrap the phase, the deviation of the equivalent phase $Δ ϕ_{eq} = {\hat{ϕ}}_{eq} - ϕ_{eq}$ , where ${\hat{ϕ}}_{eq}$ is the measured equivalent phase and $ϕ_{eq}$ is the actual equivalent phase, should satisfy

\frac{λ_{eq}}{λ_{1}} Δ ϕ_{eq} = \frac{λ_{2}}{λ_{2} - λ_{1}} Δ ϕ_{eq} < π .

Fig. 3 Part of the unwrapped phase maps of different intervals of fringe periods ∆λ. The phase maps are from the measurement results of a jade statue. (a) ∆λ = 1 pixel (fringe periods are 13, 14, and 15 pixels), K = 12; (b) ∆λ = 4 pixels (fringe periods are 16, 20, 24, 28, 32, and 36 pixels), K = 12.

Download Full Size | PDF

Multi-frequency heterodyne phase unwrapping is sensitive to intensity noise because the deviation of the equivalent phase ∆ϕ_eq is multiplied by λ₂ / (λ₂ − λ₁) in Eq. (23), which makes the errors difficult to be eliminated by the round operation in Eq. (22). The conclusion is similar for three or more frequencies [27]. In general, adjacent natural numbers are selected to be the fringe periods because fewer fringe periods are required to cover the entire range of the projector. For instance, 13, 14, and 15 are chosen to be the fringe periods when the horizontal resolution of the projector W_p = 1280 (the equivalent fringe period λ_eq = 1365). In this case, ∆ϕ_eq is multiplied by 14 and 15 during phase unwrapping. For opaque materials, it may not be a problem because the phase error is low. However, the phase error is large when translucent objects are measured; thus, the interval between λ₁ and λ₂ should be large enough to correctly unwrap the phase. In summary, increasing the interval of fringe periods ∆λ can improve the robustness of multi-frequency heterodyne phase unwrapping for translucent materials as demonstrated in Fig. 3. In addition, the temporal noise reduction described in Section 2.3 can improve the robustness of phase unwrapping because intensity noise is reduced.

2.5 Geometric error compensation

In this research, a novel geometric error compensation method that does not require any etalons or known material parameters is proposed. Fringe patterns with multiple fringe periods are projected to establish the relationship between fringe period and depth offset. Meanwhile, the measured object and the measurement system remain stationary during the projection. For each point of the object, the depth offset $ε (λ)$ corresponding to the fringe period λ is defined as

ε (λ) = \hat{z} (λ) - z,

where

\hat{z} (λ)

is the measured depth that contains depth offset, and z is the ground truth depth. On the basis of the measured depth as displayed in Fig. 2(b), the depth offset

ε (λ)

can be approximated by the following empirical expression

ε (λ) = p_{1} λ^{2} + p_{2} λ,

where p₁ and p₂ are the coefficients of the second-order polynomial. The depth offset

ε (λ)

cannot be directly obtained by Eq. (24) because the ground truth depth is unknown. Nevertheless, the ground truth depth can be eliminated by subtracting the depth offset of a fringe period from that of another fringe period to acquire the relative depth offset, which can also be expressed as the depth difference between two fringe periods as follows

ε (λ) - ε (λ_{1}) = [\hat{z} (λ) - z] - [\hat{z} (λ_{1}) - z] = \hat{z} (λ) - \hat{z} (λ_{1}),

where λ₁ is the fringe period of the highest frequency fringe pattern in the multi-frequency fringe patterns. The relative depth offsets can be obtained from the measurement results of various fringe periods. According to Eq. (25), the relative depth offset can be also expressed as

ε (λ) - ε (λ_{1}) = p_{1} (λ^{2} - λ_{1}^{2}) + p_{2} (λ - λ_{1}) .

Combining Eq. (26) with Eq. (27), p₁ and p₂ can be estimated by projecting multi-frequency fringe patterns and solving the following linear least-squares problem

[\begin{matrix} λ_{2}^{2} - λ_{1}^{2} & λ_{2} - λ_{1} \\ λ_{3}^{2} - λ_{1}^{2} & λ_{3} - λ_{1} \\ ⋮ & ⋮ \\ λ_{N}^{2} - λ_{1}^{2} & λ_{N} - λ_{1} \end{matrix}] [\begin{matrix} p_{1} \\ p_{2} \end{matrix}] = [\begin{matrix} \hat{z} (λ_{2}) - \hat{z} (λ_{1}) \\ \hat{z} (λ_{3}) - \hat{z} (λ_{1}) \\ ⋮ \\ \hat{z} (λ_{N}) - \hat{z} (λ_{1}) \end{matrix}],

where λ_i represents the ith fringe period (i = 2, 3,…, N, N > 2). The fringe periods utilized in multi-frequency heterodyne phase unwrapping can be directly applied to solve Eq. (28). Hence, no additional fringe patterns are required to be projected. Combining with Eq. (24), the depth offsets of all fringe periods are compensated for and averaged to obtain the compensated depth zʹ as follows

z^{'} = \frac{1}{N} \sum_{i = 1}^{N} [\hat{z} (λ_{i}) - ε (λ_{i})] = \frac{1}{N} \sum_{i = 1}^{N} [\hat{z} (λ_{i}) - {\hat{p}}_{1} λ_{i}^{2} - {\hat{p}}_{2} λ_{i}],

where

{\hat{p}}_{1}

and

{\hat{p}}_{2}

are the estimated values of p₁ and p₂ by solving the linear least-squares problem. The compensated depth of each fringe period is computed separately, and then all of them are averaged to further reduce random errors. In practice, the relative depth offsets (depth differences) are smoothed to mitigate the impact of random errors on geometric error compensation.

3. Experiments

The experimental system consists of a digital projector with a resolution of 1280 × 800 and two monochrome CCD cameras (Basler acA1600-20gm) with a resolution of 1626 × 1236. The projector is equipped with a blue LED light source at 465 nm. A sphere made by polyamide (nylon), which is a type of synthetic resin, and a jade statue are investigated in the experiment as exhibited in Fig. 4. The nominal diameter of the sphere is 25.4 mm, and the material of the region in the red circle is near opaque as shown in Fig. 4(b).

Fig. 4 Measured objects. (a) Polyamide sphere; (b) Jade statue (the material of the region in the red circle is near opaque).

Download Full Size | PDF

The LSFs of the measured translucent materials are first acquired to establish the relationship between fringe period and phase error for translucent materials. To measure the LSF, a light stripe with a width of 1 pixel is projected onto the object, and an image is captured by the camera. To eliminate the impact of ambient illumination, an image without the projected light stripe is captured and subtracted from the image with the projected light stripe to acquire the difference image. The intensity profile of a scanline in the difference image is regarded as the measured discrete LSF ℒ(n), and each pixel of the scanline is a sample point of LSF. The sample points far away from the peak are discarded. The LSF of opaque material is also measured for comparison. The measured LSFs and their DFT spectra are shown in Fig. 5. The number of sample points N is 21 for opaque material, 141 for polyamide, and 201 for jade.

Fig. 5 Measured LSFs and their DFT spectra. (a) LSF of opaque material; (b) LSF of polyamide; (c) LSF of jade; (d) DFT spectrum of (a); (e) DFT spectrum of (b); (f) DFT spectrum of (c).

Download Full Size | PDF

Then, the modulation function b(λ) and the unwrapped phase error can be simulated. The modulation function can be simulated by Eqs. (13) and (15) (λ = 10,11,12,...,70), and the undegraded modulation B is set at 1 because the value of B is unknown. f_c is 1898.61 and f_p is 1753.89 in Eq. (16). To validate the simulation of modulation function, the modulation function is directly measured by projecting four-step phase-shifting fringe patterns with different fringe periods onto the translucent objects and then compute the modulation corresponding to a specific fringe period by Eq. (2). To compare the simulated modulation with the measured modulation, the simulated and measured modulations are divided by the respective maximum modulations of various fringe periods to acquire the relative modulations as depicted in Fig. 6(a).

Fig. 6 Comparison of the modulation and normalized unwrapped phase error between measurement and simulation. The modulations are divided by the maximum, and the phase errors are divided by the minimum to acquire the relative modulations and phase errors for comparison between measurement and simulation. (a) Measured and simulated relative modulations across various fringe periods; (b) Measured and simulated relative normalized unwrapped phase errors across various fringe periods.

Download Full Size | PDF

Once the simulated modulation is acquired, the standard deviation of the simulated normalized unwrapped phase error can be computed by Eq. (7) and the standard deviation of the intensity noise σ_n is set at 1 because σ_n is not measured in the experiment. The simulated phase error is compared with the measured phase error, which can be represented by the repeatability of normalized unwrapped phase. To compare the simulated phase error with the measured phase error, the simulated and measured phase errors are divided by the respective minimum phase errors of various fringe periods to acquire the relative phase errors as displayed in Fig. 6(b).

According to the measurement and simulation results displayed in Fig. 6, the normalized unwrapped phase error always increases for opaque materials. For translucent materials, the modulation is approximately proportional to the fringe period. However, the phase error shows no obvious regularity, and the ratio of the maximum phase error to the minimum phase error is quite small. Considering that the objects with both translucent and opaque materials may be measured, the fringe period should not be very large. Therefore, the fringe periods are chosen to be 16, 20, 24, 28, 32, and 36 pixels in the following experiment. Hence, ∆λ = 4 pixels and the equivalent fringe period λ_eq = 2016 pixels.

To prove that temporal noise reduction can suppress random errors, the phase error corresponding to various numbers of averaged frames K (K = 1, 2, 3,…, 12) is acquired by measuring the repeatability of phase between two independent measurements. The fringe period λ = 16 pixels. The standard deviation of the simulated phase error is computed by Eq. (19). Given that the standard deviation of the intensity noise σ_n is unknown, the measured and simulated phase errors are divided by the respective maximum phase errors of various K to acquire the relative phase errors as displayed in Fig. 7. The measurement and simulation results in Fig. 7 demonstrate that temporal noise reduction can significantly reduce random errors.

Fig. 7 Measured and simulated relative phase errors across various K.

Download Full Size | PDF

To verify that increasing the interval of fringe periods and temporal noise reduction can improve the robustness of multi-frequency heterodyne phase unwrapping, the jade statue is measured by using different intervals of fringe periods ∆λ and number of averaged frames K as exhibited in Fig. 8. The intervals of fringe periods ∆λ are 1 pixel (fringes periods are 13, 14, and 15 pixels) and 4 pixels (fringes periods are 16, 20, 24, 28, 32, and 36 pixels), and the numbers of averaged frames K are 1 and 12. The measurement results demonstrate that increasing either the interval of fringe periods or the number of averaged frames can improve the robustness of phase unwrapping. Moreover, several regions in Figs. 8(a)–8(c) can be measured because the materials are nearly opaque as shown in Fig. 4(b).

Fig. 8 3D shape measurement results of the jade statue by different ∆λ and K. (a) ∆λ = 1 pixel, K = 1; (b) ∆λ = 4 pixels, K = 1; (c) ∆λ = 1 pixel, K = 12; (d) ∆λ = 4 pixels, K = 12.

Download Full Size | PDF

To evaluate the effectiveness of geometric error compensation, the polyamide sphere and the jade statue are measured by the proposed method. Multi-frequency four-step phase-shifting fringe patterns are projected to acquire 3D point clouds separately. The fringe periods are the same as those used in multi-frequency heterodyne phase unwrapping. Hence, N = 6 in Eqs. (28) and (29). The number of averaged frames is 4 for the polyamide sphere and 12 for the jade statue. For each point, p₁ and p₂ are estimated using Eq. (28), and the depth offsets are compensated by Eq. (29) to acquire the compensated point cloud.

Fig. 9 Deviations of sphere fitting. The data in the central region is missing due to the highlight. (a) Deviations of the uncompensated point cloud (λ = 16 pixels); (b) Deviations of the uncompensated point cloud (λ = 36 pixels); (c) Deviations of the uncompensated point cloud (average of all fringe periods); (d) Deviations of the compensated point cloud.

Download Full Size | PDF

To evaluate the accuracy of measuring the polyamide sphere, the uncompensated and compensated point clouds are fitted by a sphere. The deviations of sphere fitting are exhibited in Fig. 9, and the corresponding diameters of the fitted sphere, mean absolute errors (MAEs), and root mean square errors (RMSEs) of sphere fitting are displayed in Table 1. The data in the central region is missing due to the highlight. The deviations shown in Fig. 9 and the diameters of the fitted spheres displayed in Table 1 demonstrate that the proposed geometric compensation method can significantly reduce the geometric errors and larger fringe period induces larger geometric errors. In addition, random errors are suppressed by averaging the measurement results of multi-frequency fringe patterns by comparing Figs. 9(c) and 9(d) with Figs. 9(a) and 9(b).

Table 1. Fitting sphere diameter and errors (mm)

View Table | View all tables in this article

Fig. 10 Deviations between the measured point clouds and the reference. (a) Deviations between the uncompensated point cloud (λ = 16 pixels) and the reference; (b) Deviations between the uncompensated point cloud (λ = 36 pixels) and the reference; (c) Deviations between the compensated point cloud and the reference. The errors in the opaque regions are larger than those in the translucent regions because the translucent regions occupy most part of the jade statue, and ICP registration is performed between the measurement point cloud and the reference point cloud for comparison.

Download Full Size | PDF

To evaluate the measurement accuracy of the jade statue, the jade statue is coated with powders to acquire the reference point cloud. Then, uncompensated and compensated point clouds are compared with the reference point cloud to evaluate the accuracy as exhibited in Fig. 10, and the corresponding MAEs and RMSEs are displayed in Table 2. To compare with the reference point cloud, the uncompensated and compensated point clouds are aligned to the reference point cloud by iterative closest point (ICP) algorithm [28,29] because the measured object is removed from the measured position before coating. The errors in the opaque region [red circle in Fig. 4(b)] should be small. However, Figs. 10(a) and 10(b) show that the errors in the opaque regions are larger than those in the translucent regions because the translucent regions occupy most part of the jade statue, and ICP registration between the measurement point cloud and the reference point cloud reduces the deviations in the translucent regions. The deviations shown in Fig. 10 do not express the depth offset but indicate the changes of the shape. The deviations and measurement errors displayed in Fig. 10 and Table 2 demonstrate that the geometric errors are significantly reduced in the compensated point cloud and large fringe period increases the geometric errors.

Table 2. Measurement errors of the jade statue (mm)

View Table | View all tables in this article

4. Conclusion

This research presents a high-accuracy 3D shape measurement method for translucent objects based on FPP, which copes with measurement errors from different sources. The relationship between fringe period and phase error is investigated to determine the fringe periods. Random errors are suppressed by temporal noise reduction. The robustness of multi-frequency heterodyne phase unwrapping is improved by increasing the interval of fringe periods along with temporal noise reduction. Geometric errors are compensated for by establishing the relationship between fringe period and depth offset. Experimental results demonstrate that the proposed method can obtain complete measurement results and significantly reduce the overall error for measuring translucent objects. Compared with high-frequency pattern projection methods, the proposed method uses relatively lower frequency fringe patterns to maintain fringe modulation. Compared with other geometric error compensation methods, the proposed method does not require etalons with the same material as the measured object or material parameters. In addition, the hardware of the proposed method is the same as that of the conventional stereo vision based FPP. Therefore, the conventional stereo vision based FPP system can be adapted to acquire complete and accurate 3D shape measurement results of translucent objects without modification.

Funding

National Natural Science Foundation of China (NSFC) (61735003, 61875007); Program for Changjiang Scholars and Innovative Research Team in University (IRT_16R02); Leading Talents Program for Enterpriser and Innovator of Qingdao (18-1-2-22-zhc).

References

1. F. Chen, G. M. Brown, and M. Song, “Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. 39(1), 10–22 (2000). [CrossRef]

2. S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng. 48(2), 149–158 (2010). [CrossRef]

3. C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: A review,” Opt. Lasers Eng. 109, 23–59 (2018). [CrossRef]

4. T. Chen, H. P. A. Lensch, C. Fuchs, and H. P. Seidel, “Polarization and phase-shifting for 3D scanning of translucent objects,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1829–1836. [CrossRef]

5. S. K. Nayar, G. Krishnan, M. D. Grossberg, and R. Raskar, “Fast separation of direct and global components of a scene using high frequency illumination,” in Proceedings of ACM SIGGRAPH (ACM, 2006), pp. 935–944. [CrossRef]

6. T. Chen, H. P. Seidel, and H. P. A. Lensch, “Modulated phase-shifting for 3D scanning,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 3839–3846.

7. M. Gupta and S. K. Nayar, “Micro phase shifting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 813–820.

8. M. Gupta, A. Agrawal, A. Veeraraghavan, and S. G. Narasimhan, “A practical approach to 3D scanning in the presence of interreflections, subsurface scattering and defocus,” Int. J. Comput. Vis. 102(1–3), 33–55 (2013). [CrossRef]

9. T. Kobayashi, T. Higo, M. Yamasaki, K. Kobayashi, and A. Katayama, “Accurate and practical 3D measurement for translucent objects by dashed lines and complementary Gray code projection,” in Proceedings of International Conference on 3D Vision (IEEE, 2015), pp. 189–197. [CrossRef]

10. P. Lutzke, P. Kühmstedt, and G. Notni, “Measuring error compensation on three-dimensional scans of translucent objects,” Opt. Eng. 50(6), 063601 (2011). [CrossRef]

11. S. N. Jensen, J. Wilm, and H. Aanæs, “An error analysis of structured light scanning of biological tissue,” in Proceedings of the Scandinavian Conference on Image Analysis (Springer, 2017), pp. 135–145. [CrossRef]

12. L. Rao and F. Da, “Local blur analysis and phase error correction method for fringe projection profilometry systems,” Appl. Opt. 57(15), 4267–4276 (2018). [CrossRef] [PubMed]

13. M. O’Toole, J. Mather, and K. N. Kutulakos, “3D shape and indirect appearance by structured light transport,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 3246–3253. [CrossRef]

14. M. O’Toole, S. Achar, S. G. Narasimhan, and K. N. Kutulakos, “Homogeneous codes for energy-efficient illumination and imaging,” in Proceedings of ACM SIGGRAPH (ACM), 35 (2015). [CrossRef]

15. H. Zhao, Y. Xu, H. Jiang, and X. Li, “3D shape measurement in the presence of strong interreflections by epipolar imaging and regional fringe projection,” Opt. Express 26(6), 7117–7131 (2018). [CrossRef] [PubMed]

16. C. Zhang, M. Rosenberger, A. Breitbarth, and G. Notni, “Wavelength dependency of optical 3D measurements at translucent objects using fringe pattern projection,” Proc. SPIE 10220, 1022007 (2017). [CrossRef]

17. S. Heist, C. Zhang, K. Reichwald, P. Kühmstedt, G. Notni, and A. Tünnermann, “5D hyperspectral imaging: fast and accurate measurement of surface shape and spectral characteristics using structured light,” Opt. Express 26(18), 23366–23379 (2018). [CrossRef] [PubMed]

18. C. Reich, R. Ritter, and J. Thesing, “3-D shape measurement of complex objects by combining photogrammetry and fringe projection,” Opt. Eng. 39(1), 224–231 (2000). [CrossRef]

19. X. Han and P. Huang, “Combined stereovision and phase shifting method: A new approach for 3D shape measurement,” Proc. SPIE 7389, 73893C (2009). [CrossRef]

20. D. Li, H. Zhao, and H. Jiang, “Fast phase-based stereo matching method for 3D shape measurement,” in Proceedings of International Symposium on Optomechatronic Technologies (IEEE, 2011), pp. 1–5.

21. C. Zuo, L. Huang, M. Zhang, Q. Chen, and A. Asundi, “Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review,” Opt. Lasers Eng. 85, 84–103 (2016). [CrossRef]

22. S. Zhang, “Absolute phase retrieval methods for digital fringe projection profilometry: A review,” Opt. Lasers Eng. 107, 28–37 (2018). [CrossRef]

23. P. Lutzke, P. Kühmstedt, and G. Notni, “Fast error simulation of optical 3D measurements at translucent objects,” Proc. SPIE 8493, 84930U (2012). [CrossRef]

24. P. Lutzke, S. Heist, P. Kühmstedt, R. Kowarschik, and G. Notni, “Monte Carlo simulation of three-dimensional measurements of translucent objects,” Opt. Eng. 54(8), 084111 (2015). [CrossRef]

25. K. Happel, E. Dörsam, and P. Urban, “Measuring isotropic subsurface light transport,” Opt. Express 22(8), 9048–9062 (2014). [CrossRef] [PubMed]

26. Y. Wang, H. Zhao, H. Jiang, and X. Li, “Defocusing parameter selection strategies based on PSF measurement for square-binary defocusing fringe projection profilometry,” Opt. Express 26(16), 20351–20367 (2018). [CrossRef] [PubMed]

27. C. Reich, R. Ritter, and J. Thesing, “White light heterodyne principle for 3D-measurement,” Proc. SPIE 3100, 236–244 (1997). [CrossRef]

28. S. Rusinkiewicz and M. Levoy, “Efficient variants of the ICP algorithm,” in Proceedings of International Conference on 3-D Digital Imaging and Modeling (IEEE, 2001), pp. 145–152. [CrossRef]

29. T. Zinßer, J. Schmidt, and H. Niemann, “A refined ICP algorithm for robust 3-D correspondence estimation,” in Proceedings of International Conference on Image Processing (IEEE), 695–698 (2003). [CrossRef]

	Fitted diameter	Mean absolute error	Root mean square error
Uncompensated (λ = 16)	25.565	0.032	0.040
Uncompensated (λ = 36)	25.696	0.039	0.048
Uncompensated (average)	25.648	0.029	0.035
Compensated	25.411	0.020	0.025

	Mean absolute error	Root mean square error
Uncompensated (λ = 16)	0.071	0.095
Uncompensated (λ = 36)	0.102	0.138
Compensated	0.049	0.070

	Fitted diameter	Mean absolute error	Root mean square error
Uncompensated (λ = 16)	25.565	0.032	0.040
Uncompensated (λ = 36)	25.696	0.039	0.048
Uncompensated (average)	25.648	0.029	0.035
Compensated	25.411	0.020	0.025

	Mean absolute error	Root mean square error
Uncompensated (λ = 16)	0.071	0.095
Uncompensated (λ = 36)	0.102	0.138
Compensated	0.049	0.070

High-accuracy 3D shape measurement of translucent objects by fringe projection profilometry

Abstract

1. Introduction

2. Principles

2.1 Challenges for measuring translucent objects by FPP

2.2 Relationship between fringe period and phase error

2.3 Random error suppression by temporal noise reduction

2.4 Robustness improvement for multi-frequency heterodyne phase unwrapping

2.5 Geometric error compensation

3. Experiments

4. Conclusion

Funding

References

Cited By

Figures (10)

Tables (2)

Equations (29)

Optics Express