Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group
Journal Home About Issues in Progress Current Issue All Issues Feature Issues

High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system

Wei Yin, Shijie Feng, Tianyang Tao, Lei Huang, Maciej Trusiak, Qian Chen, and Chao Zuo

Open Access Open Access

Abstract

In this paper, we propose a high-speed 3D shape measurement technique based on the optimized composite fringe patterns and stereo-assisted structured light system. Stereo phase unwrapping, as a new-fashioned method for absolute phase retrieval based on the multi-view geometric constraints, can eliminate the phase ambiguities and obtain a continuous phase map without projecting any additional patterns. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which limits the accuracy of 3D measurement. To solve this problem, we develop an optimized method for designing the composite pattern, in which the speckle pattern is embedded into the conventional 4-step phase-shifting fringe patterns without compromising the fringe modulation, and thus the phase measurement accuracy. We also present a simple and effective evaluation criterion for the correlation quality of the designed speckle pattern in order to improve the matching accuracy significantly. When the embedded speckle pattern is demodulated, the periodic ambiguities in the wrapped phase can be eliminated by combining the adaptive window image correlation with geometry constraint. Finally, some mismatched regions are further corrected based on the proposed regional diffusion compensation technique (RDC). These proposed techniques constitute a complete computational framework that allows to effectively recover an accurate, unambiguous, and distortion-free 3D point cloud with only 4 projected patterns. Experimental results verify that our method can achieve high-speed, high-accuracy, robust 3D shape measurement with dense (64-period) fringe patterns at 5000 frames per second.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article

Corrections

12 March 2019: Corrections were made to the author affiliations and funding section.

More Like This
High-speed real-time 3D shape measurement based on adaptive depth constraint

Tianyang Tao, Qian Chen, Shijie Feng, Jiaming Qian, Yan Hu, Lei Huang, and Chao Zuo
Opt. Express 26(17) 22440-22456 (2018)

Single-shot 3D shape measurement using an end-to-end stereo matching network for speckle projection profilometry

Wei Yin, Yan Hu, Shijie Feng, Lei Huang, Qian Kemao, Qian Chen, and Chao Zuo
Opt. Express 29(9) 13388-13407 (2021)

High-speed three-dimensional shape measurement based on shifting Gray-code light

Zhoujie Wu, Wenbo Guo, and Qican Zhang
Opt. Express 27(16) 22631-22644 (2019)

View More...

Supplementary Material (2)

NameDescription
Visualization 1       The 3D reconstruction results for a one-time transient event: the statue of Voltaire and a free-falling balloon filled with water
Visualization 2       The high-speed 3D reconstruction results for the rip of paper

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)
Fig. 1
Fig. 1 The principle of Zhang’s method [36]. (a) The three-step phase-shifting patterns. (b) The speckle pattern. (c) The intensity of the speckle signal in conventional three-step phase-shifting patterns. (d) The three-step speckle-embedded phase-shifting patterns. (e) The embedded speckle pattern.

Download Full Size | PDF

Fig. 2
Fig. 2 (a) The slope signal D ( x , y ) is designed within the range of [ − 0.5 , 0.5 ] . (b) The designed coding signals C 0 ( x , y ) and C 1 ( x , y ) , where C 0 ( x , y ) = C 2 ( x , y ) , and C 1 ( x , y ) = C 3 ( x , y ) .

Download Full Size | PDF

Fig. 3
Fig. 3 The proposed speckle-embedded phase-shifting algorithm. (a) The four-step speckle-embedded phase-shifting patterns. (b) The embedded speckle pattern.

Download Full Size | PDF

Fig. 4
Fig. 4 The diagram of the optimized design method of the speckle pattern.

Download Full Size | PDF

Fig. 5
Fig. 5 (a) A speckle pattern is designed using the proposed method. (b) The period order map based on (a) is obtained using the proposed test method. (c) The error rate for phase unwrapping on the designed speckle patterns with different speckle size.

Download Full Size | PDF

Fig. 6
Fig. 6 Illustration of an arbitrary point p in the camera and its corresponding points in 3D space and the projector using depth constraint.

Download Full Size | PDF

Fig. 7
Fig. 7 The operation process of confirming period order based on geometry constraint and the adaptive window image correlation. (a) The wrapped phase obtained from main camera. (b) The speckle pattern obtained from main camera. (c) The wrapped phase obtained from auxiliary camera. (d) The speckle pattern obtained from auxiliary camera. (e) The correlation result obtained using the smaller L (5 pixels). (f) The period order map obtained using the smaller L (5 pixels). (g) The correlation result obtained using the larger L (10 pixels). (h) The period order map obtained using the larger L (10 pixels). (i) The enlarged detail of the region in (f). (j) The enlarged detail of the region in (h).

Download Full Size | PDF

Fig. 8
Fig. 8 The flowchart of the compensation procedure of RDC.

Download Full Size | PDF

Fig. 9
Fig. 9 (a) The elementary absolute phase map obtained from auxiliary camera. (b) The final absolute phase map obtained using the left-right consistency check. (c) The 3D reconstruction result obtained using (b). (d) The enlarged detail of the region in (c). (e) The elementary absolute phase map obtained from main camera. (f) The final absolute phase map obtained using RDC. (g) The 3D reconstruction result obtained using (f). (h) The enlarged detail of the region in (g).

Download Full Size | PDF

Fig. 10
Fig. 10 (a) The diagram of the traditional multi-view system. (b) The diagram of our multi-view system.

Download Full Size | PDF

Fig. 11
Fig. 11 This system includes two high-speed CMOS cameras and a DLP projection system.

Download Full Size | PDF

Fig. 12
Fig. 12 The phase measurement results of a standard ceramic plate. (a) The ceramic plate to be measured. (b) The 2D camera image (one of the four-step phase-shifting patterns) from main camera. (c) The wrapped phase is obtained using Eq. (12). (d) The demodulated speckle pattern is obtained using Eq. (15). (e) The phase order map based on geometry constraint. (f) The phase order map based on geometry constraint and the adaptive window image correlation. (g) The phase order map using left-right consistency check. (h) The phase order map using RDC. (i)-(l) The corresponding absolute phase map of (e)-(h).

Download Full Size | PDF

Fig. 13
Fig. 13 The 3D measurement results of a standard ceramic plate. (a) The 3D reconstruction results of the plate. (b) The distribution of the errors of (a). (c) The histogram of (b).

Download Full Size | PDF

Fig. 14
Fig. 14 The measurement results of several objects. (a) A girl statue with a ceramic plate. (b) A Voltaire model and a statue of the Goddess with discontinuous surfaces. (c) The statue of Skadi. (d)-(f) The corresponding 3D reconstruction results of (a)-(c). (g)-(i) The corresponding enlarged detail of the region in (d)-(f).

Download Full Size | PDF

Fig. 15
Fig. 15 The 3D reconstruction results for a one-time transient event: the statue of Voltaire and a free-falling balloon filled with water ( Visualization 1).

Download Full Size | PDF

Fig. 16
Fig. 16 The high-speed 3D reconstruction results for the rip of paper ( Visualization 2).

Download Full Size | PDF

Tables (1)

Tables Icon

Table 1 Accuracy results of the proposed method

View Table

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

  I n ( x , y ) = A ( x , y ) + B ( x , y ) cos  ( Φ ( x , y ) − 2 π n / N )
  I n p ( x , y ) = I n ( x , y ) + C n ( x , y )
ϕ ( x , y ) = tan − 1 I 1 p ( x , y ) − I 3 p ( x , y ) I 0 p ( x , y ) − I 2 p ( x , y ) = tan − 1 2 B ( x , y ) sin  Φ ( x , y ) + C 1 ( x , y ) − C 3 ( x , y ) 2 B ( x , y ) cos  Φ ( x , y ) + C 0 ( x , y ) − C 2 ( x , y )
C 0 ( x , y ) = C 2 ( x , y ) C 1 ( x , y ) = C 3 ( x , y )
  − I n ( x , y ) ≤ C n ( x , y ) ≤ 1 − I n ( x , y )
m a x ( − I 0 ( x , y ) , − I 2 ( x , y ) ) ≤ C 0 ( x , y ) ≤ m i n ( 1 − I 0 ( x , y ) , 1 − I 2 ( x , y ) ) m a x ( − I 1 ( x , y ) , − I 3 ( x , y ) ) ≤ C 1 ( x , y ) ≤ m i n ( 1 − I 1 ( x , y ) , 1 − I 3 ( x , y ) )
D ( x , y ) = ( I 0 p ( x , y ) + I 2 p ( x , y ) ) − ( I 1 p ( x , y ) + I 3 p ( x , y ) ) = 2 ( C 0 ( x , y ) − C 1 ( x , y ) )
{ D i n f ( x , y ) ≤ D ( x , y ) ≤ D s u p ( x , y ) D i n f ( x , y ) = 2 [ m a x ( − I 0 ( x , y ) , − I 2 ( x , y ) ) − m i n ( 1 − I 1 ( x , y ) , 1 − I 3 ( x , y ) ) ] D s u p ( x , y ) = 2 [ m i n ( 1 − I 0 ( x , y ) , 1 − I 2 ( x , y ) ) − m a x ( − I 1 ( x , y ) , − I 3 ( x , y ) ) ]
Δ δ c ≈ S Δ δ p
c o r r = ∑ i , j = − M M ( I r ( x r + i , y r + j ) − I r ¯ ) ( I t ( x t + i , y t + j ) − I t ¯ ) ∑ i , j = − M M ( I r ( x r + i , y r + j ) − I r ¯ ) 2 ∑ i , j = − M M ( I t ( x t + i , y t + j ) − I t ¯ ) 2
I 0 c ( x , y ) = A c ( x , y ) + B c ( x , y ) cos  ( Φ c ( x , y ) ) + C 0 c ( x , y ) I 1 c ( x , y ) = A c ( x , y ) + B c ( x , y ) cos  ( Φ c ( x , y ) − π / 2 ) + C 1 c ( x , y ) I 2 c ( x , y ) = A c ( x , y ) + B c ( x , y ) cos  ( Φ c ( x , y ) − π ) + C 0 c ( x , y ) I 3 c ( x , y ) = A c ( x , y ) + B c ( x , y ) cos  ( Φ c ( x , y ) − 3 π / 2 ) + C 1 c ( x , y )
  ϕ c ( x , y ) = tan − 1 I 1 c ( x , y ) − I 3 c ( x , y ) I 0 c ( x , y ) − I 2 c ( x , y )
  B c ( x , y ) = 1 2 [ ∑ n = 0 3 I n c sin  n π 2 ] 2 + [ ∑ n = 0 3 I n c cos  n π 2 ] 2
M a s k v c ( x , y ) = B c ( x , y ) > T h r 1
I s p k c ( x , y ) = ( I 0 c ( x , y ) + I 2 c ( x , y ) ) − ( I 1 c ( x , y ) + I 3 c ( x , y ) ) 4 B c ( x , y )
  0 < Φ c ( x + 1 , y ) − Φ c ( x , y ) < π for the horizontal direction   0 < a b s ( Φ c ( x , y + 1 ) − Φ c ( x , y ) ) < π for the vertical direction
  Φ s m a l l c ( x , y ) = Φ s m a l l c ( x , y ) + 2 π × R o u n d ( Φ l a r g e c ( x , y ) − Φ s m a l l c ( x , y ) 2 π )
Select as filters
Select Topics Cancel
Top
       
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.
Privacy | Terms of Use