Abstract

We describe a high-resolution, real-time 3D absolute coordinate measurement system based on a phase-shifting method. It acquires 3D shape at 30 frames per second (fps), with 266K points per frame. A tiny marker is encoded in the projected fringe pattern, and detected by software from the texture image and the gamma map. Absolute 3D coordinates are obtained from the detected marker position and the calibrated system parameters. To demonstrate the performance of the system, we measure a hand moving over a depth distance of approximately 700 mm, and human faces with expressions. Applications of such a system include manufacturing, inspection, entertainment, security, medical imaging.

©2006 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
High-speed and dense three-dimensional surface acquisition using defocused binary patterns for spatially isolated objects

Yong Li, Cuifang Zhao, Yixian Qian, Hui Wang, and Hongzhen Jin
Opt. Express 18(21) 21628-21635 (2010)

Superfast multifrequency phase-shifting technique with optimal pulse width modulation

Yajun Wang and Song Zhang
Opt. Express 19(6) 5149-5155 (2011)

GPU-assisted high-resolution, real-time 3-D shape measurement

Song Zhang, Dale Royer, and Shing-Tung Yau
Opt. Express 14(20) 9120-9129 (2006)

More Recommended Articles

References

  • View by:
  • Article Order
  • |
  • Year
  • |
  • Author
  • |
  • Publication
  1. J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004).
    [Crossref]
  2. S. Zhang and P. Huang, “High-Resolution, Real-time 3D Shape Acquisition,” in IEEE Computer Vision and Pattern Recognition Workshop on Realtime 3D Sensors and Their Uses, vol.  3, pp. 28–37 (2004).
  3. K. G. Harding, “Phase Grating Use for Slop Discrimination in Moiré Contouring,” in Proc. SPIE, vol.  1614, pp. 265–270 (1991).
    [Crossref]
  4. Z. J. Geng, “Rainbow 3D Camera: New Concept of High-Speed Three Vision System,” Opt. Eng. 35, 376–383 (1996).
    [Crossref]
  5. P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
    [Crossref]
  6. C. Guan, L. G. Hassebrook, and D. L. Lau, “Composite Structured Light Pattern for Three-Dimensional Video,” Opt. Express 11, 406–417 (2003).
    [Crossref] [PubMed]
  7. S. Rusinkiewicz, O. Hall-Holt, and L. Marc, “Real-Time 3D Model Acquisition,” in SIGGRAPH, pp. 438–446 (2002).
  8. P. S. Huang, C. Zhang, and F. P. Chiang, “High-Speed 3D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42, 163–168 (2003).
    [Crossref]
  9. P. S. Huang and S. Zhang, “A Fast Three-Step Phase Shifting Algorithm,” Appl. Opt. (under press) (2006).
    [Crossref] [PubMed]
  10. Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
    [Crossref]
  11. S. Zhang and P. S. Huang, “A Novel Structured Light System Calibration,” Opt. Eng. (under press) (2006).
    [Crossref]
  12. D. Malacara, ed., Optical Shop Testing (John Wiley and Songs, NY, 1992).
  13. D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, Inc, 1998).
  14. R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43, 464–471 (2004).
    [Crossref]
  15. D. A. Forsyth and J. Ponce, Computer Visoin-A Modern Approach (Prentice-Hall, Inc., New Jersey, 2002).
  16. S. Zhang, “High-Resolution, Real-Time 3D Shape Measurement,” Ph.D. thesis, Stony Brook University, State University of New York, (2005).

2004 (3)

J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004).
[Crossref]

S. Zhang and P. Huang, “High-Resolution, Real-time 3D Shape Acquisition,” in IEEE Computer Vision and Pattern Recognition Workshop on Realtime 3D Sensors and Their Uses, vol.  3, pp. 28–37 (2004).

R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43, 464–471 (2004).
[Crossref]

2003 (3)

C. Guan, L. G. Hassebrook, and D. L. Lau, “Composite Structured Light Pattern for Three-Dimensional Video,” Opt. Express 11, 406–417 (2003).
[Crossref] [PubMed]

P. S. Huang, C. Zhang, and F. P. Chiang, “High-Speed 3D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42, 163–168 (2003).
[Crossref]

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
[Crossref]

2002 (1)

S. Rusinkiewicz, O. Hall-Holt, and L. Marc, “Real-Time 3D Model Acquisition,” in SIGGRAPH, pp. 438–446 (2002).

1999 (1)

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
[Crossref]

1996 (1)

Z. J. Geng, “Rainbow 3D Camera: New Concept of High-Speed Three Vision System,” Opt. Eng. 35, 376–383 (1996).
[Crossref]

1992 (1)

D. Malacara, ed., Optical Shop Testing (John Wiley and Songs, NY, 1992).

1991 (1)

K. G. Harding, “Phase Grating Use for Slop Discrimination in Moiré Contouring,” in Proc. SPIE, vol.  1614, pp. 265–270 (1991).
[Crossref]

Batlle, J.

J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004).
[Crossref]

Bothe, T.

R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43, 464–471 (2004).
[Crossref]

Chiang, F. P.

P. S. Huang, C. Zhang, and F. P. Chiang, “High-Speed 3D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42, 163–168 (2003).
[Crossref]

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
[Crossref]

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
[Crossref]

Forsyth, D. A.

D. A. Forsyth and J. Ponce, Computer Visoin-A Modern Approach (Prentice-Hall, Inc., New Jersey, 2002).

Fu, Q.

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
[Crossref]

Geng, Z. J.

Z. J. Geng, “Rainbow 3D Camera: New Concept of High-Speed Three Vision System,” Opt. Eng. 35, 376–383 (1996).
[Crossref]

Ghiglia, D. C.

D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, Inc, 1998).

Guan, C.

Hall-Holt, O.

S. Rusinkiewicz, O. Hall-Holt, and L. Marc, “Real-Time 3D Model Acquisition,” in SIGGRAPH, pp. 438–446 (2002).

Harding, K. G.

K. G. Harding, “Phase Grating Use for Slop Discrimination in Moiré Contouring,” in Proc. SPIE, vol.  1614, pp. 265–270 (1991).
[Crossref]

Hassebrook, L. G.

Hu, Q.

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
[Crossref]

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
[Crossref]

Huang, P.

S. Zhang and P. Huang, “High-Resolution, Real-time 3D Shape Acquisition,” in IEEE Computer Vision and Pattern Recognition Workshop on Realtime 3D Sensors and Their Uses, vol.  3, pp. 28–37 (2004).

Huang, P. S.

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
[Crossref]

P. S. Huang, C. Zhang, and F. P. Chiang, “High-Speed 3D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42, 163–168 (2003).
[Crossref]

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
[Crossref]

P. S. Huang and S. Zhang, “A Fast Three-Step Phase Shifting Algorithm,” Appl. Opt. (under press) (2006).
[Crossref] [PubMed]

S. Zhang and P. S. Huang, “A Novel Structured Light System Calibration,” Opt. Eng. (under press) (2006).
[Crossref]

Jin, F.

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
[Crossref]

Jüptner, W. P.

R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43, 464–471 (2004).
[Crossref]

Lau, D. L.

Legarda-Sáenz, R.

R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43, 464–471 (2004).
[Crossref]

Marc, L.

S. Rusinkiewicz, O. Hall-Holt, and L. Marc, “Real-Time 3D Model Acquisition,” in SIGGRAPH, pp. 438–446 (2002).

Pages, J.

J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004).
[Crossref]

Ponce, J.

D. A. Forsyth and J. Ponce, Computer Visoin-A Modern Approach (Prentice-Hall, Inc., New Jersey, 2002).

Pritt, M. D.

D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, Inc, 1998).

Rusinkiewicz, S.

S. Rusinkiewicz, O. Hall-Holt, and L. Marc, “Real-Time 3D Model Acquisition,” in SIGGRAPH, pp. 438–446 (2002).

Salvi, J.

J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004).
[Crossref]

Zhang, C.

P. S. Huang, C. Zhang, and F. P. Chiang, “High-Speed 3D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42, 163–168 (2003).
[Crossref]

Zhang, S.

S. Zhang and P. Huang, “High-Resolution, Real-time 3D Shape Acquisition,” in IEEE Computer Vision and Pattern Recognition Workshop on Realtime 3D Sensors and Their Uses, vol.  3, pp. 28–37 (2004).

S. Zhang, “High-Resolution, Real-Time 3D Shape Measurement,” Ph.D. thesis, Stony Brook University, State University of New York, (2005).

P. S. Huang and S. Zhang, “A Fast Three-Step Phase Shifting Algorithm,” Appl. Opt. (under press) (2006).
[Crossref] [PubMed]

S. Zhang and P. S. Huang, “A Novel Structured Light System Calibration,” Opt. Eng. (under press) (2006).
[Crossref]

in IEEE Computer Vision and Pattern Recognition Workshop on Realtime 3D Sensors and Their Uses (1)

S. Zhang and P. Huang, “High-Resolution, Real-time 3D Shape Acquisition,” in IEEE Computer Vision and Pattern Recognition Workshop on Realtime 3D Sensors and Their Uses, vol.  3, pp. 28–37 (2004).

in Proc. SPIE (1)

K. G. Harding, “Phase Grating Use for Slop Discrimination in Moiré Contouring,” in Proc. SPIE, vol.  1614, pp. 265–270 (1991).
[Crossref]

in SIGGRAPH (1)

S. Rusinkiewicz, O. Hall-Holt, and L. Marc, “Real-Time 3D Model Acquisition,” in SIGGRAPH, pp. 438–446 (2002).

Opt. Eng. (5)

P. S. Huang, C. Zhang, and F. P. Chiang, “High-Speed 3D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42, 163–168 (2003).
[Crossref]

R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43, 464–471 (2004).
[Crossref]

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3D Shape Measurement System,” Opt. Eng. 42, 487–493 (2003).
[Crossref]

Z. J. Geng, “Rainbow 3D Camera: New Concept of High-Speed Three Vision System,” Opt. Eng. 35, 376–383 (1996).
[Crossref]

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-Encoded Digital Fringe Projection Technique for High-Speed Three-Dimensional Surface Contouring,” Opt. Eng. 38, 1065–1071 (1999).
[Crossref]

Opt. Express (1)

Pattern Recogn. (1)

J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827–849 (2004).
[Crossref]

Other (6)

S. Zhang and P. S. Huang, “A Novel Structured Light System Calibration,” Opt. Eng. (under press) (2006).
[Crossref]

D. Malacara, ed., Optical Shop Testing (John Wiley and Songs, NY, 1992).

D. C. Ghiglia and M. D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, Inc, 1998).

D. A. Forsyth and J. Ponce, Computer Visoin-A Modern Approach (Prentice-Hall, Inc., New Jersey, 2002).

S. Zhang, “High-Resolution, Real-Time 3D Shape Measurement,” Ph.D. thesis, Stony Brook University, State University of New York, (2005).

P. S. Huang and S. Zhang, “A Fast Three-Step Phase Shifting Algorithm,” Appl. Opt. (under press) (2006).
[Crossref] [PubMed]

Supplementary Material (3)

» Media 1: MOV (2103 KB)     
» Media 2: MOV (626 KB)     
» Media 3: MOV (762 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1.
Fig. 1. Marker detection and removal. (a) 2D texture image with the marker. (b) Gamma map of the fringe images. (c) Inverted gamma map of the fringe images. (d) Texture image after the maker is removed.

Download Full Size | PPT Slide | PDF

Fig. 2.
Fig. 2. Experimental result. Seven frames selected from a smiling sequence of a subject.

Download Full Size | PPT Slide | PDF

Fig. 3.
Fig. 3. (a) (2.1MB) Movie of facial expression. (b) (628KB) Movie of facial expression with texture. (c) (767KB) Movie of hand video moving over a depth range of 700 mm.

Download Full Size | PPT Slide | PDF

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

I 1 ( x , y ) = I′ ( x , y ) + I′′ ( x , y ) cos ⁡ [ ϕ ( x , y ) − 2 π / 3 ] ,
I 2 ( x , y ) = I′ ( x , y ) + I′′ ( x , y ) cos ⁡ [ ϕ ( x , y ) ] ,
I 3 ( x , y ) = I′ ( x , y ) + I′′ ( x , y ) cos ⁡ [ ϕ ( x , y ) − 2 π / 3 ] ,
ϕ ( x , y ) = tan ⁡ − 1 ( 3 I 1 − I 3 2 I 2 − I 1 − I 3 ) ,
γ ( x , y ) = I′′ ( x , y ) I′ ( x , y ) = 3 ( I 1 − I 3 ) 2 + ( 2 I 2 − I 1 − I 3 ) 2 I 1 + I 2 + I 3 ,
f ( x , x ) = 0 1 1 1 0 1 1 2 1 1 1 2 4 2 1 1 1 2 1 1 0 1 1 1 0 .
g ( x , h ( x ) ) = ∑ i = − 2 i = 2 ∑ j = − 2 j = 2 { [ w g × I g ( x + i , h ( x ) + j ) + w t × I t ( x + i , h ( x ) + j ) ] f ( i + 2 , j + 2 ) }
ϕ a ( x , y ) = ϕ ( x , y ) − ϕ 0 ( x 0 , y 0 ) .
ϕ a ( u c , ν c ) = ϕ a ( u p ) .
s c [ u c , ν c , 1 ] T = A c M c [ X w , Y w , Z w , 1 ] T ,
s p [ u p , ν p , 1 ] T = A p M p [ X w , Y w , Z w , 1 ] T
t ( x , y ) = { ( I 1 + I 2 + I 3 ) / 3 = I′ ( x , y ) ∈ fringe area I 0 ( x , y ) ∈ marker area .
f ( x , y ) = ( I 1 + I 2 + I 3 ) / 3 ( 1 + γ ( x , y ) ) / 2
= { I′ ( x , y ) ( x , y ) ∈ fringe area I 0 / 2 ( x , y ) ∈ marker area
I 1 ( x , y ) = a + b ( 1 + cos ⁡ ( ϕ ( x , y ) − 2 π / 3 ) ) ,
I 2 ( x , y ) = a + b ( 1 + cos ⁡ ( ϕ ( x , y ) ) ) ,
I 3 ( x , y ) = a + b ( 1 + cos ⁡ ( ϕ ( x , y ) + 2 π / 3 ) ) .

Metrics