Abstract

The majority of existing real-time 3D shape measurement systems only generate non-nature texture (i.e., having illumination other than ambient lights) that induces shadow related issues. This paper presents a method that can simultaneously capture natural 2D color texture and 3D shape in real time. Specifically, we use an infrared fringe projection system to acquire 3D shapes, and a secondary color camera to simultaneously capture 2D color images of the object. Finally, we develop a flexible and simple calibration technique to determine the mapping between the 2D color image and the 3D geometry. Experimental results demonstrate the success of the proposed technique.

© 2013 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Geng, “Structured-light 3D surface imaging: a tutorial,” Adv. Opt. Photon.3(2), 128–160 (2011).
    [CrossRef]
  2. S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng.48(2), 149–158 (2010).
    [CrossRef]
  3. S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph.21(3), 438–446 (2002).
    [CrossRef]
  4. L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
    [CrossRef]
  5. S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng.45(12), 123601 (2006).
    [CrossRef]
  6. R. Höfling and P. Aswendt, “Real time 3D shape recording by DLP-based all-digital surface encoding,” in Proc. SPIE, 7210, 72,100E1–8 (2009).
  7. K. Liu, Y. Wang, D. L. Lau, Q. Hao, and L. G. Hassebrook, “Dual-frequency pattern scheme for high-speed 3-D shape measurement,” Opt. Express18(5), 5229–5244 (2010).
    [CrossRef] [PubMed]
  8. M. Schaffer, M. Grosse, and R. Kowarschik, “High-speed pattern projection for three-dimensional shape measurement using laser speckles,” Appl. Opt.49(18), 3622–3629 (2010).
    [CrossRef] [PubMed]
  9. Y. Li, C. Zhao, Y. Qian, H. Wang, and H. Jin, “High-speed and dense three-dimensional surface acquisition using defocused binary patterns for spatially isolated objects,” Opt. Express18(21), 21628–21635 (2010).
    [CrossRef] [PubMed]
  10. S. Zhang, D. Van Der Weide, and J. Oliver, “Superfast phase-shifting method for 3-D shape measurement,” Opt. Express18(9), 9684–9689 (2010).
    [CrossRef] [PubMed]
  11. J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng.45(12), 013602 (2006).
  12. L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in The 1st IEEE International Symposium on 3D Data Processing, Visualization, and Transmission, 24–36 (2002).
  13. Z. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in color fringe projection,” J. Opt. A, Pure Appl. Opt.9(6), S81–S86 (2007).
    [CrossRef]
  14. Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express14(14), 6444–6455 (2006).
    [CrossRef] [PubMed]
  15. S. Zhang and S.-T. Yau, “Simultaneous three-dimensional geometry and color texture acquisition using single color camera,” Opt. Eng.47(12), 123604 (2008).
    [CrossRef]
  16. X. Liu, X. Peng, H. Chen, D. He, and B. Z. Gao, “Strategy for automatic and complete three-dimensional optical digitization,” Opt. Lett.37(15), 3126–3128 (2012).
    [CrossRef] [PubMed]
  17. Y. Wang, J. I. Laughner, I. R. Efimov, and S. Zhang, “3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique,” Opt. Express21(5), 5822–5832 (2013).
    [CrossRef] [PubMed]
  18. S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng.45(8), 083601 (2006).
    [CrossRef]
  19. Z. Y. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell.22(11), 1330–1334 (2000).
    [CrossRef]
  20. A. Geiger, F. Moosmann, O. Car, and B. Schuster, “A toolbox for automatic calibration of range and camera sensors using a single shot,” InProc. of International Conference on Robotics and Automation (ICRA),3936–3943 (2012).

2013

2012

2011

2010

2008

S. Zhang and S.-T. Yau, “Simultaneous three-dimensional geometry and color texture acquisition using single color camera,” Opt. Eng.47(12), 123604 (2008).
[CrossRef]

2007

Z. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in color fringe projection,” J. Opt. A, Pure Appl. Opt.9(6), S81–S86 (2007).
[CrossRef]

2006

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng.45(8), 083601 (2006).
[CrossRef]

S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng.45(12), 123601 (2006).
[CrossRef]

J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng.45(12), 013602 (2006).

Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express14(14), 6444–6455 (2006).
[CrossRef] [PubMed]

2004

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
[CrossRef]

2002

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph.21(3), 438–446 (2002).
[CrossRef]

2000

Z. Y. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell.22(11), 1330–1334 (2000).
[CrossRef]

Car, O.

A. Geiger, F. Moosmann, O. Car, and B. Schuster, “A toolbox for automatic calibration of range and camera sensors using a single shot,” InProc. of International Conference on Robotics and Automation (ICRA),3936–3943 (2012).

Chen, H.

Chiang, F.-P.

J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng.45(12), 013602 (2006).

Curless, B.

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
[CrossRef]

Efimov, I. R.

Gao, B. Z.

Geiger, A.

A. Geiger, F. Moosmann, O. Car, and B. Schuster, “A toolbox for automatic calibration of range and camera sensors using a single shot,” InProc. of International Conference on Robotics and Automation (ICRA),3936–3943 (2012).

Geng, G.

Grosse, M.

Hall-Holt, O.

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph.21(3), 438–446 (2002).
[CrossRef]

Hao, Q.

Hassebrook, L. G.

He, D.

Huang, P. S.

S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng.45(12), 123601 (2006).
[CrossRef]

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng.45(8), 083601 (2006).
[CrossRef]

J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng.45(12), 013602 (2006).

Jin, H.

Kowarschik, R.

Lau, D. L.

Laughner, J. I.

Levoy, M.

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph.21(3), 438–446 (2002).
[CrossRef]

Li, Y.

Liu, K.

Liu, X.

Moosmann, F.

A. Geiger, F. Moosmann, O. Car, and B. Schuster, “A toolbox for automatic calibration of range and camera sensors using a single shot,” InProc. of International Conference on Robotics and Automation (ICRA),3936–3943 (2012).

Oliver, J.

Pan, J.

J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng.45(12), 013602 (2006).

Peng, X.

Qian, Y.

Rusinkiewicz, S.

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph.21(3), 438–446 (2002).
[CrossRef]

Schaffer, M.

Schuster, B.

A. Geiger, F. Moosmann, O. Car, and B. Schuster, “A toolbox for automatic calibration of range and camera sensors using a single shot,” InProc. of International Conference on Robotics and Automation (ICRA),3936–3943 (2012).

Seitz, S. M.

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
[CrossRef]

Snavely, N.

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
[CrossRef]

Towers, C. E.

Z. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in color fringe projection,” J. Opt. A, Pure Appl. Opt.9(6), S81–S86 (2007).
[CrossRef]

Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express14(14), 6444–6455 (2006).
[CrossRef] [PubMed]

Towers, D. P.

Z. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in color fringe projection,” J. Opt. A, Pure Appl. Opt.9(6), S81–S86 (2007).
[CrossRef]

Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express14(14), 6444–6455 (2006).
[CrossRef] [PubMed]

Van Der Weide, D.

Wang, H.

Wang, Y.

Yau, S.-T.

S. Zhang and S.-T. Yau, “Simultaneous three-dimensional geometry and color texture acquisition using single color camera,” Opt. Eng.47(12), 123604 (2008).
[CrossRef]

Zhang, L.

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
[CrossRef]

Zhang, S.

Y. Wang, J. I. Laughner, I. R. Efimov, and S. Zhang, “3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique,” Opt. Express21(5), 5822–5832 (2013).
[CrossRef] [PubMed]

S. Zhang, D. Van Der Weide, and J. Oliver, “Superfast phase-shifting method for 3-D shape measurement,” Opt. Express18(9), 9684–9689 (2010).
[CrossRef] [PubMed]

S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng.48(2), 149–158 (2010).
[CrossRef]

S. Zhang and S.-T. Yau, “Simultaneous three-dimensional geometry and color texture acquisition using single color camera,” Opt. Eng.47(12), 123604 (2008).
[CrossRef]

S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng.45(12), 123601 (2006).
[CrossRef]

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng.45(8), 083601 (2006).
[CrossRef]

Zhang, Z.

Z. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in color fringe projection,” J. Opt. A, Pure Appl. Opt.9(6), S81–S86 (2007).
[CrossRef]

Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express14(14), 6444–6455 (2006).
[CrossRef] [PubMed]

Zhang, Z. Y.

Z. Y. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell.22(11), 1330–1334 (2000).
[CrossRef]

Zhao, C.

ACM Trans. Graph.

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D model acquisition,” ACM Trans. Graph.21(3), 438–446 (2002).
[CrossRef]

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime faces: high-resolution capture for modeling and animation,” ACM Trans. Graph.23(3), 548–558 (2004).
[CrossRef]

Adv. Opt. Photon.

Appl. Opt.

IEEE Trans. Pattern Anal. Mach. Intell.

Z. Y. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell.22(11), 1330–1334 (2000).
[CrossRef]

J. Opt. A, Pure Appl. Opt.

Z. Zhang, C. E. Towers, and D. P. Towers, “Phase and colour calculation in color fringe projection,” J. Opt. A, Pure Appl. Opt.9(6), S81–S86 (2007).
[CrossRef]

Opt. Eng.

S. Zhang and S.-T. Yau, “Simultaneous three-dimensional geometry and color texture acquisition using single color camera,” Opt. Eng.47(12), 123604 (2008).
[CrossRef]

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng.45(8), 083601 (2006).
[CrossRef]

J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng.45(12), 013602 (2006).

S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng.45(12), 123601 (2006).
[CrossRef]

Opt. Express

Opt. Lasers Eng.

S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng.48(2), 149–158 (2010).
[CrossRef]

Opt. Lett.

Other

A. Geiger, F. Moosmann, O. Car, and B. Schuster, “A toolbox for automatic calibration of range and camera sensors using a single shot,” InProc. of International Conference on Robotics and Automation (ICRA),3936–3943 (2012).

R. Höfling and P. Aswendt, “Real time 3D shape recording by DLP-based all-digital surface encoding,” in Proc. SPIE, 7210, 72,100E1–8 (2009).

L. Zhang, B. Curless, and S. M. Seitz, “Rapid shape acquisition using color structured light and multi-pass dynamic programming,” in The 1st IEEE International Symposium on 3D Data Processing, Visualization, and Transmission, 24–36 (2002).

Supplementary Material (6)

» Media 1: MOV (5207 KB)     
» Media 2: MOV (2243 KB)     
» Media 3: MOV (2149 KB)     
» Media 4: MOV (2243 KB)     
» Media 5: MOV (2149 KB)     
» Media 6: MOV (5207 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1
Fig. 1

Photograph of measurement system.

Fig. 2
Fig. 2

Absolute phase retrieval using two frequency phase shifting technique. (a) High-frequency fringe pattern; (b) Low-frequency fringe pattern; (c) Wrapped phase map using high-frequency patterns; (d) Wrapped phase map using low-frequency patterns; (e) Unwrapped phase map for high-frequency fringe patterns.

Fig. 3
Fig. 3

Experimental results with texture mapping. (a) 3D reconstructed face using the infrared fringe patterns (Media 1); (b) Average b/w texture images obtained from the phase-shifted fringe patterns; (c) 3D results with b/w texture mapping (Media 2); (d) 2D color texture image captured by the color camera; (e) 2D color image mapped to the infrared camera; (f) 3D results with color texture mapping (Media 3).

Equations (7)

Equations on this page are rendered with MathJax. Learn more.

I n (x,y)= I (x,y)+ I (x,y)cos[ϕ(x,y)+2πn/5].
ϕ(x,y) = tan 1 [ n=1 5 I n (x,y)sin(2πn/5) n=1 5 I n (x,y)cos(2πn/5) ].
sI=A[R,t] X w =H X w .
s[ u v 1 ]=H X w =[ h 11 h 12 h 13 h 14 h 21 h 22 h 23 h 24 h 31 h 32 h 33 h 34 ][ x w y w z w 1 ].
Qh=0,
h=[ h 11 h 12 h 13 h 14 h 21 h 22 h 23 h 24 h 31 h 32 h 33 h 34 ],
Q=[ x 1 w y 1 w z 1 w 1 0 0 0 0 u 1 x 1 w u 1 y 1 w u 1 z 1 w u 1 0 0 0 0 x 1 w y 1 w z 1 w 1 v 1 x 1 w v 1 y 1 w v 1 y 1 w v 1 x N w y N w z N w 1 0 0 0 0 u N x N w u N y N w u N z N w u N 0 0 0 0 x N w y N w z N w 1 v N x N w v N y N w v N z N w v N ].

Metrics