Abstract

We present a method for dynamic recalibration and 3D reconstruction via a structured light system. Assuming that the light planes cast from the digital light projector have been calibrated off-line, we show that the focal length, aspect ratio, and all motion parameters of the camera can be determined on-line. Then the 3D reconstruction can be carried out by either a traditional triangulation method or a more efficient transformation-based method. In the latter method, a single image is sufficient for the whole process of calibration and reconstruction. Thus a hand-held camera can be used. Computer simulation and real data experiments were carried out to validate the method.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
    [CrossRef]
  2. Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000).
    [CrossRef]
  3. P. Sturm and S. Maybank, "On plane-based camera calibration: a general algorithm, singularities, applications," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 23-25 (IEEE, 1999), pp. 432-437.
  4. A. M. McIvor, "Nonlinear calibration of a laser stripe profiler," Opt. Eng. (Bellingham) 41, 205-212 (2002).
    [CrossRef]
  5. J. Y. Bouguet and P. Perona, "3D photography on your desk," Tech. Rep. (California Institute of Technology, 1997).
  6. Y. F. Li and S. Y. Chen, "Automatic recalibration of a structured light vision system," IEEE Trans. Rob. Autom. 19, 259-268 (2003).
    [CrossRef]
  7. S. Y. Chen and Y. F. Li, "Self-recalibration of a color-encoded light system for automated three-dimensional measurements," Meas. Sci. Technol. 14, 33-40 (2003).
    [CrossRef]
  8. D. Fofi, J. Salvi, and E. Mouaddib, "Uncalibrated reconstruction: an adaptation to structured light vision," Pattern Recogn. 36, 1631-1644 (2003).
    [CrossRef]
  9. C. H. Chen and A. C. Kak, "Modeling and calibration of a structured light scanner for 3-D robot vision," in Proceedings of the IEEE Conference on Robotics and Automation (IEEE, 1987), pp. 807-815.
  10. I. D. Reid, "Projective calibration of a laser-stripe range finder," Image Vis. Comput. 14, 659-666 (1996).
    [CrossRef]
  11. D. Q. Huynh, R. A. Owens, and P. E. Hartmann, "Calibrating a structured light stripe system: a novel approach," Int. J. Comput. Vis. 33, 73-86 (1999).
    [CrossRef]
  12. B. Zhang and Y. F. Li, "An efficient method for dynamic calibration and 3D reconstruction using homographic transformation," Sens. Actuators A 119, 349-357 (2005).
  13. M. Li and J. Lavest, "Some aspects of zoom lens camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 8, 1105-1110 (1996).
  14. P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
    [CrossRef]
  15. R. G. Wilson, "Modeling and calibration of automated zoom lenses," Ph.D. dissertation (Carnegie Mellon University, 1994).
  16. A. G. Wiley and K. W. Wong, "Geometric calibration of zoom lenses for computer vision metrology," Photogramm. Eng. Remote Sens. 61, 69-74 (1995).
  17. J. Salvi, J. Pages, and J. Batlle, "Pattern codification strategies in structured light systems," Pattern Recogn. 37, 827-849 (2004).
    [CrossRef]
  18. P. M. Griffin, L. S. Narasimhan, and S. R. Yee, "Generation of uniquely encoded light patterns for range data acquisition," Pattern Recogn. 25, 609-616 (1992).
    [CrossRef]

2005 (2)

B. Zhang and Y. F. Li, "An efficient method for dynamic calibration and 3D reconstruction using homographic transformation," Sens. Actuators A 119, 349-357 (2005).

P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
[CrossRef]

2004 (1)

J. Salvi, J. Pages, and J. Batlle, "Pattern codification strategies in structured light systems," Pattern Recogn. 37, 827-849 (2004).
[CrossRef]

2003 (3)

Y. F. Li and S. Y. Chen, "Automatic recalibration of a structured light vision system," IEEE Trans. Rob. Autom. 19, 259-268 (2003).
[CrossRef]

S. Y. Chen and Y. F. Li, "Self-recalibration of a color-encoded light system for automated three-dimensional measurements," Meas. Sci. Technol. 14, 33-40 (2003).
[CrossRef]

D. Fofi, J. Salvi, and E. Mouaddib, "Uncalibrated reconstruction: an adaptation to structured light vision," Pattern Recogn. 36, 1631-1644 (2003).
[CrossRef]

2002 (1)

A. M. McIvor, "Nonlinear calibration of a laser stripe profiler," Opt. Eng. (Bellingham) 41, 205-212 (2002).
[CrossRef]

2000 (1)

Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000).
[CrossRef]

1999 (1)

D. Q. Huynh, R. A. Owens, and P. E. Hartmann, "Calibrating a structured light stripe system: a novel approach," Int. J. Comput. Vis. 33, 73-86 (1999).
[CrossRef]

1996 (3)

T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
[CrossRef]

I. D. Reid, "Projective calibration of a laser-stripe range finder," Image Vis. Comput. 14, 659-666 (1996).
[CrossRef]

M. Li and J. Lavest, "Some aspects of zoom lens camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 8, 1105-1110 (1996).

1995 (1)

A. G. Wiley and K. W. Wong, "Geometric calibration of zoom lenses for computer vision metrology," Photogramm. Eng. Remote Sens. 61, 69-74 (1995).

1992 (1)

P. M. Griffin, L. S. Narasimhan, and S. R. Yee, "Generation of uniquely encoded light patterns for range data acquisition," Pattern Recogn. 25, 609-616 (1992).
[CrossRef]

Batlle, J.

J. Salvi, J. Pages, and J. Batlle, "Pattern codification strategies in structured light systems," Pattern Recogn. 37, 827-849 (2004).
[CrossRef]

Bouguet, J. Y.

J. Y. Bouguet and P. Perona, "3D photography on your desk," Tech. Rep. (California Institute of Technology, 1997).

Chen, C.

P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
[CrossRef]

Chen, C. H.

C. H. Chen and A. C. Kak, "Modeling and calibration of a structured light scanner for 3-D robot vision," in Proceedings of the IEEE Conference on Robotics and Automation (IEEE, 1987), pp. 807-815.

Chen, S. Y.

S. Y. Chen and Y. F. Li, "Self-recalibration of a color-encoded light system for automated three-dimensional measurements," Meas. Sci. Technol. 14, 33-40 (2003).
[CrossRef]

Y. F. Li and S. Y. Chen, "Automatic recalibration of a structured light vision system," IEEE Trans. Rob. Autom. 19, 259-268 (2003).
[CrossRef]

Cheng, Z.

P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
[CrossRef]

Fofi, D.

D. Fofi, J. Salvi, and E. Mouaddib, "Uncalibrated reconstruction: an adaptation to structured light vision," Pattern Recogn. 36, 1631-1644 (2003).
[CrossRef]

Griffin, P. M.

P. M. Griffin, L. S. Narasimhan, and S. R. Yee, "Generation of uniquely encoded light patterns for range data acquisition," Pattern Recogn. 25, 609-616 (1992).
[CrossRef]

Hartmann, P. E.

D. Q. Huynh, R. A. Owens, and P. E. Hartmann, "Calibrating a structured light stripe system: a novel approach," Int. J. Comput. Vis. 33, 73-86 (1999).
[CrossRef]

Huynh, D. Q.

D. Q. Huynh, R. A. Owens, and P. E. Hartmann, "Calibrating a structured light stripe system: a novel approach," Int. J. Comput. Vis. 33, 73-86 (1999).
[CrossRef]

Kak, A. C.

C. H. Chen and A. C. Kak, "Modeling and calibration of a structured light scanner for 3-D robot vision," in Proceedings of the IEEE Conference on Robotics and Automation (IEEE, 1987), pp. 807-815.

Lavest, J.

M. Li and J. Lavest, "Some aspects of zoom lens camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 8, 1105-1110 (1996).

Li, M.

M. Li and J. Lavest, "Some aspects of zoom lens camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 8, 1105-1110 (1996).

Li, Y. F.

B. Zhang and Y. F. Li, "An efficient method for dynamic calibration and 3D reconstruction using homographic transformation," Sens. Actuators A 119, 349-357 (2005).

Y. F. Li and S. Y. Chen, "Automatic recalibration of a structured light vision system," IEEE Trans. Rob. Autom. 19, 259-268 (2003).
[CrossRef]

S. Y. Chen and Y. F. Li, "Self-recalibration of a color-encoded light system for automated three-dimensional measurements," Meas. Sci. Technol. 14, 33-40 (2003).
[CrossRef]

Maybank, S.

P. Sturm and S. Maybank, "On plane-based camera calibration: a general algorithm, singularities, applications," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 23-25 (IEEE, 1999), pp. 432-437.

McIvor, A. M.

A. M. McIvor, "Nonlinear calibration of a laser stripe profiler," Opt. Eng. (Bellingham) 41, 205-212 (2002).
[CrossRef]

Moons, T.

T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
[CrossRef]

Mouaddib, E.

D. Fofi, J. Salvi, and E. Mouaddib, "Uncalibrated reconstruction: an adaptation to structured light vision," Pattern Recogn. 36, 1631-1644 (2003).
[CrossRef]

Narasimhan, L. S.

P. M. Griffin, L. S. Narasimhan, and S. R. Yee, "Generation of uniquely encoded light patterns for range data acquisition," Pattern Recogn. 25, 609-616 (1992).
[CrossRef]

Owens, R. A.

D. Q. Huynh, R. A. Owens, and P. E. Hartmann, "Calibrating a structured light stripe system: a novel approach," Int. J. Comput. Vis. 33, 73-86 (1999).
[CrossRef]

Pages, J.

J. Salvi, J. Pages, and J. Batlle, "Pattern codification strategies in structured light systems," Pattern Recogn. 37, 827-849 (2004).
[CrossRef]

Pauwels, E.

T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
[CrossRef]

Perona, P.

J. Y. Bouguet and P. Perona, "3D photography on your desk," Tech. Rep. (California Institute of Technology, 1997).

Poo, A.

P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
[CrossRef]

Proesmans, M.

T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
[CrossRef]

Reid, I. D.

I. D. Reid, "Projective calibration of a laser-stripe range finder," Image Vis. Comput. 14, 659-666 (1996).
[CrossRef]

Salvi, J.

J. Salvi, J. Pages, and J. Batlle, "Pattern codification strategies in structured light systems," Pattern Recogn. 37, 827-849 (2004).
[CrossRef]

D. Fofi, J. Salvi, and E. Mouaddib, "Uncalibrated reconstruction: an adaptation to structured light vision," Pattern Recogn. 36, 1631-1644 (2003).
[CrossRef]

Sturm, P.

P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
[CrossRef]

P. Sturm and S. Maybank, "On plane-based camera calibration: a general algorithm, singularities, applications," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 23-25 (IEEE, 1999), pp. 432-437.

van Gool, L.

T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
[CrossRef]

Wiley, A. G.

A. G. Wiley and K. W. Wong, "Geometric calibration of zoom lenses for computer vision metrology," Photogramm. Eng. Remote Sens. 61, 69-74 (1995).

Wilson, R. G.

R. G. Wilson, "Modeling and calibration of automated zoom lenses," Ph.D. dissertation (Carnegie Mellon University, 1994).

Wong, K. W.

A. G. Wiley and K. W. Wong, "Geometric calibration of zoom lenses for computer vision metrology," Photogramm. Eng. Remote Sens. 61, 69-74 (1995).

Yee, S. R.

P. M. Griffin, L. S. Narasimhan, and S. R. Yee, "Generation of uniquely encoded light patterns for range data acquisition," Pattern Recogn. 25, 609-616 (1992).
[CrossRef]

Zhang, B.

B. Zhang and Y. F. Li, "An efficient method for dynamic calibration and 3D reconstruction using homographic transformation," Sens. Actuators A 119, 349-357 (2005).

Zhang, Z.

Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000).
[CrossRef]

Comput. Vis. Image Underst. (1)

P. Sturm, Z. Cheng, C. Chen, and A. Poo, "Focal length calibration from two views: method and analysis of singular cases," Comput. Vis. Image Underst. 99, 58-95 (2005).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (3)

M. Li and J. Lavest, "Some aspects of zoom lens camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 8, 1105-1110 (1996).

T. Moons, L. van Gool, M. Proesmans, and E. Pauwels, "Affine reconstruction from perspective image pairs with a relative object camera translation in between," IEEE Trans. Pattern Anal. Mach. Intell. 18, 77-83 (1996).
[CrossRef]

Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000).
[CrossRef]

IEEE Trans. Rob. Autom. (1)

Y. F. Li and S. Y. Chen, "Automatic recalibration of a structured light vision system," IEEE Trans. Rob. Autom. 19, 259-268 (2003).
[CrossRef]

Image Vis. Comput. (1)

I. D. Reid, "Projective calibration of a laser-stripe range finder," Image Vis. Comput. 14, 659-666 (1996).
[CrossRef]

Int. J. Comput. Vis. (1)

D. Q. Huynh, R. A. Owens, and P. E. Hartmann, "Calibrating a structured light stripe system: a novel approach," Int. J. Comput. Vis. 33, 73-86 (1999).
[CrossRef]

Meas. Sci. Technol. (1)

S. Y. Chen and Y. F. Li, "Self-recalibration of a color-encoded light system for automated three-dimensional measurements," Meas. Sci. Technol. 14, 33-40 (2003).
[CrossRef]

Opt. Eng. (Bellingham) (1)

A. M. McIvor, "Nonlinear calibration of a laser stripe profiler," Opt. Eng. (Bellingham) 41, 205-212 (2002).
[CrossRef]

Pattern Recogn. (3)

D. Fofi, J. Salvi, and E. Mouaddib, "Uncalibrated reconstruction: an adaptation to structured light vision," Pattern Recogn. 36, 1631-1644 (2003).
[CrossRef]

J. Salvi, J. Pages, and J. Batlle, "Pattern codification strategies in structured light systems," Pattern Recogn. 37, 827-849 (2004).
[CrossRef]

P. M. Griffin, L. S. Narasimhan, and S. R. Yee, "Generation of uniquely encoded light patterns for range data acquisition," Pattern Recogn. 25, 609-616 (1992).
[CrossRef]

Photogramm. Eng. Remote Sens. (1)

A. G. Wiley and K. W. Wong, "Geometric calibration of zoom lenses for computer vision metrology," Photogramm. Eng. Remote Sens. 61, 69-74 (1995).

Sens. Actuators A (1)

B. Zhang and Y. F. Li, "An efficient method for dynamic calibration and 3D reconstruction using homographic transformation," Sens. Actuators A 119, 349-357 (2005).

Other (4)

R. G. Wilson, "Modeling and calibration of automated zoom lenses," Ph.D. dissertation (Carnegie Mellon University, 1994).

C. H. Chen and A. C. Kak, "Modeling and calibration of a structured light scanner for 3-D robot vision," in Proceedings of the IEEE Conference on Robotics and Automation (IEEE, 1987), pp. 807-815.

J. Y. Bouguet and P. Perona, "3D photography on your desk," Tech. Rep. (California Institute of Technology, 1997).

P. Sturm and S. Maybank, "On plane-based camera calibration: a general algorithm, singularities, applications," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 23-25 (IEEE, 1999), pp. 432-437.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Layout of the proposed vision system.

Fig. 2
Fig. 2

Relationship between F w and F 3 Π where T is a function of R and M ¯ .

Fig. 3
Fig. 3

Residual errors as a function of the number of used points.

Fig. 4
Fig. 4

Residual errors in the dynamic calibration and 3D reconstruction. (a) Residual errors as a function of noise levels, (b) effects of deviation of the principal point.

Fig. 5
Fig. 5

Configuration of our structured light system.

Fig. 6
Fig. 6

Screen shot of a color-encoded light pattern.

Fig. 7
Fig. 7

Experiment on a workpiece. (a) Workpiece model, (b) CAD model of the restructured result.

Fig. 8
Fig. 8

Experiment on a fan model. (a) fan model, (b) image captured, (c) reconstructed 3D point clouds, (d) CAD model of the clouds.

Fig. 9
Fig. 9

Example of the demonstration system. (a) Profile of the system setup, (b) measured distance between the two planes.

Tables (2)

Tables Icon

Table 1 Measurements and Relative Errors

Tables Icon

Table 2 Comparison of the Error Distance

Equations (38)

Equations on this page are rendered with MathJax. Learn more.

K c = [ τ f 0 0 0 f 0 0 0 1 ] .
λ m ̃ = K c [ R c t c ] M ̃ ,
R = cos θ I + ( 1 cos θ ) rr T + sin θ [ r ] × ,
[ r ] × = [ 0 r 3 r 2 r 3 0 r 1 r 2 r 1 0 ] .
M p i = RM i R M ¯ .
M ̃ p i = T M ̃ i ,
T = [ R R M ¯ 0 T 1 ]
M ̃ p i = ( m p 1 m p 2 0 1 ) T .
m ̃ p i = ST M ̃ i ,
S = [ 1 0 0 0 0 1 0 0 0 0 0 1 ] .
m ̃ c i = λ i H m ̃ p i .
M ̃ i = 1 λ i T w i m ̃ c i ,
T w i = T 1 S T H 1 .
M ̃ i = T 1 S T m ̃ p i .
λ m ̃ c i = K c [ R c t c ] T 1 S T m ̃ p i .
σ H = K c [ R c t c ] T 1 S T ,
T s 1 = [ R s T M ¯ s 0 T 1 ] ,
R c = [ r 1 r 2 r 3 ] R s = [ r 1 s r 2 s r 3 s ] ,
σ H s = [ τ f r 1 r 1 s T τ f r 1 r 2 s T τ f ( r 1 M ¯ s + t 1 ) f r 2 r 1 s T f r 2 r 2 s T f ( r 2 M ¯ s + t 2 ) r 3 r 1 s T r 3 r 2 s T r 3 M ¯ s + t 3 ] .
R c r 1 s T = [ σ τ f H 11 σ f H 21 σ H 31 ] ,
R c r 2 s T = [ σ τ f H 12 σ f H 22 σ H 32 ] ,
[ σ τ f H 11 ] 2 + [ σ f H 21 ] 2 + ( σ H 31 ) 2 = 1 ,
[ σ τ f H 12 ] 2 + [ σ f H 22 ] 2 + ( σ H 32 ) 2 = 1 ,
[ σ τ f ] 2 H 11 H 12 + [ σ f ] 2 H 21 H 22 + σ 2 H 31 H 32 = 0 .
f = a 2 c 1 a 1 c 2 b 1 c 2 b 2 c 1 ,
τ = b 1 c 2 b 2 c 1 a 1 b 2 a 2 b 1 ,
σ = ± τ 2 f 2 τ 2 f 2 H 31 2 + τ 2 H 21 2 + H 11 2 ,
a 1 = H 21 2 H 22 2 , b 1 = H 31 2 H 32 2 , c 1 = H 11 2 H 12 2 ,
a 2 = H 21 H 22 , b 2 = H 31 H 32 , c 2 = H 11 H 12 .
r 1 r 1 s T = 1 τ f A 11 ,
r 1 r 2 s T = 1 f A 12 ,
r 1 2 = 1 .
r 2 r 1 s T = 1 f A 21 ,
r 2 r 2 s T = 1 f A 22 ,
r 2 2 = 1 .
t 1 = 1 τ f A ̃ 13 r 1 M ¯ s ,
t 2 = 1 f A ̃ 23 r 2 M ¯ s ,
t 3 = A 33 r 3 M ¯ s .

Metrics