Abstract

Automatic optical inspection (AOI) for printed circuit board (PCB) assembly plays a very important role in modern electronics manufacturing industries. Well-developed inspection machines in each assembly process are required to ensure the manufacturing quality of the electronics products. However, generally almost all AOI machines are based on 2D image-analysis technology. In this paper, a 3D-measurement-method-based AOI system is proposed consisting of a phase shifting profilometer and a stereo vision system for assembled electronic components on a PCB after component mounting and the reflow process. In this system information from two visual systems is fused to extend the shape measurement range limited by 2π phase ambiguity of the phase shifting profilometer, and finally to maintain fine measurement resolution and high accuracy of the phase shifting profilometer with the measurement range extended by the stereo vision. The main purpose is to overcome the low inspection reliability problem of 2D-based inspection machines by using 3D information of components. The 3D shape measurement results on PCB-mounted electronic components are shown and compared with results from contact and noncontact 3D measuring machines. Based on a series of experiments, the usefulness of the proposed sensor system and its fusion technique are discussed and analyzed in detail.

© 2009 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
    [CrossRef]
  2. K. Gasvik, Optical Metrology, 3rd ed. (Wiley, 2002).
    [CrossRef]
  3. V. I. Gushov and Y. N. Solodkin, “Automatic processing of fringe patterns in integer interferometer,” Opt. Lasers Eng. 14, 311-324 (1991).
    [CrossRef]
  4. J. Zhong and M. Wang, “Phase unwrapping by lookup table method: application to phase maps with singular points,” Opt. Eng. 38, 2075-2081 (1999).
    [CrossRef]
  5. J. M. Huntley and H. O. Saldner, “Temporal phase-unwrapping algorithm for automated interferogram analysis,” Appl. Opt. 32, 3047-3052 (1993).
    [CrossRef] [PubMed]
  6. H. O. Saldner and J. M. Huntley, “Temporal phase unwrapping: application to surface profiling of discontinuous objects,” Appl. Opt. 36, 2770-2775 (1997).
    [CrossRef] [PubMed]
  7. S. Kakunai, K. Iwata, and T. Sakamoto, “Profile measurement by projecting two gratings with different pitches,” Opt. Rev. 1, 296-298 (1994).
  8. D. Bergmann, “New approach for automatic surface reconstruction with coded light,” Proc. SPIE 2572, 2-9 (1995).
    [CrossRef]
  9. J. Gühring, “Dense 3-D surface acquisition by structured light using off-the-shelf components,” Proc. SPIE 4309, 220-231 (2001).
  10. G. Wiora, “High resolution measurement of phase-shift amplitude and numeric object phase calculation,” Proc. SPIE 4117, 289-299, (2000).
    [CrossRef]
  11. K. Sato, “Range imaging based on moving pattern light and spatio-temporal matched filter”, in Proceedings of IEEE International Conference on Image Processing (IEEE, 1996), Vol. 1, pp. 33-36.
    [CrossRef]
  12. H. Lee and H. Cho, “Stereo Moire technique: a novel 3D measurement method using a stereo camera and a digital pattern projector,” Int. J. Optomechatr. 1, 209-230 (2007).
    [CrossRef]
  13. R. Ishiyama, S. Sakamoto, J. Tajima, T. Okatani, and K. Deguchi, “Absolute phase measurements using geometric constraints between multiple cameras and projectors,” Appl. Opt. 46, 3528-3538 (2007).
    [CrossRef] [PubMed]
  14. D. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).
  15. M. Laikin, Lens Design (Marcel Dekker, 1990).
  16. X. Su and W. Chen, “Reliability-guided phase unwrapping algorithm: a review,” Opt. Lasers Eng. 42, 245-261 (2004).
    [CrossRef]
  17. A. Asundi and Z. Wensen, “Fast phase-unwrapping algorithm based on a gray-scale mask and flood fill,” Appl. Opt. 37, 5416-5420 (1998).
    [CrossRef]
  18. R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed. (Prentice-Hall, 2002).
  19. M. Y. Kim and K. I. Koh, “Shadow free Moire interferometer with dual projection for in-line inspection of light emitting diodes,” Int. J. Optomechatr. 1, 404-424 (2007).
    [CrossRef]
  20. R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge, 2000).
  21. H. Cho, Optomechatronics (Taylor & Francis, 2005).
    [CrossRef]
  22. Electronic Industries Alliance (EIA), http://www.eia.org/.
  23. JEDEC, http://jedec.org/.

2007 (3)

H. Lee and H. Cho, “Stereo Moire technique: a novel 3D measurement method using a stereo camera and a digital pattern projector,” Int. J. Optomechatr. 1, 209-230 (2007).
[CrossRef]

M. Y. Kim and K. I. Koh, “Shadow free Moire interferometer with dual projection for in-line inspection of light emitting diodes,” Int. J. Optomechatr. 1, 404-424 (2007).
[CrossRef]

R. Ishiyama, S. Sakamoto, J. Tajima, T. Okatani, and K. Deguchi, “Absolute phase measurements using geometric constraints between multiple cameras and projectors,” Appl. Opt. 46, 3528-3538 (2007).
[CrossRef] [PubMed]

2004 (1)

X. Su and W. Chen, “Reliability-guided phase unwrapping algorithm: a review,” Opt. Lasers Eng. 42, 245-261 (2004).
[CrossRef]

2000 (1)

G. Wiora, “High resolution measurement of phase-shift amplitude and numeric object phase calculation,” Proc. SPIE 4117, 289-299, (2000).
[CrossRef]

1999 (1)

J. Zhong and M. Wang, “Phase unwrapping by lookup table method: application to phase maps with singular points,” Opt. Eng. 38, 2075-2081 (1999).
[CrossRef]

1998 (1)

1997 (1)

1996 (1)

M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
[CrossRef]

1995 (1)

D. Bergmann, “New approach for automatic surface reconstruction with coded light,” Proc. SPIE 2572, 2-9 (1995).
[CrossRef]

1994 (1)

S. Kakunai, K. Iwata, and T. Sakamoto, “Profile measurement by projecting two gratings with different pitches,” Opt. Rev. 1, 296-298 (1994).

1993 (1)

1991 (1)

V. I. Gushov and Y. N. Solodkin, “Automatic processing of fringe patterns in integer interferometer,” Opt. Lasers Eng. 14, 311-324 (1991).
[CrossRef]

Asundi, A.

Bergmann, D.

D. Bergmann, “New approach for automatic surface reconstruction with coded light,” Proc. SPIE 2572, 2-9 (1995).
[CrossRef]

Chen, W.

X. Su and W. Chen, “Reliability-guided phase unwrapping algorithm: a review,” Opt. Lasers Eng. 42, 245-261 (2004).
[CrossRef]

Cho, H.

H. Lee and H. Cho, “Stereo Moire technique: a novel 3D measurement method using a stereo camera and a digital pattern projector,” Int. J. Optomechatr. 1, 209-230 (2007).
[CrossRef]

H. Cho, Optomechatronics (Taylor & Francis, 2005).
[CrossRef]

Dagli, C. H.

M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
[CrossRef]

Deguchi, K.

Ercal, F.

M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
[CrossRef]

Forsyth, D.

D. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).

Gasvik, K.

K. Gasvik, Optical Metrology, 3rd ed. (Wiley, 2002).
[CrossRef]

Gonzalez, R.

R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed. (Prentice-Hall, 2002).

Gühring, J.

J. Gühring, “Dense 3-D surface acquisition by structured light using off-the-shelf components,” Proc. SPIE 4309, 220-231 (2001).

Gushov, V. I.

V. I. Gushov and Y. N. Solodkin, “Automatic processing of fringe patterns in integer interferometer,” Opt. Lasers Eng. 14, 311-324 (1991).
[CrossRef]

Hartley, R.

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge, 2000).

Huntley, J. M.

Ishiyama, R.

Iwata, K.

S. Kakunai, K. Iwata, and T. Sakamoto, “Profile measurement by projecting two gratings with different pitches,” Opt. Rev. 1, 296-298 (1994).

Kakunai, S.

S. Kakunai, K. Iwata, and T. Sakamoto, “Profile measurement by projecting two gratings with different pitches,” Opt. Rev. 1, 296-298 (1994).

Kim, M. Y.

M. Y. Kim and K. I. Koh, “Shadow free Moire interferometer with dual projection for in-line inspection of light emitting diodes,” Int. J. Optomechatr. 1, 404-424 (2007).
[CrossRef]

Koh, K. I.

M. Y. Kim and K. I. Koh, “Shadow free Moire interferometer with dual projection for in-line inspection of light emitting diodes,” Int. J. Optomechatr. 1, 404-424 (2007).
[CrossRef]

Laikin, M.

M. Laikin, Lens Design (Marcel Dekker, 1990).

Lee, H.

H. Lee and H. Cho, “Stereo Moire technique: a novel 3D measurement method using a stereo camera and a digital pattern projector,” Int. J. Optomechatr. 1, 209-230 (2007).
[CrossRef]

Moganti, M.

M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
[CrossRef]

Okatani, T.

Ponce, J.

D. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).

Sakamoto, S.

Sakamoto, T.

S. Kakunai, K. Iwata, and T. Sakamoto, “Profile measurement by projecting two gratings with different pitches,” Opt. Rev. 1, 296-298 (1994).

Saldner, H. O.

Sato, K.

K. Sato, “Range imaging based on moving pattern light and spatio-temporal matched filter”, in Proceedings of IEEE International Conference on Image Processing (IEEE, 1996), Vol. 1, pp. 33-36.
[CrossRef]

Solodkin, Y. N.

V. I. Gushov and Y. N. Solodkin, “Automatic processing of fringe patterns in integer interferometer,” Opt. Lasers Eng. 14, 311-324 (1991).
[CrossRef]

Su, X.

X. Su and W. Chen, “Reliability-guided phase unwrapping algorithm: a review,” Opt. Lasers Eng. 42, 245-261 (2004).
[CrossRef]

Tajima, J.

Tsunekawa, S.

M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
[CrossRef]

Wang, M.

J. Zhong and M. Wang, “Phase unwrapping by lookup table method: application to phase maps with singular points,” Opt. Eng. 38, 2075-2081 (1999).
[CrossRef]

Wensen, Z.

Wiora, G.

G. Wiora, “High resolution measurement of phase-shift amplitude and numeric object phase calculation,” Proc. SPIE 4117, 289-299, (2000).
[CrossRef]

Woods, R.

R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed. (Prentice-Hall, 2002).

Zhong, J.

J. Zhong and M. Wang, “Phase unwrapping by lookup table method: application to phase maps with singular points,” Opt. Eng. 38, 2075-2081 (1999).
[CrossRef]

Zisserman, A.

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge, 2000).

Appl. Opt. (4)

Comput. Vis. Image Underst. (1)

M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, “Automatic PCB inspection algorithms: a survey,” Comput. Vis. Image Underst. 63, 287-313 (1996).
[CrossRef]

Int. J. Optomechatr. (2)

H. Lee and H. Cho, “Stereo Moire technique: a novel 3D measurement method using a stereo camera and a digital pattern projector,” Int. J. Optomechatr. 1, 209-230 (2007).
[CrossRef]

M. Y. Kim and K. I. Koh, “Shadow free Moire interferometer with dual projection for in-line inspection of light emitting diodes,” Int. J. Optomechatr. 1, 404-424 (2007).
[CrossRef]

Opt. Eng. (1)

J. Zhong and M. Wang, “Phase unwrapping by lookup table method: application to phase maps with singular points,” Opt. Eng. 38, 2075-2081 (1999).
[CrossRef]

Opt. Lasers Eng. (2)

X. Su and W. Chen, “Reliability-guided phase unwrapping algorithm: a review,” Opt. Lasers Eng. 42, 245-261 (2004).
[CrossRef]

V. I. Gushov and Y. N. Solodkin, “Automatic processing of fringe patterns in integer interferometer,” Opt. Lasers Eng. 14, 311-324 (1991).
[CrossRef]

Opt. Rev. (1)

S. Kakunai, K. Iwata, and T. Sakamoto, “Profile measurement by projecting two gratings with different pitches,” Opt. Rev. 1, 296-298 (1994).

Proc. SPIE (3)

D. Bergmann, “New approach for automatic surface reconstruction with coded light,” Proc. SPIE 2572, 2-9 (1995).
[CrossRef]

J. Gühring, “Dense 3-D surface acquisition by structured light using off-the-shelf components,” Proc. SPIE 4309, 220-231 (2001).

G. Wiora, “High resolution measurement of phase-shift amplitude and numeric object phase calculation,” Proc. SPIE 4117, 289-299, (2000).
[CrossRef]

Other (9)

K. Sato, “Range imaging based on moving pattern light and spatio-temporal matched filter”, in Proceedings of IEEE International Conference on Image Processing (IEEE, 1996), Vol. 1, pp. 33-36.
[CrossRef]

D. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).

M. Laikin, Lens Design (Marcel Dekker, 1990).

K. Gasvik, Optical Metrology, 3rd ed. (Wiley, 2002).
[CrossRef]

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge, 2000).

H. Cho, Optomechatronics (Taylor & Francis, 2005).
[CrossRef]

Electronic Industries Alliance (EIA), http://www.eia.org/.

JEDEC, http://jedec.org/.

R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed. (Prentice-Hall, 2002).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1
Fig. 1

Manufacturing process of a printed circuit board.

Fig. 2
Fig. 2

Illustration of fringe distortion in the existence of an object.

Fig. 3
Fig. 3

Fringe shift due to object shape.

Fig. 4
Fig. 4

Basic configuration of a telecentric system and its imaging characteristic.

Fig. 5
Fig. 5

Basic concept of fusion of two shape measurement methods: phase-based and stereo vision.

Fig. 6
Fig. 6

Rectified stereo vision system of orthographic cameras.

Fig. 7
Fig. 7

Schematic diagram of the hardware system. Ray path is illustrated for right illumination and left camera acquisition.

Fig. 8
Fig. 8

System configuration.

Fig. 9
Fig. 9

Projective distortion of the fringe pattern.

Fig. 10
Fig. 10

Calibrated k map in the image coordinate.

Fig. 11
Fig. 11

Illustration of the measurement procedure of the proposed method.

Fig. 12
Fig. 12

Standard height specimen.

Fig. 13
Fig. 13

Calculation of the column height of the standard specimen from the acquired height map.

Fig. 14
Fig. 14

Electronic components of which the results are compared with those of contact type CMM measurement. The shape and size is denoted in the parenthesis by the JEDEC standard.

Fig. 15
Fig. 15

Electronic components of which the results are compared with those of noncontact type CMM measurement. The shape and size is denoted in the parenthesis by the JEDEC standard. (a) component 5 (3216), (b) component 6 (2012), (c) component 7 (SOT-23), (d) component 8 (2012), (e) component 9 (3216).

Fig. 16
Fig. 16

Some examples of component shape measurement results.

Tables (6)

Tables Icon

Table 1 Classification of Pixels According to Reliability for Phase Unwrapping

Tables Icon

Table 2 Specifications of the Proposed System

Tables Icon

Table 3 Height Measurement Results of the Standard Specimen for Two Columns a Using the Proposed System

Tables Icon

Table 4 Specifications of Contact and Noncontact Type 3D CMMs Used to Measure the Reference Height of the Electronic Components

Tables Icon

Table 5 Height Measurement Results of the Electronic Components by Two Types of CMM

Tables Icon

Table 6 Height Measurement Results of the Electronic Components Using the Proposed System and Their Comparison to Those of CMM

Equations (21)

Equations on this page are rendered with MathJax. Learn more.

Δ ( x , y ) = h ( x , y ) ( tan θ p + tan θ c ) ,
φ ( x , y ) = 2 π λ Δ ( x , y ) ,
I ( x , y ) = a ( x , y ) + b ( x , y ) cos { φ r ( x , y ) + φ ( x , y ) } .
φ r ( x , y ) = 2 π λ x x + 2 π λ y y .
V ( x , y ) = b ( x , y ) a ( x , y ) .
h ( x , y ) = λ 2 π ( tan θ p + tan θ c ) φ ( x , y ) = k φ ( x , y )
k = λ 2 π ( tan θ p + tan θ c ) .
I n ( x , y ) = a ( x , y ) + b ( x , y ) cos { φ r ( x , y ) + φ ( x , y ) + π 2 n } , n = 0 , 1 , 2 , 3.
φ ( x , y ) = tan 1 [ I 3 ( x , y ) I 1 ( x , y ) I 0 ( x , y ) I 2 ( x , y ) ] φ r ( x , y ) + 2 N π = φ p ( x , y ) + 2 N π ,
V ( x , y ) = ( I 3 ( x , y ) I 1 ( x , y ) ) 2 + ( I 0 ( x , y ) I 2 ( x , y ) ) 2 2 I A ( x , y ) ,
I A ( x , y ) = 1 4 n = 0 3 I n ( x , y ) .
φ ( x , y ) = φ p ( x , y ) + 2 N π .
d p = h ( sin θ 1 + sin θ 2 ) .
h = d p ( sin θ 1 + sin θ 2 ) .
N = [ h 2 π k ] = [ d p 2 π k ( sin θ 1 + sin θ 2 ) ] .
R ( x , y ) = 0.5 V ( x , y ) + 0.5 R diff ( x , y )
R diff ( x , y ) = max [ 1 α D ( x , y ) , 0 ] .
D ( x , y ) = 1 4 π [ a b s { p ( x + 1 , y ) p ( x 1 , y ) } + a b s { p ( x , y + 1 ) p ( x , y 1 ) } ] ,
h ( x , y ) = k { φ p ( x , y ) + 2 N π } .
d p = min δ { x , y W a b s ( I r ( x , y ) I l ( x + Δ , y ) ) } ,
e = | h T μ h | h T ,

Metrics