Abstract

This paper addresses 3D distortion-tolerant object recognition using Integral Imaging (II). II is one of the techniques considered for 3D image recording and display. Multiple elemental images are captured to record different but continuous viewing zones of 3D scenes. We develop a distortion-tolerant 3D object recognition by using the II capture system. We adopt Principal Component Analysis (PCA) and Fisher Linear Discriminant (FLD) classifier followed by the nearest neighbor decision rule and the statistical distance metric. Performance is analyzed in terms of probability of correct decision, Root Mean Square Error (RMSE) between raw input vectors and reconstructed vectors from the PCA subspace and the PCA-FLD cost function. Decision strategies are compared by the varying number of training data.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification2nd, (Wiley Interscience, New York, 2001).
  2. A. K. Jain, Fundamentals of digital image processing, (Prentice-Hall Inc., 1989).
  3. C. M. Bishop, Neural networks for pattern recognition, (Oxford University Press, New Yeork, 1995).
  4. A. Mahalanobis and F. Goudail, “Methods for automatic target recognition by use of electro-optic sensors: introduction to the feature Issue,” Appl. Opt. 43, 207–209 (2004).
    [CrossRef]
  5. C. F. Olson and D. P. Huttenlocher, “Automatic target recognition by matching oriented edge pixels,” IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103–113 (1997).
  6. P. Refregier, Noise Theory and applications to physics, (Spinger, 2004).
  7. F. A. Sadjadi, ed., Selected papers on Automatic Target Recognition, (SPIE-CDROM, 1999).
  8. F. A. Sadjadi, “IR Target detection using probability density functions of wavelet transform subbands,” Appl. Opt. 43, 315–323 (2004).
    [CrossRef] [PubMed]
  9. B. Javidi, ed., Image Recognition and Classification: Algorithms, Systems, and Applications, (Marcel Dekker, New York, 2002).
    [CrossRef]
  10. J. Rosen, “Three-dimensional joint transform correlator,” Appl. Opt. 37, 7538–7544 (1998).
    [CrossRef]
  11. J. J. Esteve-Taboada, D. Mas, and J. Garcia, “Three-dimensional object recognition by Fourier transform profilometry,” Appl. Opt. 38, 4760–4765 (1999).
    [CrossRef]
  12. S. Yeom and B. Javidi, “Three-dimensional object feature extraction and classification with computational holographic imaging,” Appl. Opt. 43, 442–451 (2004).
    [CrossRef] [PubMed]
  13. Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488–5496 (2002).
    [CrossRef] [PubMed]
  14. S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11, 3528–3541 (2003), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528.
    [CrossRef] [PubMed]
  15. J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, “Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,” Appl. Opt. 43, 4882–4895 (2004).
    [CrossRef] [PubMed]
  16. C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).
  17. S.-H. Lin, S.-Y. Kung, and L.-J. Lin, “Face recognition/detection by probabilistic decision-based neural network,” IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114–132 (1997).
    [PubMed]
  18. P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. on PAMI. 19, 711–720 (1997).
    [CrossRef]
  19. M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Trans. on PAMI. 21, 1357–1362 (1999).
    [CrossRef]
  20. A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: a review,” IEEE Trans. on PAMI. 22, 4–37 (2000).
    [CrossRef]
  21. M. Egmont-petersen, D. de Ridder, and H. Handels, “Image processing with neural networks — a review,” J. patt. recog. soc.,  35, 2279–2301 (2002).
    [CrossRef]
  22. T. Okoshi, “Three-dimensional displays,” in Proceedings of the IEEE 68, 548–564 (1980)
    [CrossRef]
  23. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
    [CrossRef] [PubMed]
  24. J.-S. Jang and B. Javidi, “Time-multiplexed integral imaging for 3D sensing and display,” Optics and Photonics News 15, 36–43 (2004), http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36.

2004 (5)

2003 (1)

2002 (3)

Y. Frauel and B. Javidi, “Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,” Appl. Opt. 41, 5488–5496 (2002).
[CrossRef] [PubMed]

M. Egmont-petersen, D. de Ridder, and H. Handels, “Image processing with neural networks — a review,” J. patt. recog. soc.,  35, 2279–2301 (2002).
[CrossRef]

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).

2000 (1)

A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: a review,” IEEE Trans. on PAMI. 22, 4–37 (2000).
[CrossRef]

1999 (2)

M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Trans. on PAMI. 21, 1357–1362 (1999).
[CrossRef]

J. J. Esteve-Taboada, D. Mas, and J. Garcia, “Three-dimensional object recognition by Fourier transform profilometry,” Appl. Opt. 38, 4760–4765 (1999).
[CrossRef]

1998 (1)

1997 (4)

C. F. Olson and D. P. Huttenlocher, “Automatic target recognition by matching oriented edge pixels,” IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103–113 (1997).

S.-H. Lin, S.-Y. Kung, and L.-J. Lin, “Face recognition/detection by probabilistic decision-based neural network,” IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114–132 (1997).
[PubMed]

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. on PAMI. 19, 711–720 (1997).
[CrossRef]

F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
[CrossRef] [PubMed]

1980 (1)

T. Okoshi, “Three-dimensional displays,” in Proceedings of the IEEE 68, 548–564 (1980)
[CrossRef]

Aggoun, A.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).

Akamatsu, S.

M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Trans. on PAMI. 21, 1357–1362 (1999).
[CrossRef]

Arai, J.

Belhumer, P. N.

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. on PAMI. 19, 711–720 (1997).
[CrossRef]

Bishop, C. M.

C. M. Bishop, Neural networks for pattern recognition, (Oxford University Press, New Yeork, 1995).

Budynek, J.

M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Trans. on PAMI. 21, 1357–1362 (1999).
[CrossRef]

Choi, H.

de Ridder, D.

M. Egmont-petersen, D. de Ridder, and H. Handels, “Image processing with neural networks — a review,” J. patt. recog. soc.,  35, 2279–2301 (2002).
[CrossRef]

Duda, R. O.

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification2nd, (Wiley Interscience, New York, 2001).

Duin, R. P. W.

A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: a review,” IEEE Trans. on PAMI. 22, 4–37 (2000).
[CrossRef]

Egmont-petersen, M.

M. Egmont-petersen, D. de Ridder, and H. Handels, “Image processing with neural networks — a review,” J. patt. recog. soc.,  35, 2279–2301 (2002).
[CrossRef]

Esteve-Taboada, J. J.

Frauel, Y.

Garcia, J.

Goudail, F.

Handels, H.

M. Egmont-petersen, D. de Ridder, and H. Handels, “Image processing with neural networks — a review,” J. patt. recog. soc.,  35, 2279–2301 (2002).
[CrossRef]

Hart, P. E.

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification2nd, (Wiley Interscience, New York, 2001).

Hespanha, J. P.

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. on PAMI. 19, 711–720 (1997).
[CrossRef]

Hoshino, H.

Huttenlocher, D. P.

C. F. Olson and D. P. Huttenlocher, “Automatic target recognition by matching oriented edge pixels,” IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103–113 (1997).

Jain, A. K.

A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: a review,” IEEE Trans. on PAMI. 22, 4–37 (2000).
[CrossRef]

A. K. Jain, Fundamentals of digital image processing, (Prentice-Hall Inc., 1989).

Jang, J.-S.

J.-S. Jang and B. Javidi, “Time-multiplexed integral imaging for 3D sensing and display,” Optics and Photonics News 15, 36–43 (2004), http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36.

Javidi, B.

Jung, S.

Kim, Y.

Kishk, S.

Kriegman, D. J.

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. on PAMI. 19, 711–720 (1997).
[CrossRef]

Kung, S. Y.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).

Kung, S.-Y.

S.-H. Lin, S.-Y. Kung, and L.-J. Lin, “Face recognition/detection by probabilistic decision-based neural network,” IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114–132 (1997).
[PubMed]

Lee, B.

Lin, L.-J.

S.-H. Lin, S.-Y. Kung, and L.-J. Lin, “Face recognition/detection by probabilistic decision-based neural network,” IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114–132 (1997).
[PubMed]

Lin, S.-H.

S.-H. Lin, S.-Y. Kung, and L.-J. Lin, “Face recognition/detection by probabilistic decision-based neural network,” IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114–132 (1997).
[PubMed]

Lyons, M. J.

M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Trans. on PAMI. 21, 1357–1362 (1999).
[CrossRef]

Mahalanobis, A.

Mao, J.

A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: a review,” IEEE Trans. on PAMI. 22, 4–37 (2000).
[CrossRef]

Mas, D.

McCormick, M.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).

Okano, F.

Okoshi, T.

T. Okoshi, “Three-dimensional displays,” in Proceedings of the IEEE 68, 548–564 (1980)
[CrossRef]

Olson, C. F.

C. F. Olson and D. P. Huttenlocher, “Automatic target recognition by matching oriented edge pixels,” IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103–113 (1997).

Park, J.-H.

Refregier, P.

P. Refregier, Noise Theory and applications to physics, (Spinger, 2004).

Rosen, J.

Sadjadi, F. A.

Stork, D. G.

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification2nd, (Wiley Interscience, New York, 2001).

Wu, C.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).

Yeom, S.

Yuyama, I.

Appl. Opt. (8)

IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition (1)

C. F. Olson and D. P. Huttenlocher, “Automatic target recognition by matching oriented edge pixels,” IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103–113 (1997).

IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition (1)

S.-H. Lin, S.-Y. Kung, and L.-J. Lin, “Face recognition/detection by probabilistic decision-based neural network,” IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114–132 (1997).
[PubMed]

IEEE Trans. on PAMI. (3)

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” IEEE Trans. on PAMI. 19, 711–720 (1997).
[CrossRef]

M. J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Trans. on PAMI. 21, 1357–1362 (1999).
[CrossRef]

A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical pattern recognition: a review,” IEEE Trans. on PAMI. 22, 4–37 (2000).
[CrossRef]

in Proceedings of the IEEE (1)

T. Okoshi, “Three-dimensional displays,” in Proceedings of the IEEE 68, 548–564 (1980)
[CrossRef]

in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE (1)

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE,  4660, 135–145 (2002).

J. patt. recog. soc. (1)

M. Egmont-petersen, D. de Ridder, and H. Handels, “Image processing with neural networks — a review,” J. patt. recog. soc.,  35, 2279–2301 (2002).
[CrossRef]

Opt. Express (1)

Optics and Photonics News (1)

J.-S. Jang and B. Javidi, “Time-multiplexed integral imaging for 3D sensing and display,” Optics and Photonics News 15, 36–43 (2004), http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36.

Other (6)

B. Javidi, ed., Image Recognition and Classification: Algorithms, Systems, and Applications, (Marcel Dekker, New York, 2002).
[CrossRef]

P. Refregier, Noise Theory and applications to physics, (Spinger, 2004).

F. A. Sadjadi, ed., Selected papers on Automatic Target Recognition, (SPIE-CDROM, 1999).

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification2nd, (Wiley Interscience, New York, 2001).

A. K. Jain, Fundamentals of digital image processing, (Prentice-Hall Inc., 1989).

C. M. Bishop, Neural networks for pattern recognition, (Oxford University Press, New Yeork, 1995).

Supplementary Material (6)

» Media 1: AVI (1817 KB)     
» Media 2: AVI (1817 KB)     
» Media 3: AVI (1817 KB)     
» Media 4: AVI (13975 KB)     
» Media 5: AVI (13975 KB)     
» Media 6: AVI (13975 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1.
Fig. 1.

Optical system for II. Each lenslet produces an elemental image.

Fig. 2.
Fig. 2.

Frameworks of object recognition system using II.

Fig. 3.
Fig. 3.

Three toy cars used in the experiments; object type 1, 2 and 3 are shown from right to left.

Fig. 4.
Fig. 4.

(Each 1.82 MB) Movies of II frames for out-of-plane rotated objects. (a) car 1 (13.98 MB version), (b) car 2 (13.98 MB version), (c) car 3 (13.98 MB version). [Media 1, Media 2, Media 3]

Fig. 5.
Fig. 5.

Examples of basis-images of PCA subspace (column vectors in WP ) for rotation angle sets of experiments. c=6, nj =15, l=30, and k=2. (a) 1st, (b) 2nd, (c) 3rd, (d) 30th basis-images.

Fig. 6.
Fig. 6.

Examples of basis-images of PCA-FLD subspace (column vectors in WPWF ) for rotation angle sets of experiments. c=6, nj =15, l=30, and k=2. (a) 1st, (b) 2nd basis images.

Fig. 7.
Fig. 7.

Average probability of correct decision (Pd ) for each type of objects. (a) nearest neighbor decision rule, (b) statistical distance decision rule.

Fig. 8.
Fig. 8.

Average RMSE for each type of objects.

Fig. 9.
Fig. 9.

Examples of basis-images of PCA subspace (column vectors in WP ) for the second experiment. c=18, nj =15, l=70, and k=4. (a) 1st, (b) 2nd, (c) 3rd, (d) 70th basis images.

Fig. 10.
Fig. 10.

Examples of basis-images of PCA-FLD subspace (column vectors in WPWF ) for the second experiment. c=18, nj =15, l=70, and k=4. (a) 1st, (b) 2nd, (c) 3rd, (d) 4th basis-images.

Fig. 11.
Fig. 11.

Average probability of correct decision (Pd ) for all 18 classes. (a) nearest neighbor decision rule, (b) statistical distance decision rule.

Fig. 12.
Fig. 12.

Average RMSE for all 18 classes.

Tables (3)

Tables Icon

Table 1. 6 out-of-plane rotation angles

Tables Icon

Table 2. Performance evaluation for each type of objects

Tables Icon

Table 3. Performance evaluation for all 18 classes

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

[ p ̂ , q ̂ ] = max p , q c ( p , q ) ,
c ( p , q ) = x = 1 M x y = 1 M y e r ( x , y ) e i ( x + p , y + q ) x = 1 M x y = 1 M y e r ( x , y ) 2 x = 1 M x y = 1 M y e i ( x , y ) 2 ,
e 1 ( x , y ) = e i ( x + p ̂ , y + q ̂ ) , x = 1 , , M x , y = 1 , , M y .
MSE ( x ̂ ) = E x x ̂ 2 = i = l + 1 d λ i
S B = j = 1 c n j ( m j m ) ( m j m ) t ,
S W = j = 1 c n = 1 n j ( y j ( n ) m j ) ( y j ( n ) m j ) t ,
S ˜ B = W F t S B W F = j = 1 c n j ( m ˜ j m ˜ ) ( m ˜ j m ˜ ) t ,
S ˜ W = W F t S W W F = j = 1 c n = 1 n j ( z j ( n ) m ˜ j ) ( z j ( n ) m ˜ j ) t ,
J ( W F , W P ) = W F t S B W F W F t S W W F = W F t W P t S B 0 W P W F W F t W P t S W 0 W P W F , .
z C j ̂ if j ̂ = arg min j z m z j , j = 1 , , c ,
z C j ̂ if j ̂ = arg min j g j ( z ) , j = 1 , , c ,
P d = Number of correct decisions Number of test data .
rmse = 1 n test n = 1 n test X test ( n ) X ̂ test ( n ) 2 X test ( n ) 2 ,
X ̂ test ( n ) = m x + W ̂ p W ̂ p t ( X test ( n ) m x ) ,
m x = 1 n t n = 1 n t x ( n ) ,
̂ x x = 1 n t 1 n = 1 n t ( x ( n ) m x ) ( x ( n ) m x ) t ,

Metrics