Abstract

This paper addresses 3D distortion-tolerant object recognition using Integral Imaging (II). II is one of the techniques considered for 3D image recording and display. Multiple elemental images are captured to record different but continuous viewing zones of 3D scenes. We develop a distortion-tolerant 3D object recognition by using the II capture system. We adopt Principal Component Analysis (PCA) and Fisher Linear Discriminant (FLD) classifier followed by the nearest neighbor decision rule and the statistical distance metric. Performance is analyzed in terms of probability of correct decision, Root Mean Square Error (RMSE) between raw input vectors and reconstructed vectors from the PCA subspace and the PCA-FLD cost function. Decision strategies are compared by the varying number of training data.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |

  1. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification 2nd, (Wiley Interscience, New York, 2001).
  2. A. K. Jain, Fundamentals of digital image processing, (Prentice-Hall Inc., 1989).
  3. C. M. Bishop, Neural networks for pattern recognition, (Oxford University Press, New Yeork, 1995).
  4. A. Mahalanobis and F. Goudail, �??Methods for automatic target recognition by use of electro-optic sensors: introduction to the feature Issue,�?? Appl. Opt. 43, 207-209 (2004).
  5. P. Refregier, Noise Theory and applications to physics, (Spinger, 2004).
  6. F. A. Sadjadi, ed., Selected papers on Automatic Target Recognition, (SPIE-CDROM, 1999).
  7. F. A. Sadjadi, �??IR Target detection using probability density functions of wavelet transform subbands,�?? Appl. Opt. 43, 315-323 (2004).
    [CrossRef] [PubMed]
  8. B. Javidi, ed., Image Recognition and Classification: Algorithms, Systems, and Applications, (Marcel Dekker, New York, 2002).
    [CrossRef]
  9. J. Rosen, �??Three-dimensional joint transform correlator,�?? Appl. Opt. 37, 7538-7544 (1998).
    [CrossRef]
  10. J. J. Esteve-Taboada, D. Mas, and J. Garcia, �??Three-dimensional object recognition by Fourier transform profilometry,�?? Appl. Opt. 38, 4760-4765 (1999).
    [CrossRef]
  11. S. Yeom and B. Javidi, �??Three-dimensional object feature extraction and classification with computational holographic imaging,�?? Appl. Opt. 43, 442-451 (2004).
    [CrossRef] [PubMed]
  12. Y. Frauel and B. Javidi, �??Digital three-dimensional image correlation by use of computer-reconstructed integral imaging,�?? Appl. Opt. 41, 5488-5496 (2002).
    [CrossRef] [PubMed]
  13. S. Kishk and B. Javidi, �??Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,�?? Opt. Express 11, 3528-3541 (2003), <a href="http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-11-26-3528</a>.
    [CrossRef] [PubMed]
  14. J.-H. Park, S. Jung, H. Choi, Y. Kim, and B. Lee, �??Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification,�?? Appl. Opt. 43, 4882-4895 (2004).
    [CrossRef] [PubMed]
  15. C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, �??Depth extraction from unidirectional image using a modified multi-baseline technique,�?? in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE, 4660, 135-145 (2002).
  16. P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, �??Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,�?? IEEE Trans. on PAMI. 19, 711-720 (1997).
    [CrossRef]
  17. M. J. Lyons, J. Budynek, and S. Akamatsu, �??Automatic classification of single facial images,�?? IEEE Trans. on PAMI. 21, 1357-1362 (1999).
    [CrossRef]
  18. A. K. Jain, R. P. W. Duin, and J. Mao, �??Statistical pattern recognition: a review,�?? IEEE Trans. on PAMI. 22, 4-37 (2000).
    [CrossRef]
  19. M. Egmont-petersen, D. de Ridder, and H. Handels, �??Image processing with neural networks �?? a review,�?? J. Patt. Recog. Soc. 35, 2279-2301 (2002).
    [CrossRef]
  20. T. Okoshi, �??Three-dimensional displays,�?? in Proceedings of the IEEE 68, 548-564 (1980)
    [CrossRef]
  21. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, �??Real-time pickup method for a three-dimensional image based on integral photography,�?? Appl. Opt. 36, 1598-1603 (1997).
    [CrossRef] [PubMed]
  22. J.-S. Jang and B. Javidi, �??Time-multiplexed integral imaging for 3D sensing and display,�?? Optics and Photonics News 15, 36-43 (2004), <a href="http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36">http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36</a>.
  23. C. F. Olson and D. P. Huttenlocher, �??Automatic target recognition by matching oriented edge pixels,�?? IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103-113 (1997).
  24. S.-H. Lin, S.-Y. Kung, and L.-J. Lin, �??Face recognition/detection by probabilistic decision-based neural network,�?? IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114-132 (1997).

Appl. Opt.

IEEE Trans. on Image processing

C. F. Olson and D. P. Huttenlocher, �??Automatic target recognition by matching oriented edge pixels,�?? IEEE Trans. on Image processing, special issue on Automatic Target Detection and Recognition 6, 103-113 (1997).

IEEE Trans. on Neural Networks

S.-H. Lin, S.-Y. Kung, and L.-J. Lin, �??Face recognition/detection by probabilistic decision-based neural network,�?? IEEE Trans. on Neural Networks, special issue on Artificial Neural Network and Pattern Recognition 8, 114-132 (1997).

IEEE Trans. on PAMI.

P. N. Belhumer, J. P. Hespanha, and D. J. Kriegman, �??Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,�?? IEEE Trans. on PAMI. 19, 711-720 (1997).
[CrossRef]

M. J. Lyons, J. Budynek, and S. Akamatsu, �??Automatic classification of single facial images,�?? IEEE Trans. on PAMI. 21, 1357-1362 (1999).
[CrossRef]

A. K. Jain, R. P. W. Duin, and J. Mao, �??Statistical pattern recognition: a review,�?? IEEE Trans. on PAMI. 22, 4-37 (2000).
[CrossRef]

J. Patt. Recog. Soc.

M. Egmont-petersen, D. de Ridder, and H. Handels, �??Image processing with neural networks �?? a review,�?? J. Patt. Recog. Soc. 35, 2279-2301 (2002).
[CrossRef]

Opt. Express

Opt. Photon. News

J.-S. Jang and B. Javidi, �??Time-multiplexed integral imaging for 3D sensing and display,�?? Optics and Photonics News 15, 36-43 (2004), <a href="http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36">http://www.osa-opn.org/abstract.cfm?URI=OPN-15-4-36</a>.

Proc. of the IEEE

T. Okoshi, �??Three-dimensional displays,�?? in Proceedings of the IEEE 68, 548-564 (1980)
[CrossRef]

Proc. SPIE

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, �??Depth extraction from unidirectional image using a modified multi-baseline technique,�?? in Stereoscopic Displays and Virtual Reality Systems IX, Proc. SPIE, 4660, 135-145 (2002).

Other

B. Javidi, ed., Image Recognition and Classification: Algorithms, Systems, and Applications, (Marcel Dekker, New York, 2002).
[CrossRef]

P. Refregier, Noise Theory and applications to physics, (Spinger, 2004).

F. A. Sadjadi, ed., Selected papers on Automatic Target Recognition, (SPIE-CDROM, 1999).

R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification 2nd, (Wiley Interscience, New York, 2001).

A. K. Jain, Fundamentals of digital image processing, (Prentice-Hall Inc., 1989).

C. M. Bishop, Neural networks for pattern recognition, (Oxford University Press, New Yeork, 1995).

Supplementary Material (6)

» Media 1: AVI (1817 KB)     
» Media 2: AVI (1817 KB)     
» Media 3: AVI (1817 KB)     
» Media 4: AVI (13975 KB)     
» Media 5: AVI (13975 KB)     
» Media 6: AVI (13975 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1.
Fig. 1.

Optical system for II. Each lenslet produces an elemental image.

Fig. 2.
Fig. 2.

Frameworks of object recognition system using II.

Fig. 3.
Fig. 3.

Three toy cars used in the experiments; object type 1, 2 and 3 are shown from right to left.

Fig. 4.
Fig. 4.

(Each 1.82 MB) Movies of II frames for out-of-plane rotated objects. (a) car 1 (13.98 MB version), (b) car 2 (13.98 MB version), (c) car 3 (13.98 MB version). [Media 1, Media 2, Media 3]

Fig. 5.
Fig. 5.

Examples of basis-images of PCA subspace (column vectors in WP ) for rotation angle sets of experiments. c=6, nj =15, l=30, and k=2. (a) 1st, (b) 2nd, (c) 3rd, (d) 30th basis-images.

Fig. 6.
Fig. 6.

Examples of basis-images of PCA-FLD subspace (column vectors in WPWF ) for rotation angle sets of experiments. c=6, nj =15, l=30, and k=2. (a) 1st, (b) 2nd basis images.

Fig. 7.
Fig. 7.

Average probability of correct decision (Pd ) for each type of objects. (a) nearest neighbor decision rule, (b) statistical distance decision rule.

Fig. 8.
Fig. 8.

Average RMSE for each type of objects.

Fig. 9.
Fig. 9.

Examples of basis-images of PCA subspace (column vectors in WP ) for the second experiment. c=18, nj =15, l=70, and k=4. (a) 1st, (b) 2nd, (c) 3rd, (d) 70th basis images.

Fig. 10.
Fig. 10.

Examples of basis-images of PCA-FLD subspace (column vectors in WPWF ) for the second experiment. c=18, nj =15, l=70, and k=4. (a) 1st, (b) 2nd, (c) 3rd, (d) 4th basis-images.

Fig. 11.
Fig. 11.

Average probability of correct decision (Pd ) for all 18 classes. (a) nearest neighbor decision rule, (b) statistical distance decision rule.

Fig. 12.
Fig. 12.

Average RMSE for all 18 classes.

Tables (3)

Tables Icon

Table 1. 6 out-of-plane rotation angles

Tables Icon

Table 2. Performance evaluation for each type of objects

Tables Icon

Table 3. Performance evaluation for all 18 classes

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

[ p ̂ , q ̂ ] = max p , q c ( p , q ) ,
c ( p , q ) = x = 1 M x y = 1 M y e r ( x , y ) e i ( x + p , y + q ) x = 1 M x y = 1 M y e r ( x , y ) 2 x = 1 M x y = 1 M y e i ( x , y ) 2 ,
e 1 ( x , y ) = e i ( x + p ̂ , y + q ̂ ) , x = 1 , , M x , y = 1 , , M y .
MSE ( x ̂ ) = E x x ̂ 2 = i = l + 1 d λ i
S B = j = 1 c n j ( m j m ) ( m j m ) t ,
S W = j = 1 c n = 1 n j ( y j ( n ) m j ) ( y j ( n ) m j ) t ,
S ˜ B = W F t S B W F = j = 1 c n j ( m ˜ j m ˜ ) ( m ˜ j m ˜ ) t ,
S ˜ W = W F t S W W F = j = 1 c n = 1 n j ( z j ( n ) m ˜ j ) ( z j ( n ) m ˜ j ) t ,
J ( W F , W P ) = W F t S B W F W F t S W W F = W F t W P t S B 0 W P W F W F t W P t S W 0 W P W F , .
z C j ̂ if j ̂ = arg min j z m z j , j = 1 , , c ,
z C j ̂ if j ̂ = arg min j g j ( z ) , j = 1 , , c ,
P d = Number of correct decisions Number of test data .
rmse = 1 n test n = 1 n test X test ( n ) X ̂ test ( n ) 2 X test ( n ) 2 ,
X ̂ test ( n ) = m x + W ̂ p W ̂ p t ( X test ( n ) m x ) ,
m x = 1 n t n = 1 n t x ( n ) ,
̂ x x = 1 n t 1 n = 1 n t ( x ( n ) m x ) ( x ( n ) m x ) t ,

Metrics