Abstract

A novel system for recognizing three-dimensional (3D) objects by use of multiple perspectives imaging is proposed. A 3D object under incoherent illumination is projected into an array of two-dimensional (2D) elemental images by use of a microlens array. Each elemental 2D image corresponds to a different perspective of the 3D object. Multiple perspectives imaging based on integral photography has been used for 3D display. In this way, the whole set of 2D elemental images records 3D information about the input object. After an optical incoherent-to-coherent conversion, an optical processor is employed to perform the correlation between the input and the reference 3D objects. Use of micro-optics allows us to process the 3D information in real time and with a compact optical system. To the best of our knowledge this 3D processor is the first to apply the principle of integral photography to 3D image recognition. We present experimental results obtained with both a digital and an optical implementation of the system. We also show that the system can recognize a slightly out-of-plane rotated 3D object.

© 2001 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. A. VanderLught, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory IT-10, 139–145 (1964).
  2. C. S. Weaver, J. W. Goodman, “Technique for optically convolving two functions,” Appl. Opt. 5, 1248–1249 (1966).
    [CrossRef] [PubMed]
  3. Ph. Réfrégier, “Filter design for optical pattern recognition: multicriteria optimization approach,” Opt. Lett. 15, 854–856 (1990).
    [CrossRef] [PubMed]
  4. A. D. McAulay, Optical Computer Architectures (Wiley, New York, 1991).
  5. A. Mahalanobis, “Review of correlation filters and their application for scene matching,” in Optoelectronic Devices and Systems for Processing, B. Javidi, K. M. Johnson, eds., Vol. CR65 of SPIE Critical Reviews of Optical Science and Technology (SPIE Press, Bellingham, Wash., 1996), pp. 240–260.
  6. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1996).
  7. A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
    [CrossRef]
  8. J. Rosen, “Three-dimensional optical Fourier transform and correlation,” Opt. Lett. 22, 964–966 (1997).
    [CrossRef] [PubMed]
  9. B. Javidi, E. Tajahuerce, “Three-dimensional object recognition by use of digital holography,” Opt. Lett. 25, 610–612 (2000).
    [CrossRef]
  10. J. J. Esteve-Taboada, D. Mas, J. García, “Three-dimensional object recognition by Fourier transform profilometry,” Appl. Opt. 38, 4760–4765 (1999).
    [CrossRef]
  11. H. J. Caulfield, ed., Handbook of Optical Holography (Academic, London, 1979).
  12. F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
    [CrossRef] [PubMed]
  13. F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
  14. B. Javidi, “Nonlinear joint transform correlators,” in Real-Time Optical Information Processing, B. Javidi, J. L. Horner eds. (Academic, San Diego, Calif., 1994), Chap. 4, pp. 115–183.

2000 (1)

1999 (2)

J. J. Esteve-Taboada, D. Mas, J. García, “Three-dimensional object recognition by Fourier transform profilometry,” Appl. Opt. 38, 4760–4765 (1999).
[CrossRef]

F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).

1997 (3)

1990 (1)

1966 (1)

1964 (1)

A. VanderLught, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory IT-10, 139–145 (1964).

Arai, J.

Arai, Jun

F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).

Denkewalter, R.

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Esteve-Taboada, J. J.

García, J.

Goodman, J. W.

Hoshino, H.

F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).

F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
[CrossRef] [PubMed]

Javidi, B.

B. Javidi, E. Tajahuerce, “Three-dimensional object recognition by use of digital holography,” Opt. Lett. 25, 610–612 (2000).
[CrossRef]

B. Javidi, “Nonlinear joint transform correlators,” in Real-Time Optical Information Processing, B. Javidi, J. L. Horner eds. (Academic, San Diego, Calif., 1994), Chap. 4, pp. 115–183.

Mahalanobis, A.

A. Mahalanobis, “Review of correlation filters and their application for scene matching,” in Optoelectronic Devices and Systems for Processing, B. Javidi, K. M. Johnson, eds., Vol. CR65 of SPIE Critical Reviews of Optical Science and Technology (SPIE Press, Bellingham, Wash., 1996), pp. 240–260.

Mas, D.

McAulay, A. D.

A. D. McAulay, Optical Computer Architectures (Wiley, New York, 1991).

Okano, F.

F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).

F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
[CrossRef] [PubMed]

Psaltis, D.

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Pu, A.

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Réfrégier, Ph.

Rosen, J.

Tajahuerce, E.

VanderLught, A.

A. VanderLught, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory IT-10, 139–145 (1964).

Weaver, C. S.

Yuyama, I.

F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).

F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
[CrossRef] [PubMed]

Appl. Opt. (3)

IEEE Trans. Inf. Theory (1)

A. VanderLught, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory IT-10, 139–145 (1964).

Opt. Eng. (2)

F. Okano, Jun Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Opt. Lett. (3)

Other (5)

B. Javidi, “Nonlinear joint transform correlators,” in Real-Time Optical Information Processing, B. Javidi, J. L. Horner eds. (Academic, San Diego, Calif., 1994), Chap. 4, pp. 115–183.

H. J. Caulfield, ed., Handbook of Optical Holography (Academic, London, 1979).

A. D. McAulay, Optical Computer Architectures (Wiley, New York, 1991).

A. Mahalanobis, “Review of correlation filters and their application for scene matching,” in Optoelectronic Devices and Systems for Processing, B. Javidi, K. M. Johnson, eds., Vol. CR65 of SPIE Critical Reviews of Optical Science and Technology (SPIE Press, Bellingham, Wash., 1996), pp. 240–260.

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1996).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

Schematic diagram of the proposed 3D recognition system.

Fig. 2
Fig. 2

Projection of multiple angular perspectives of the 3D object with a microlens array. Only two dimensions are considered in the figure.

Fig. 3
Fig. 3

Projection of the 3D input into elemental images: (a) without and (b) with rotation.

Fig. 4
Fig. 4

Experimental setup. (a) Recording of elemental images and (b) joint transform correlator (JTC). Lens L2 images the joint power spectrum (JPS) into the CCD.

Fig. 5
Fig. 5

Elemental images associated with the reference. (a) Central 19 × 19 images of the reference; (b), (c), and (d) enlarged images of central, left, and right regions of Fig. 5(a), respectively.

Fig. 6
Fig. 6

(a) Input object of a die with a face completely different from that of the reference object, (b) input object with a face similar to that of the reference object, and (c) rotated reference with an angle of 0.26°.

Fig. 7
Fig. 7

Numerical results. (a) Autocorrelation of the reference, (b) cross correlation between completely different faces of the dice, and (c) cross correlation between similar faces of the dice.

Fig. 8
Fig. 8

Cross correlations between the reference and the rotated reference with angles of (a) 0.26°, (b) 0.52°, and (c) 0.78°.

Fig. 9
Fig. 9

Experimental results. (a) Autocorrelation of the reference, (b) cross correlations between completely different faces of the dice, and (c) cross correlations between similar faces of the dice.

Fig. 10
Fig. 10

Cross correlations between the reference and the rotated reference with an angle of 0.26°.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

Δαn=αn-αn=nΔa-nΔa+Δzn Δa2 Δz,
Ix, y=k=-NxNxI=-NyNy Sklx, yδx-kΔx-kMΔx,y-IΔy-kMΔy+k=-NxNxI=-NyNy Rmnx, yδx-kΔx-kMΔx+α,y-IΔy-kMΔy+β,
J˜ν, η=|Ĩν, η|2,
Ĩν, η= Ix, yexp-j 2πλfxν+yηdxdy=k=-NxNxI=-NyNy S˜klν, ηexp-j 2πλfk1+MΔxν+I1+MΔyη+m=-NxNxn=-NyNy R˜mnν, ηexp-j 2πλfm1+MΔx-αν+n1+MΔy-βη.
J˜cν, η=k=-NxNxI=-NyNym=-NxNxn=-NyNy S˜klν, ηR˜mn*ν, η×exp-j 2πλfk-m1+MΔx+αν+I-n1+MΔy+βη,
Jcx, y=k=-NxNxI=-NyNym=-NxNxn=-NyNy Sklx, yRmnx-k-m1+MΔx-α,y-I-n1+MΔy-β,

Metrics