Abstract

We use integral images of a three-dimensional (3D) scene to estimate the longitudinal depth of multiple objects present in the scene. With this information, we digitally reconstruct the objects in three dimensions and compute 3D correlations of input objects. We investigate the use of nonlinear techniques for 3D correlations. We present experimental results for 3D reconstruction and correlation of 3D objects. We demonstrate that it is possible to perform 3D segmentation of 3D objects in a scene. We finally present experiments to demonstrate that the 3D correlation is more discriminant than the two-dimensional correlation.

© 2002 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. Javidi, ed., 3D Television, Video, and Digital Technologies (Springer-Verlag, Berlin, 2002).
  2. E. N. Leith, J. Upatniecks, “Reconstructed wavefronts and communication theory,” J. Opt. Soc. Am. 52, 1123–1130 (1962).
    [CrossRef]
  3. J. Caulfield, Handbook of Optical Holography (Academic, London, 1979).
  4. G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).
  5. F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
    [CrossRef]
  6. S. Nakajima, K. Masamune, I. Sakuma, T. Dohi, “Three-dimensional display system for medical imaging with computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VII, J. O. Merritt, S. A. Benton, A. J. Woods, M. T. Bolas, eds., Proc. SPIE3957, 60–67 (2000).
    [CrossRef]
  7. H. Arimoto, B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001).
    [CrossRef]
  8. T. Okoshi, Three-Dimensional Imaging Techniques (Academic, New York, 1971).
  9. S.-W. Min, S. Jung, J.-H. Park, B. Lee, “Three-dimensional display system based on computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt, S. A. Benton, eds., Proc. SPIE4297, 187–195 (2001).
    [CrossRef]
  10. J.-H. Park, S.-W. Min, S. Jung, B. Lee, “New stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
    [CrossRef]
  11. N. Mayer, R. Sand, “Stereoscopic television,” Rundfunktech. Mitt. 13, 123–134 (1969).
  12. T. Motoki, H. Isono, I. Yuyama, “Recent status of 3-dimensional television research,” Proc. IEEE 83, 1009–1021 (1995).
    [CrossRef]
  13. A. R. L. Travis, “Display of three-dimensional video images,” Proc. IEEE 85, 1817–1832 (1997).
    [CrossRef]
  14. A. B. VanderLugt, “Signal detection by complex spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
    [CrossRef]
  15. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1968).
  16. Ph. Réfrégier, “Filter design for optical pattern recognition: multicriteria optimization approach,” Opt. Lett. 15, 854–856 (1990).
    [CrossRef] [PubMed]
  17. B. Javidi, ed., Image Recognition Classification (Marcel Dekker, New York, 2002).
    [CrossRef]
  18. B. Javidi, “Nonlinear joint power spectrum based optical correlation,” Appl. Opt. 28, 2358–2367 (1989).
    [CrossRef] [PubMed]
  19. B. Javidi, E. Tajahuerce, “Three-dimensional object recognition by use of digital holography,” Opt. Lett. 25, 610–612 (2000).
    [CrossRef]
  20. Y. Frauel, E. Tajahuerce, M. A. Castro, B. Javidi, “Distortion-tolerant three-dimensional object recognition with digital holography,” Appl. Opt. 40, 3887–3893 (2001).
    [CrossRef]
  21. Y. Frauel, B. Javidi, “Neural network for three-dimensional object recognition based on digital holography,” Opt. Lett. 26, 1478–1480 (2001).
    [CrossRef]
  22. A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
    [CrossRef]
  23. J. Rosen, “Three-dimensional electro-optical correlation,” J. Opt. Soc. Am. A 15, 430–436 (1998).
    [CrossRef]
  24. J. Rosen, “Three-dimensional joint transform correlator,” Appl. Opt. 37, 7538–7544 (1998).
    [CrossRef]
  25. O. Matoba, E. Tajahuerce, B. Javidi, “Real-time three-dimensional object recognition with multiple perspectives imaging,” Appl. Opt. 40, 3318–3325 (2001).
    [CrossRef]

2001 (4)

2000 (1)

1999 (1)

F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
[CrossRef]

1998 (2)

1997 (2)

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

A. R. L. Travis, “Display of three-dimensional video images,” Proc. IEEE 85, 1817–1832 (1997).
[CrossRef]

1995 (1)

T. Motoki, H. Isono, I. Yuyama, “Recent status of 3-dimensional television research,” Proc. IEEE 83, 1009–1021 (1995).
[CrossRef]

1990 (1)

1989 (1)

1969 (1)

N. Mayer, R. Sand, “Stereoscopic television,” Rundfunktech. Mitt. 13, 123–134 (1969).

1964 (1)

A. B. VanderLugt, “Signal detection by complex spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
[CrossRef]

1962 (1)

1908 (1)

G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Arai, J.

F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
[CrossRef]

Arimoto, H.

Castro, M. A.

Caulfield, J.

J. Caulfield, Handbook of Optical Holography (Academic, London, 1979).

Denkewalter, R.

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Dohi, T.

S. Nakajima, K. Masamune, I. Sakuma, T. Dohi, “Three-dimensional display system for medical imaging with computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VII, J. O. Merritt, S. A. Benton, A. J. Woods, M. T. Bolas, eds., Proc. SPIE3957, 60–67 (2000).
[CrossRef]

Frauel, Y.

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1968).

Hoshino, H.

F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
[CrossRef]

Isono, H.

T. Motoki, H. Isono, I. Yuyama, “Recent status of 3-dimensional television research,” Proc. IEEE 83, 1009–1021 (1995).
[CrossRef]

Javidi, B.

Jung, S.

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “New stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

S.-W. Min, S. Jung, J.-H. Park, B. Lee, “Three-dimensional display system based on computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt, S. A. Benton, eds., Proc. SPIE4297, 187–195 (2001).
[CrossRef]

Lee, B.

S.-W. Min, S. Jung, J.-H. Park, B. Lee, “Three-dimensional display system based on computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt, S. A. Benton, eds., Proc. SPIE4297, 187–195 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “New stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

Leith, E. N.

Lippmann, G.

G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).

Masamune, K.

S. Nakajima, K. Masamune, I. Sakuma, T. Dohi, “Three-dimensional display system for medical imaging with computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VII, J. O. Merritt, S. A. Benton, A. J. Woods, M. T. Bolas, eds., Proc. SPIE3957, 60–67 (2000).
[CrossRef]

Matoba, O.

Mayer, N.

N. Mayer, R. Sand, “Stereoscopic television,” Rundfunktech. Mitt. 13, 123–134 (1969).

Min, S.-W.

S.-W. Min, S. Jung, J.-H. Park, B. Lee, “Three-dimensional display system based on computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt, S. A. Benton, eds., Proc. SPIE4297, 187–195 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “New stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

Motoki, T.

T. Motoki, H. Isono, I. Yuyama, “Recent status of 3-dimensional television research,” Proc. IEEE 83, 1009–1021 (1995).
[CrossRef]

Nakajima, S.

S. Nakajima, K. Masamune, I. Sakuma, T. Dohi, “Three-dimensional display system for medical imaging with computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VII, J. O. Merritt, S. A. Benton, A. J. Woods, M. T. Bolas, eds., Proc. SPIE3957, 60–67 (2000).
[CrossRef]

Okano, F.

F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
[CrossRef]

Okoshi, T.

T. Okoshi, Three-Dimensional Imaging Techniques (Academic, New York, 1971).

Park, J.-H.

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “New stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

S.-W. Min, S. Jung, J.-H. Park, B. Lee, “Three-dimensional display system based on computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt, S. A. Benton, eds., Proc. SPIE4297, 187–195 (2001).
[CrossRef]

Psaltis, D.

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Pu, A.

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

Réfrégier, Ph.

Rosen, J.

Sakuma, I.

S. Nakajima, K. Masamune, I. Sakuma, T. Dohi, “Three-dimensional display system for medical imaging with computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VII, J. O. Merritt, S. A. Benton, A. J. Woods, M. T. Bolas, eds., Proc. SPIE3957, 60–67 (2000).
[CrossRef]

Sand, R.

N. Mayer, R. Sand, “Stereoscopic television,” Rundfunktech. Mitt. 13, 123–134 (1969).

Tajahuerce, E.

Travis, A. R. L.

A. R. L. Travis, “Display of three-dimensional video images,” Proc. IEEE 85, 1817–1832 (1997).
[CrossRef]

Upatniecks, J.

VanderLugt, A. B.

A. B. VanderLugt, “Signal detection by complex spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
[CrossRef]

Yuyama, I.

F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
[CrossRef]

T. Motoki, H. Isono, I. Yuyama, “Recent status of 3-dimensional television research,” Proc. IEEE 83, 1009–1021 (1995).
[CrossRef]

Appl. Opt. (4)

C. R. Acad. Sci. (1)

G. Lippmann, “La photographic intégrale,” C. R. Acad. Sci. 146, 446–451 (1908).

IEEE Trans. Inf. Theory (1)

A. B. VanderLugt, “Signal detection by complex spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
[CrossRef]

J. Opt. Soc. Am. (1)

J. Opt. Soc. Am. A (1)

Opt. Eng. (2)

A. Pu, R. Denkewalter, D. Psaltis, “Real-time vehicle navigation using a holographic memory,” Opt. Eng. 36, 2737–2746 (1997).
[CrossRef]

F. Okano, J. Arai, H. Hoshino, I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999).
[CrossRef]

Opt. Lett. (4)

Proc. IEEE (2)

T. Motoki, H. Isono, I. Yuyama, “Recent status of 3-dimensional television research,” Proc. IEEE 83, 1009–1021 (1995).
[CrossRef]

A. R. L. Travis, “Display of three-dimensional video images,” Proc. IEEE 85, 1817–1832 (1997).
[CrossRef]

Rundfunktech. Mitt. (1)

N. Mayer, R. Sand, “Stereoscopic television,” Rundfunktech. Mitt. 13, 123–134 (1969).

Other (8)

B. Javidi, ed., 3D Television, Video, and Digital Technologies (Springer-Verlag, Berlin, 2002).

B. Javidi, ed., Image Recognition Classification (Marcel Dekker, New York, 2002).
[CrossRef]

J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, New York, 1968).

T. Okoshi, Three-Dimensional Imaging Techniques (Academic, New York, 1971).

S.-W. Min, S. Jung, J.-H. Park, B. Lee, “Three-dimensional display system based on computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VIII, A. J. Woods, M. T. Bolas, J. O. Merritt, S. A. Benton, eds., Proc. SPIE4297, 187–195 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “New stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

S. Nakajima, K. Masamune, I. Sakuma, T. Dohi, “Three-dimensional display system for medical imaging with computer-generated integral photography,” in Stereoscopic Displays and Virtual Reality Systems VII, J. O. Merritt, S. A. Benton, A. J. Woods, M. T. Bolas, eds., Proc. SPIE3957, 60–67 (2000).
[CrossRef]

J. Caulfield, Handbook of Optical Holography (Academic, London, 1979).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1

Experimental setup.

Fig. 2
Fig. 2

Formation of the images by each microlens.

Fig. 3
Fig. 3

Displacement of the projected object point versus depth of the object point for the central microlens.

Fig. 4
Fig. 4

Example of integral image of a 3D scene. The elementary images marked are the ones used for determining the scene depth.

Fig. 5
Fig. 5

Matching criterion [see Eq. (5)] versus depth. (a) Without filtering and (b) with convolution by a 15-mm-wide filter.

Fig. 6
Fig. 6

Views of the 3D objects used in the experiments. (a), (b), and (c) 3D reference objects; (d) and (e) 3D input scenes with the reference objects at various distances from the detector; (f) map of the estimated depths for the 3D scene shown in (d).

Fig. 7
Fig. 7

Perspective views of the 3D scenes used in the experiments. (a) Scene 1 and (b) scene 2.

Fig. 8
Fig. 8

Three-dimensional representation of the reconstructed scenes. (a) Scene 1 and (b) scene 2.

Fig. 9
Fig. 9

Normalized values of the correlation peaks versus the kth-law nonlinearity. The detection peaks are the ones corresponding to the presented reference object. The other peaks are undesirable (false alarms).

Fig. 10
Fig. 10

Discrimination of 3D correlation versus the kth-law nonlinearity.

Fig. 11
Fig. 11

Segmentation of the depth of the detected 3D objects.

Fig. 12
Fig. 12

Two correlation planes extracted from the 3D correlation between scene 1 and the 3D square reference object. (a) Correlation plane corresponding to Δz = -20 mm and (b) correlation plane corresponding to Δz = +10 mm.

Fig. 13
Fig. 13

Three-dimensional representation of the correlation volumes. (a) Scene 1, square reference and (b) scene 1, circle reference.

Tables (1)

Tables Icon

Table 1 Comparison between 2D and 3D Correlations for Discriminating between Two 3D Objects

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

pϕ-xz=Xp-pϕd,
Xp=pϕ1+dz-dz x.
Xq-Xp=q-pϕ1+dz.
Cp, q, p, q=m=-44n=-44 IXp+m, Yq+nIXp+m, Yq+nm=-44n=-44 I2Xp+m, Yq+nm=-44n=-44 I2Xp+m, Yq+n1/2.
Mz=p=-33q=-23 Cp, q-1, p, q+q=-33p=-23 Cp-1, q, p, q.
x=-z/dX0, y=-z/dY0.
SAB=|AB|2=|FT-1ÃB˜*|2,
SABk=|AkB|2=|FT-1|Ã|k expiϕÃ|B˜|k exp-iϕB˜|2,

Metrics