Abstract

We present a novel technique to extract depth information from 3D scenes recorded using an Integral Imaging system. The technique exploits the periodic structure of the recorded integral image to implement a Fourier-domain filtering algorithm. A proper projection of the filtered integral image permits reconstruction of different planes that constitute the 3D scene. The main feature of our method is that the Fourier-domain filtering allows the reduction of out-of-focus information, providing the InI system with real optical sectioning capacity.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. G. Lippmann, "Epreuves reversibles donnant la sensation du relief," J. Phys. (Paris) 7, 821-825 (1908).
  2. H. E. Ives, "Optical properties of a Lippman lenticulated sheet," J. Opt. Soc. Am. 21, 171-176 (1931).
    [CrossRef]
  3. C. B. Burckhardt, "Optimum parameters and resolution limitation of Integral Photography," J. Opt. Soc. Am. 58, 71-76 (1968).
    [CrossRef]
  4. T. Okoshi, "Three-dimensional displays," Proc. IEEE 68, 548-564 (1980).
    [CrossRef]
  5. F. Okano, H. Hoshino, J. Arai, and I. Yayuma, "Real time pickup method for a three-dimensional image based on integral photography," Appl. Opt. 36, 1598-1603 (1997).
    [CrossRef] [PubMed]
  6. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, "Gradient-index lens-array method based on real-time integral photography for three-dimensional images," Appl. Opt. 37,2034-2045 (1998).
    [CrossRef]
  7. H. Arimoto and B. Javidi, "Integral 3D imaging with digital reconstruction," Opt. Lett. 26, 157-159 (2001).
    [CrossRef]
  8. J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
    [CrossRef]
  9. S. Jung, J.-H. Park, H. Choi and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 12, 1346-1356 (2003).
    [CrossRef]
  10. J. Arai, M. Okui, T. Yamashita, and F. Okano, "Integral three-dimensional television using a 2000-scanning-line video system," Appl. Opt. 45, 1704-1712 (2006)
    [CrossRef] [PubMed]
  11. J.-S. Jang and B. Javidi, "Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes," Opt. Lett. 28, 1924-1926 (2003).
    [CrossRef] [PubMed]
  12. R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004).
    [CrossRef] [PubMed]
  13. R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools," J. Disp. Technol. 1, 321-327 (2005).
    [CrossRef]
  14. J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, "Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging," Opt. Lett. 29, 2734-2736 (2004).
    [CrossRef] [PubMed]
  15. H. Choi, S.-W. Min, S. Yung, J.-H. Park, and B. Lee, "Multiple viewing zone integral image using dynamic barrier array fro three-dimensional displays," Opt. Express 11, 927-932 (2003).
    [CrossRef] [PubMed]
  16. R.  Martínez-Cuenca, H.  Navarro, G.  Saavedra, B.  Javidi, and M. Martínez-Corral, "Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system," Opt. Express 15, 16255-16260 (2007).
    [CrossRef] [PubMed]
  17. J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
    [CrossRef]
  18. M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Multifacet structure of observed reconstructed integral images," J. Opt. Soc. Am. A 22, 597-603 (2005).
    [CrossRef]
  19. J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, "Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging," Opt. Lett. 29, 2734-2736 (2004).
    [CrossRef] [PubMed]
  20. W. Matusik and H. Pfister, "3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes," ACM Trans. Graph. 23, 814-824 (2004).
    [CrossRef]
  21. H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
    [CrossRef]
  22. Y. Frauel and B. Javidi, "Digital Three-Dimensional Image Correlation by use of Computer-Reconstructed Integral Imaging," Appl. Opt. 41, 5488-5496 (2002)
    [CrossRef] [PubMed]
  23. S. Yeom and B. Javidi, "Three-dimensional distortion-tolerant object recognition using integral imaging," Opt. Express 12, 5795-5809 (2004)
    [CrossRef] [PubMed]
  24. S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004).
    [CrossRef] [PubMed]
  25. C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
    [CrossRef]
  26. J.-S. Jang and B. Javidi, "Three-dimensional synthetic aperture integral imaging," Opt. Lett. 27, 1144-1146 (2002).
    [CrossRef]
  27. S.-H. Hong and B. Javidi, "Distortion-tolerant 3D recognition of occluded objects using computational integral imaging," Opt. Express 14, 12085- 12095 (2006).
    [CrossRef] [PubMed]
  28. B. Javidi, R. Ponce-Díaz, and S.-H. Hong, "Three-dimensional recognition of occluded objects by using computational integral imaging," Opt. Lett. 31, 1106-1108 (2006).
    [CrossRef] [PubMed]
  29. T. Wilson, ed., Confocal Microscopy (Academic, London 1990).
  30. R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral and B. Javidi, "Optically-corrected elemental images for undistorted integral image display," Opt. Express 14, 9657-9663 (2006).
    [CrossRef] [PubMed]

2007 (1)

2006 (4)

2005 (3)

M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Multifacet structure of observed reconstructed integral images," J. Opt. Soc. Am. A 22, 597-603 (2005).
[CrossRef]

R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools," J. Disp. Technol. 1, 321-327 (2005).
[CrossRef]

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
[CrossRef]

2004 (6)

2003 (3)

2002 (5)

2001 (1)

1998 (1)

1997 (1)

1980 (1)

T. Okoshi, "Three-dimensional displays," Proc. IEEE 68, 548-564 (1980).
[CrossRef]

1968 (1)

1931 (1)

1908 (1)

M. G. Lippmann, "Epreuves reversibles donnant la sensation du relief," J. Phys. (Paris) 7, 821-825 (1908).

Aggoun, A.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
[CrossRef]

Arai, J.

Arimoto, H.

Burckhardt, C. B.

Choi, H.

H. Choi, S.-W. Min, S. Yung, J.-H. Park, and B. Lee, "Multiple viewing zone integral image using dynamic barrier array fro three-dimensional displays," Opt. Express 11, 927-932 (2003).
[CrossRef] [PubMed]

S. Jung, J.-H. Park, H. Choi and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 12, 1346-1356 (2003).
[CrossRef]

Dohi, S. I. y T.

H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
[CrossRef]

Frauel, Y.

Hata, N.

H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
[CrossRef]

Hong, J.

Hong, S.-H.

Hoshino, H.

Ives, H. E.

Iwahara, M.

H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
[CrossRef]

Jang, J.-S

Jang, J.-S.

Javidi, B.

R.  Martínez-Cuenca, H.  Navarro, G.  Saavedra, B.  Javidi, and M. Martínez-Corral, "Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system," Opt. Express 15, 16255-16260 (2007).
[CrossRef] [PubMed]

S.-H. Hong and B. Javidi, "Distortion-tolerant 3D recognition of occluded objects using computational integral imaging," Opt. Express 14, 12085- 12095 (2006).
[CrossRef] [PubMed]

R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral and B. Javidi, "Optically-corrected elemental images for undistorted integral image display," Opt. Express 14, 9657-9663 (2006).
[CrossRef] [PubMed]

B. Javidi, R. Ponce-Díaz, and S.-H. Hong, "Three-dimensional recognition of occluded objects by using computational integral imaging," Opt. Lett. 31, 1106-1108 (2006).
[CrossRef] [PubMed]

M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Multifacet structure of observed reconstructed integral images," J. Opt. Soc. Am. A 22, 597-603 (2005).
[CrossRef]

R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools," J. Disp. Technol. 1, 321-327 (2005).
[CrossRef]

R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004).
[CrossRef] [PubMed]

S. Yeom and B. Javidi, "Three-dimensional distortion-tolerant object recognition using integral imaging," Opt. Express 12, 5795-5809 (2004)
[CrossRef] [PubMed]

S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004).
[CrossRef] [PubMed]

J.-S. Jang and B. Javidi, "Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes," Opt. Lett. 28, 1924-1926 (2003).
[CrossRef] [PubMed]

Y. Frauel and B. Javidi, "Digital Three-Dimensional Image Correlation by use of Computer-Reconstructed Integral Imaging," Appl. Opt. 41, 5488-5496 (2002)
[CrossRef] [PubMed]

J.-S. Jang and B. Javidi, "Three-dimensional synthetic aperture integral imaging," Opt. Lett. 27, 1144-1146 (2002).
[CrossRef]

J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
[CrossRef]

J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
[CrossRef]

H. Arimoto and B. Javidi, "Integral 3D imaging with digital reconstruction," Opt. Lett. 26, 157-159 (2001).
[CrossRef]

Jung, S.

S. Jung, J.-H. Park, H. Choi and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 12, 1346-1356 (2003).
[CrossRef]

Kim, H.-R.

Kim, J.

Kim, Y.

Kung, S. Y.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
[CrossRef]

Lee, B.

Lee, S.-D.

Liao, H.

H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
[CrossRef]

Lippmann, M. G.

M. G. Lippmann, "Epreuves reversibles donnant la sensation du relief," J. Phys. (Paris) 7, 821-825 (1908).

Martínez-Corral, M.

Martínez-Cuenca, R.

Matusik, W.

W. Matusik and H. Pfister, "3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes," ACM Trans. Graph. 23, 814-824 (2004).
[CrossRef]

McCormick, M.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
[CrossRef]

Min, S.-W.

Nakajima, S.

H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
[CrossRef]

Navarro, H.

Okano, F.

Okoshi, T.

T. Okoshi, "Three-dimensional displays," Proc. IEEE 68, 548-564 (1980).
[CrossRef]

Okui, M.

Park, J.-H.

Pfister, H.

W. Matusik and H. Pfister, "3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes," ACM Trans. Graph. 23, 814-824 (2004).
[CrossRef]

Ponce-Díaz, R.

Pons, A.

Saavedra, G.

Wu, C.

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
[CrossRef]

Yamashita, T.

Yayuma, I.

Yeom, S.

Yung, S.

Yuyama, I.

ACM Trans. Graph. (1)

W. Matusik and H. Pfister, "3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes," ACM Trans. Graph. 23, 814-824 (2004).
[CrossRef]

Appl. Opt. (4)

J. Disp. Technol. (1)

R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools," J. Disp. Technol. 1, 321-327 (2005).
[CrossRef]

J. Electron. Imaging (1)

C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005)
[CrossRef]

J. Opt. Soc. Am. (2)

J. Opt. Soc. Am. A (1)

J. Phys. (Paris) (1)

M. G. Lippmann, "Epreuves reversibles donnant la sensation du relief," J. Phys. (Paris) 7, 821-825 (1908).

Opt. Express (8)

S. Jung, J.-H. Park, H. Choi and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 12, 1346-1356 (2003).
[CrossRef]

R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral and B. Javidi, "Optically-corrected elemental images for undistorted integral image display," Opt. Express 14, 9657-9663 (2006).
[CrossRef] [PubMed]

S.-H. Hong and B. Javidi, "Distortion-tolerant 3D recognition of occluded objects using computational integral imaging," Opt. Express 14, 12085- 12095 (2006).
[CrossRef] [PubMed]

R.  Martínez-Cuenca, H.  Navarro, G.  Saavedra, B.  Javidi, and M. Martínez-Corral, "Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system," Opt. Express 15, 16255-16260 (2007).
[CrossRef] [PubMed]

H. Choi, S.-W. Min, S. Yung, J.-H. Park, and B. Lee, "Multiple viewing zone integral image using dynamic barrier array fro three-dimensional displays," Opt. Express 11, 927-932 (2003).
[CrossRef] [PubMed]

S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004).
[CrossRef] [PubMed]

R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004).
[CrossRef] [PubMed]

S. Yeom and B. Javidi, "Three-dimensional distortion-tolerant object recognition using integral imaging," Opt. Express 12, 5795-5809 (2004)
[CrossRef] [PubMed]

Opt. Lett. (8)

J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, "Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging," Opt. Lett. 29, 2734-2736 (2004).
[CrossRef] [PubMed]

J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, "Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging," Opt. Lett. 29, 2734-2736 (2004).
[CrossRef] [PubMed]

J.-S. Jang and B. Javidi, "Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes," Opt. Lett. 28, 1924-1926 (2003).
[CrossRef] [PubMed]

B. Javidi, R. Ponce-Díaz, and S.-H. Hong, "Three-dimensional recognition of occluded objects by using computational integral imaging," Opt. Lett. 31, 1106-1108 (2006).
[CrossRef] [PubMed]

H. Arimoto and B. Javidi, "Integral 3D imaging with digital reconstruction," Opt. Lett. 26, 157-159 (2001).
[CrossRef]

J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
[CrossRef]

J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
[CrossRef]

J.-S. Jang and B. Javidi, "Three-dimensional synthetic aperture integral imaging," Opt. Lett. 27, 1144-1146 (2002).
[CrossRef]

Proc. IEEE (1)

T. Okoshi, "Three-dimensional displays," Proc. IEEE 68, 548-564 (1980).
[CrossRef]

Proc. SPIE (1)

H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002).
[CrossRef]

Other (1)

T. Wilson, ed., Confocal Microscopy (Academic, London 1990).

Supplementary Material (1)

» Media 1: AVI (4846 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1.

Each microlens is labeled by its position in the array. The origin for the indexes is the center of the central microlens. The images of a point source through the microlenses are depicted.

Fig. 2.
Fig. 2.

The Fourier transform of an integral image. Left side illustrates the signals that correspond to two sources at different depths, namely S and S’. On the right, we show the performance of the filtering. Only signals with pitch close to TR pass though the filter.

Fig. 3.
Fig. 3.

The volumetric reconstruction calculates the reconstructed field by projecting the integral image through the pinhole array. Optical barriers are also simulated to avoid overlapping.

Fig. 4.
Fig. 4.

The 3D scene was composed by two toy cars. The cars were about 20–10 mm in size.

Fig. 5.
Fig. 5.

a) Set of 1×3 elemental images of the recorded integral image. b) Central part of its spectrum. c) The filtering with comb functions of different period permits to discriminate the information at a given depth in the 3D scene. We show two different filtering pitchs, in red red for zR =70 mm and in blue for zR =120 mm..

Fig. 6.
Fig. 6.

a) Set of 1×3 elemental images of the integral image filtered at zR=70 mm. b) The same part of the integral image, but filtered at zR=92 mm.

Fig. 7.
Fig. 7.

Five frames excerpts, corresponding to depth planes 50, 70, 80, 92 and 120 mm, from a video that shows the reconstruction over different depth planes between 50 and 120 mm (Media 1).

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

x m ( z S ) = M S x S + m T S ,
T S = ( 1 + g z S ) p .
I S ( x ) = rect ( x x S Δ x ) · Σ m = δ ( x x m ) 1 M S O ( x M S ) ,
I ˜ S ( u ) = Δ x sin c [ Δ x u ] exp ( i 2 π g z S u . x S ) O ˜ ( M S u ) 1 T S Σ m = δ ( u m T S ) ,
I ˜ ( u ) = 0 I ˜ S ( u ) d z s
I ˜ R ( u ) = I ˜ ( u ) F R ( u ) ,
F R ( u ) = Σ m = δ ( u + m T R )
I ( x , z R ) = Σ k = 0 K 1 O k ( x + ( 1 M S ) T S k ) R 2 ( x ) ,
R 2 ( x ) = ( z S + g ) 2 + ( M S 1 x + T S k ) 2 ( 1 M S ) 2 ,

Metrics