Abstract

Integral Imaging provides spatial and angular information of three-dimensional (3D) objects, which can be used both for 3D display and for computational post-processing purposes. In order to recover the depth information from an integral image, several algorithms have been developed. In this paper, we propose a new free depth synthesis and reconstruction method based on the two-dimensional (2D) deconvolution between the integral image and a simplified version of the periodic impulse response function (IRF) of the system. The period of the IRF depends directly on the axial position within the object space. Then, we can retrieve the depth information by performing the deconvolution with computed impulse responses with different periods. In addition, alternative reconstructions can be obtained by deconvolving with non-conventional synthetic impulse responses. Our experiments show the feasibility of the proposed method as well as its potential applications.

© 2015 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Synthetic aperture integral imaging using edge depth maps of unstructured monocular video

Jian Wei, Shigang Wang, Yan Zhao, and Meilan Piao
Opt. Express 26(26) 34894-34908 (2018)

Digital slicing of 3D scenes by Fourier filtering of integral images

G. Saavedra, R. Martínez-Cuenca, M. Martínez-Corral, H. Navarro, M. Daneshpanah, and B. Javidi
Opt. Express 16(22) 17154-17160 (2008)

References

  • View by:
  • |
  • |
  • |

  1. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).
  2. H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. 21(3), 171–176 (1931).
    [Crossref]
  3. C. B. Burckhardt, “Optimum parameters and resolution limitation of Integral Photography,” J. Opt. Soc. Am. 58(1), 71–76 (1968).
    [Crossref]
  4. T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
    [Crossref]
  5. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997).
    [Crossref] [PubMed]
  6. A. Stern, Y. Yitzhaky, and B. Javidi, “Perceivable light fields: Matching the requirements between the human visual system and autostereoscopic 3-D displays,” Proc. IEEE 102(10), 1571–1587 (2014).
    [Crossref]
  7. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. 37(11), 2034–2045 (1998).
    [Crossref] [PubMed]
  8. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27(5), 324–326 (2002).
    [Crossref] [PubMed]
  9. S. Jung, J.-H. Park, H. Choi, and B. Lee, “Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement,” Opt. Express 11(12), 1346–1356 (2003).
    [Crossref] [PubMed]
  10. P. Latorre-Carmona, E. Sánchez-Ortiga, X. Xiao, F. Pla, M. Martínez-Corral, H. Navarro, G. Saavedra, and B. Javidi, “Multispectral integral imaging acquisition and processing using a monochrome camera and a liquid crystal tunable filter,” Opt. Express 20(23), 25960–25969 (2012).
    [Crossref] [PubMed]
  11. A. Carnicer and B. Javidi, “Polarimetric 3D integral imaging in photon-starved conditions,” Opt. Express 23(5), 6408–6417 (2015).
    [Crossref] [PubMed]
  12. X. Xiao, B. Javidi, M. Martínez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications [Invited],” Appl. Opt. 52(4), 546–560 (2013).
    [Crossref] [PubMed]
  13. H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001).
    [Crossref] [PubMed]
  14. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12(3), 483–491 (2004).
    [Crossref] [PubMed]
  15. S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12(19), 4579–4588 (2004).
    [Crossref] [PubMed]
  16. M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
    [Crossref]
  17. D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
    [Crossref]
  18. J.-Y. Jang, J. I. Ser, S. Cha, and S. H. Shin, “Depth extraction by using the correlation of the periodic function with an elemental image in integral imaging,” Appl. Opt. 51(16), 3279–3286 (2012).
    [Crossref] [PubMed]
  19. J. Y. Jang, D. Shin, B. G. Lee, S. P. Hong, and E. S. Kim, “3D image correlator using computational integral imaging reconstruction based on modified convolution property of periodic functions,” J. Opt. Soc. Korea 18(4), 388–394 (2014).
    [Crossref]
  20. J. Y. Jang, D. Shin, and E. S. Kim, “Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-imaging,” Opt. Express 22(2), 1533–1550 (2014).
    [Crossref] [PubMed]
  21. C. W. Helstrom, “Image restoration by the method of least squares,” J. Opt. Soc. Am. 57(3), 297–303 (1967).
    [Crossref]
  22. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002).
    [Crossref] [PubMed]
  23. H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
    [Crossref]

2015 (1)

2014 (3)

2013 (2)

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

X. Xiao, B. Javidi, M. Martínez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications [Invited],” Appl. Opt. 52(4), 546–560 (2013).
[Crossref] [PubMed]

2012 (2)

2009 (2)

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

2004 (2)

2003 (1)

2002 (2)

2001 (1)

1998 (1)

1997 (1)

1980 (1)

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
[Crossref]

1968 (1)

1967 (1)

1931 (1)

1908 (1)

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Arai, J.

Arimoto, H.

Burckhardt, C. B.

Carnicer, A.

Cha, S.

Cho, M.

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

Choi, H.

Dorado, A.

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

Helstrom, C. W.

Hong, S. P.

Hong, S.-H.

Hoshino, H.

Ives, H. E.

Jang, J. S.

Jang, J. Y.

Jang, J.-S.

Jang, J.-Y.

Javidi, B.

A. Carnicer and B. Javidi, “Polarimetric 3D integral imaging in photon-starved conditions,” Opt. Express 23(5), 6408–6417 (2015).
[Crossref] [PubMed]

A. Stern, Y. Yitzhaky, and B. Javidi, “Perceivable light fields: Matching the requirements between the human visual system and autostereoscopic 3-D displays,” Proc. IEEE 102(10), 1571–1587 (2014).
[Crossref]

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

X. Xiao, B. Javidi, M. Martínez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications [Invited],” Appl. Opt. 52(4), 546–560 (2013).
[Crossref] [PubMed]

P. Latorre-Carmona, E. Sánchez-Ortiga, X. Xiao, F. Pla, M. Martínez-Corral, H. Navarro, G. Saavedra, and B. Javidi, “Multispectral integral imaging acquisition and processing using a monochrome camera and a liquid crystal tunable filter,” Opt. Express 20(23), 25960–25969 (2012).
[Crossref] [PubMed]

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12(3), 483–491 (2004).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12(19), 4579–4588 (2004).
[Crossref] [PubMed]

J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27(5), 324–326 (2002).
[Crossref] [PubMed]

J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002).
[Crossref] [PubMed]

H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001).
[Crossref] [PubMed]

Jung, S.

Kim, E. S.

Latorre-Carmona, P.

Lee, B.

Lee, B. G.

Lippmann, G.

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Llavador, A.

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

Martinez-Corral, M.

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

Martínez-Corral, M.

Navarro, H.

Okano, F.

Okoshi, T.

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
[Crossref]

Park, J.-H.

Pla, F.

Saavedra, G.

Sánchez-Ortiga, E.

Ser, J. I.

Shin, D.

Shin, D. H.

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Shin, S. H.

Stern, A.

A. Stern, Y. Yitzhaky, and B. Javidi, “Perceivable light fields: Matching the requirements between the human visual system and autostereoscopic 3-D displays,” Proc. IEEE 102(10), 1571–1587 (2014).
[Crossref]

X. Xiao, B. Javidi, M. Martínez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications [Invited],” Appl. Opt. 52(4), 546–560 (2013).
[Crossref] [PubMed]

Xiao, X.

Yitzhaky, Y.

A. Stern, Y. Yitzhaky, and B. Javidi, “Perceivable light fields: Matching the requirements between the human visual system and autostereoscopic 3-D displays,” Proc. IEEE 102(10), 1571–1587 (2014).
[Crossref]

Yoo, H.

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Yuyama, I.

Appl. Opt. (4)

J. Disp. Technol. (2)

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

J. Opt. Soc. Am. (3)

J. Opt. Soc. Korea (1)

J. Phys. (1)

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Opt. Commun. (1)

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Opt. Express (6)

Opt. Lett. (3)

Proc. IEEE (2)

T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68(5), 548–564 (1980).
[Crossref]

A. Stern, Y. Yitzhaky, and B. Javidi, “Perceivable light fields: Matching the requirements between the human visual system and autostereoscopic 3-D displays,” Proc. IEEE 102(10), 1571–1587 (2014).
[Crossref]

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1 Scheme of the pickup process in an InI system. Any point source located in a given transverse plane in the object space produces a pattern with the same periodicity over the sensor (a), whereas point sources placed at different axial positions generate diverse IRFs as shown in (b)
Fig. 2
Fig. 2 Acquisition process with an InI system. The integral image is obtained by summation of the individual intensity distributions relative to each object of the scene. These individual intensity distributions are the result of the convolution of the object intensity distribution with the corresponding IRF.
Fig. 3
Fig. 3 (a) Integral image obtained with the experimental setup. The integral image is composed of 11x11 EIs. (b) 3x3 EIs extracted from (a). The inset shows the central view of the integral image.
Fig. 4
Fig. 4 Depth reconstruction of the 3D scene after applying our algorithm based on 2D deconvolution. The reconstruction is calculated at planes located at distances (a) 320 mm, and (b) 350 mm. The corresponding IRFs are depicted in the bottom-right of the figures. The impulse response array is shown in the red box, with a periodicity of (a) 515 and (b) 523 pixels.
Fig. 5
Fig. 5 USAF 1951 test chart reconstructed with (a) the standard reconstruction algorithm and (b) our proposed method. We achieve a resolution of 0.4 mm, corresponding to the Element 3 of Group 0 (green box).
Fig. 6
Fig. 6 Application of the proposed algorithm for simultaneously synthesizing different depths. (a) and (b) show the simultaneously performed reconstruction of two non-consecutive planes in the 3D scene. Besides, an extended depth of field reconstruction is possible. In (c), all planes of the object located at 320mm are focused, whereas in (d) the whole 3D scene is focused.

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

p ( z 0 ) = d ( 1 + g z 0 ) .
I ( x ;z ) = 1 M z 2 O ( x M z ; z M z 2 ) 2 h ( x ;z ) ,
h ( x ;z ) = m x m y δ ( x m p ( z ) ) ,
I ( x ) = z = 2 g I ( x ;z ) d z .
I ( x ) = n = 1 N I ( x ; z n ) .
I ˜ ( u ) = n = 1 N O ˜ ( M z n u ; z n M z n 2 ) H ( u ; z n ) ,
I ˜ R ( u ) = I ˜ ( u ) H ^ * ( u ; z 1 ) | H ^ ( u ; z 1 ) | 2 + w 2 ,
I ˜ R ( u ) = O ˜ ( M z 1 u ; z 1 M z 1 2 ) + n = 2 N O ˜ ( M z n u ; z n M z n 2 ) H ( u ; z n ) H ^ * ( u ; z 1 ) | H ^ ( u ; z 1 ) | 2 + w 2 .
I R ( x ) O ( x M z 1 ; z 1 M z 1 2 ) + n = 2 N O z n ( x M z n ; z n M z n 2 ) h d e f ( x ; z 1 ) ,

Metrics