Abstract

We propose a method to capture light ray field of three-dimensional scene using focal plane sweeping. Multiple images are captured using a usual camera at different focal distances, spanning the three-dimensional scene. The captured images are then back-projected to four-dimensional spatio-angular space to obtain the light ray field. The obtained light ray field can be visualized either using digital processing or optical reconstruction using various three-dimensional display techniques including integral imaging, layered display, and holography.

© 2014 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Liquid-crystal-display-based touchable light field three-dimensional display using display-capture mapping calibration

Yifan Peng, Haifeng Li, Qing Zhong, and Xu Liu
Appl. Opt. 51(25) 6014-6019 (2012)

Hologram synthesis of three-dimensional real objects using portable integral imaging camera

Sung-Keun Lee, Sung-In Hong, Yong-Soo Kim, Hong-Gi Lim, Na-Young Jo, and Jae-Hyeung Park
Opt. Express 21(20) 23662-23670 (2013)

Analysis of the noise in backprojection light field acquisition and its optimization

Ni Chen, Zhenbo Ren, Dayan Li, Edmund Y. Lam, and Guohai Situ
Appl. Opt. 56(13) F20-F26 (2017)

References

  • View by:
  • |
  • |
  • |

  1. R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Rep. CTSR 2005–02 (Stanford University, 2005).
  2. B. Javidi, S. Yeom, I. Moon, and M. Daneshpanah, “Real-time automated 3D sensing, detection, and recognition of dynamic biological micro-organic events,” Opt. Express 14(9), 3806–3829 (2006).
    [Crossref] [PubMed]
  3. M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009).
    [Crossref] [PubMed]
  4. Y. T. Lim, J. H. Park, K. C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009).
    [Crossref] [PubMed]
  5. M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, and M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Express 21(21), 25418–25439 (2013).
    [Crossref] [PubMed]
  6. J. Arai, T. Yamashita, M. Miura, H. Hiura, N. Okaichi, F. Okano, and R. Funatsu, “Integral three-dimensional image capture equipment with closely positioned lens array and image sensor,” Opt. Lett. 38(12), 2044–2046 (2013).
    [Crossref] [PubMed]
  7. J. Kim, J.-H. Jung, Y. Jeong, K. Hong, and B. Lee, “Real-time integral imaging system for light field microscopy,” Opt. Express 22(9), 10210–10220 (2014).
    [Crossref] [PubMed]
  8. S.-K. Lee, S.-I. Hong, Y.-S. Kim, H.-G. Lim, N.-Y. Jo, and J.-H. Park, “Hologram synthesis of three-dimensional real objects using portable integral imaging camera,” Opt. Express 21(20), 23662–23670 (2013).
    [Crossref] [PubMed]
  9. A. Orth and K. Crozier, “Microscopy with microlens arrays: high throughput, high resolution and light-field imaging,” Opt. Express 20(12), 13522–13531 (2012).
    [Crossref] [PubMed]
  10. H. Navarro, J. C. Barreiro, G. Saavedra, M. Martínez-Corral, and B. Javidi, “High-resolution far-field integral-imaging camera by double snapshot,” Opt. Express 20(2), 890–895 (2012).
    [Crossref] [PubMed]
  11. S. A. Shroff and K. Berkner, “Image formation analysis and high resolution image reconstruction for plenoptic imaging systems,” Appl. Opt. 52(10), D22–D31 (2013).
    [Crossref] [PubMed]
  12. T. E. Bishop and P. Favaro, “The light field camera: extended depth of field, aliasing, and super-resolution,” IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 972–986 (2012).
    [Crossref] [PubMed]
  13. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27(5), 324–326 (2002).
    [Crossref] [PubMed]
  14. K. Hong, J. Hong, J.-H. Jung, J.-H. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18(11), 12002–12016 (2010).
    [Crossref] [PubMed]
  15. S. X. Pan and A. C. Kak, “A computational study of reconstruction algorithms for diffraction tomography: Interpolation versus filtered backpropagation,” IEEE Trans. Acoust. Speech Signal Process. 31(5), 1262–1275 (1983).
    [Crossref]
  16. R. Schulein, M. DaneshPanah, and B. Javidi, “3D imaging with axially distributed sensing,” Opt. Lett. 34(13), 2012–2014 (2009).
    [Crossref] [PubMed]
  17. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009).
    [Crossref] [PubMed]
  18. G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
    [Crossref]
  19. N.-Y. Jo, H.-G. Lim, S.-K. Lee, Y.-S. Kim, and J.-H. Park, “Depth enhancement of multi-layer light field display using polarization dependent internal reflection,” Opt. Express 21(24), 29628–29636 (2013).
    [Crossref] [PubMed]
  20. J.-H. Park and K.-M. Jeong, “Frequency domain depth filtering of integral imaging,” Opt. Express 19(19), 18729–18741 (2011).
    [Crossref] [PubMed]

2014 (1)

2013 (5)

2012 (3)

2011 (2)

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
[Crossref]

J.-H. Park and K.-M. Jeong, “Frequency domain depth filtering of integral imaging,” Opt. Express 19(19), 18729–18741 (2011).
[Crossref] [PubMed]

2010 (1)

2009 (4)

2006 (1)

2002 (1)

1983 (1)

S. X. Pan and A. C. Kak, “A computational study of reconstruction algorithms for diffraction tomography: Interpolation versus filtered backpropagation,” IEEE Trans. Acoust. Speech Signal Process. 31(5), 1262–1275 (1983).
[Crossref]

Andalman, A.

Arai, J.

Barreiro, J. C.

Berkner, K.

Bishop, T. E.

T. E. Bishop and P. Favaro, “The light field camera: extended depth of field, aliasing, and super-resolution,” IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 972–986 (2012).
[Crossref] [PubMed]

Broxton, M.

Cohen, N.

Crozier, K.

DaneshPanah, M.

Deisseroth, K.

Favaro, P.

T. E. Bishop and P. Favaro, “The light field camera: extended depth of field, aliasing, and super-resolution,” IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 972–986 (2012).
[Crossref] [PubMed]

Funatsu, R.

Grosenick, L.

Heidrich, W.

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
[Crossref]

Hiura, H.

Hong, J.

Hong, K.

Hong, S.-I.

Jang, J.-S.

Javidi, B.

Jeong, K.-M.

Jeong, Y.

Jo, N.-Y.

Jung, J.-H.

Kak, A. C.

S. X. Pan and A. C. Kak, “A computational study of reconstruction algorithms for diffraction tomography: Interpolation versus filtered backpropagation,” IEEE Trans. Acoust. Speech Signal Process. 31(5), 1262–1275 (1983).
[Crossref]

Kim, J.

Kim, N.

Kim, Y.-S.

Kwon, K. C.

Lanman, D.

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
[Crossref]

Lee, B.

Lee, S.-K.

Levoy, M.

M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, and M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Express 21(21), 25418–25439 (2013).
[Crossref] [PubMed]

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009).
[Crossref] [PubMed]

Lim, H.-G.

Lim, Y. T.

Martínez-Corral, M.

McDowall, I.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009).
[Crossref] [PubMed]

Miura, M.

Moon, I.

Navarro, H.

Okaichi, N.

Okano, F.

Orth, A.

Pan, S. X.

S. X. Pan and A. C. Kak, “A computational study of reconstruction algorithms for diffraction tomography: Interpolation versus filtered backpropagation,” IEEE Trans. Acoust. Speech Signal Process. 31(5), 1262–1275 (1983).
[Crossref]

Park, J. H.

Park, J.-H.

Raskar, R.

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
[Crossref]

Saavedra, G.

Schulein, R.

Shroff, S. A.

Wetzstein, G.

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
[Crossref]

Yamashita, T.

Yang, S.

Yeom, S.

Zhang, Z.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009).
[Crossref] [PubMed]

ACM Trans. Graph. (1)

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 1–11 (2011).
[Crossref]

Appl. Opt. (2)

IEEE Trans. Acoust. Speech Signal Process. (1)

S. X. Pan and A. C. Kak, “A computational study of reconstruction algorithms for diffraction tomography: Interpolation versus filtered backpropagation,” IEEE Trans. Acoust. Speech Signal Process. 31(5), 1262–1275 (1983).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

T. E. Bishop and P. Favaro, “The light field camera: extended depth of field, aliasing, and super-resolution,” IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 972–986 (2012).
[Crossref] [PubMed]

J. Microsc. (1)

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009).
[Crossref] [PubMed]

Opt. Express (10)

Y. T. Lim, J. H. Park, K. C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009).
[Crossref] [PubMed]

M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, and M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Express 21(21), 25418–25439 (2013).
[Crossref] [PubMed]

J. Kim, J.-H. Jung, Y. Jeong, K. Hong, and B. Lee, “Real-time integral imaging system for light field microscopy,” Opt. Express 22(9), 10210–10220 (2014).
[Crossref] [PubMed]

S.-K. Lee, S.-I. Hong, Y.-S. Kim, H.-G. Lim, N.-Y. Jo, and J.-H. Park, “Hologram synthesis of three-dimensional real objects using portable integral imaging camera,” Opt. Express 21(20), 23662–23670 (2013).
[Crossref] [PubMed]

A. Orth and K. Crozier, “Microscopy with microlens arrays: high throughput, high resolution and light-field imaging,” Opt. Express 20(12), 13522–13531 (2012).
[Crossref] [PubMed]

H. Navarro, J. C. Barreiro, G. Saavedra, M. Martínez-Corral, and B. Javidi, “High-resolution far-field integral-imaging camera by double snapshot,” Opt. Express 20(2), 890–895 (2012).
[Crossref] [PubMed]

B. Javidi, S. Yeom, I. Moon, and M. Daneshpanah, “Real-time automated 3D sensing, detection, and recognition of dynamic biological micro-organic events,” Opt. Express 14(9), 3806–3829 (2006).
[Crossref] [PubMed]

K. Hong, J. Hong, J.-H. Jung, J.-H. Park, and B. Lee, “Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging,” Opt. Express 18(11), 12002–12016 (2010).
[Crossref] [PubMed]

N.-Y. Jo, H.-G. Lim, S.-K. Lee, Y.-S. Kim, and J.-H. Park, “Depth enhancement of multi-layer light field display using polarization dependent internal reflection,” Opt. Express 21(24), 29628–29636 (2013).
[Crossref] [PubMed]

J.-H. Park and K.-M. Jeong, “Frequency domain depth filtering of integral imaging,” Opt. Express 19(19), 18729–18741 (2011).
[Crossref] [PubMed]

Opt. Lett. (3)

Other (1)

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Rep. CTSR 2005–02 (Stanford University, 2005).

Supplementary Material (14)

» Media 1: MP4 (346 KB)     
» Media 2: MP4 (150 KB)     
» Media 3: MP4 (346 KB)     
» Media 4: MP4 (387 KB)     
» Media 5: MP4 (389 KB)     
» Media 6: MP4 (335 KB)     
» Media 7: MP4 (382 KB)     
» Media 8: MP4 (405 KB)     
» Media 9: MP4 (512 KB)     
» Media 10: MP4 (410 KB)     
» Media 11: MP4 (1681 KB)     
» Media 12: MP4 (1292 KB)     
» Media 13: MP4 (265 KB)     
» Media 14: MP4 (1207 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1 Conceptual diagram of the proposed method.
Fig. 2
Fig. 2 Relationship between the captured image and the light ray field.
Fig. 3
Fig. 3 Light ray field reconstruction from captured images using back projection.
Fig. 4
Fig. 4 Spectrum area of the light ray field corresponding to lambertian 3D object and the reconstructed slices by the proposed method. The spatial bandwidth Bx of individual slice is dependent on Δz due to slightly different magnification but it is ignored in the figure for simplicity.
Fig. 5
Fig. 5 (a) Experimental setup and (b) examples of the captured images.
Fig. 6
Fig. 6 Reconstructed light ray field.
Fig. 7
Fig. 7 Digital visualization: parallax images (a) Parallax view synthesis from light ray field (b) Movie of synthesized parallax views (Media 1).
Fig. 8
Fig. 8 Digital visualization: refocus images (a) Refocus image synthesis from light ray field (b) Movie of synthesized refocused images (Media 2).
Fig. 9
Fig. 9 Movie of reconstructed parallax using (a) 36 images with 2mm step (Media 3) (b) 9 images with 8mm step (Media 4) (c) 6 images with 12mm step (Media 5) (d) 3 images with 24mm step (Media 6).
Fig. 10
Fig. 10 Movie of reconstructed parallax with different systems (a) Camera without extension tube (Media 7) (b) Microscopy (insect object, Media 8) (c) Microscopy (ant skin, Media 9) (d) Microscopy (ant eye, Media 10).
Fig. 11
Fig. 11 Optical reconstruction with integral imaging display (a) Synthesis of elemental images from the light ray field (b) Synthesized elemental images (c) Observed images from different directions (Media 11).
Fig. 12
Fig. 12 Optical reconstruction with layered display (a) Synthesized images for front and rear panels (b) Observed images from different directions (Media 12).
Fig. 13
Fig. 13 Optical reconstruction using holography (a) Synthesized complex field (b) Optical reconstruction of scene 1 (Media 13) (c) Optical reconstruction of scene 2 (Media 14).

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

I z r + Δ z n ( x c , y c ) = N A L ( x c m Δ z n θ x , y c m Δ z n θ y , θ x , θ y ) d θ x d θ y ,
L r e c ( x , y , θ x , θ y ) = 1 N n I z r + Δ z n ( x + Δ z n θ x m , y + Δ z n θ y m ) ,

Metrics