Abstract

A major theme of computational photography is the acquisition of lightfield, which opens up new imaging capabilities, such as focusing after image capture. However, to capture the lightfield, one normally has to sacrifice significant spatial resolution as compared to normal imaging for a fixed sensor size. In this work, we present a new design for lightfield acquisition, which allows for the capture of a higher resolution lightfield by using two attenuation masks. They are positioned at the aperture stop and the optical path respectively, so that the four-dimensional (4D) lightfield spectrum is encoded and sampled by a two-dimensional (2D) camera sensor in a single snapshot. Then, during post-processing, by exploiting the coherence embedded in a lightfield, we can retrieve the desired 4D lightfield with a higher resolution using inverse imaging. The performance of our proposed method is demonstrated with simulations based on actual lightfield datasets.

© 2012 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. E. Y. Lam, “Computational photography: Advances and challenges,” in Tribute to Joseph W. Goodman, H. J. Caulfield and H. H. Arsenault, eds., Proc. SPIE 8122, 81220O (2011).
  2. W. T. Cathey and E. R. Dowski, “New paradigm for imaging systems,” Appl. Opt. 41, 6080–6092 (2002).
    [CrossRef] [PubMed]
  3. J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
    [CrossRef] [PubMed]
  4. W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
    [CrossRef]
  5. T. Mirani, D. Rajan, M. P. Christensen, S. C. Douglas, and S. L. Wood, “Computational imaging systems: Joint design and end-to-end optimality,” Appl. Opt. 47, B86–B103 (2008).
    [CrossRef] [PubMed]
  6. M. Levoy and P. Hanrahan, “Light field rendering,” in Proceedings of ACM SIGGRAPH (1996), pp. 31–42.
  7. J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts and Company Publishers, 2004).
  8. E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds. (MIT Press, 1991), pp. 3–20.
  9. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of ACM SIGGRAPH (1996), pp. 43–54.
  10. G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. Théor. Appl. 7, 821–825 (1908).
    [CrossRef] [PubMed]
  11. B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
    [CrossRef]
  12. E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
    [CrossRef]
  13. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.
  14. A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).
  15. A. Agrawal, A. Veeraraghavan, and R. Raskar, “Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography,” Comput. Graph. Forum 29, 763–772 (2010).
    [CrossRef]
  16. T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.
  17. Z. Xu and E. Y. Lam, “Light field superresolution reconstruction in computational photography,” in Signal Recovery and Synthesis, (Optical Society of America, 2011), p. SMB3.
  18. C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
    [CrossRef]
  19. A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.
    [CrossRef]
  20. R. N. Bracewell, The Fourier Transform and Its Applications, 3rd ed. (McGraw-Hill, 1999).
  21. J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.
  22. A. Levin, W. T. Freeman, and F. Durand, “Understanding camera trade-offs through a Bayesian analysis of light field projections,” in Proceedings of the 10th European Conference on Computer Vision (2008), pp. 88–101.
  23. Z. Xu and E. Y. Lam, “A spatial projection analysis of light field capture,” in Frontiers in Optics, (Optical Society of America, 2010), p. FWH2.
  24. W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
    [CrossRef]
  25. W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).
  26. L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992).
    [CrossRef]
  27. E. Y. Lam, X. Zhang, H. Vo, T.-C. Poon, and G. Indebetouw, “Three-dimensional microscopy and sectional image reconstruction using optical scanning holography,” Appl. Opt. 48, H113–H119 (2009).
    [CrossRef] [PubMed]
  28. X. Zhang and E. Y. Lam, “Edge-preserving sectional image reconstruction in optical scanning holography,” J. Opt. Soc. Am. A 27, 1630–1637 (2010).
    [CrossRef]
  29. Z. Xu and E. Y. Lam, “Image reconstruction using spectroscopic and hyperspectral information for compressive terahertz imaging,” J. Opt. Soc. Am. A 27, 1638–1646 (2010).
    [CrossRef]
  30. “The (new) Stanford light field archive,” http://lightfield.stanford.edu/lfs.html .

2010

A. Agrawal, A. Veeraraghavan, and R. Raskar, “Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography,” Comput. Graph. Forum 29, 763–772 (2010).
[CrossRef]

W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).

X. Zhang and E. Y. Lam, “Edge-preserving sectional image reconstruction in optical scanning holography,” J. Opt. Soc. Am. A 27, 1630–1637 (2010).
[CrossRef]

Z. Xu and E. Y. Lam, “Image reconstruction using spectroscopic and hyperspectral information for compressive terahertz imaging,” J. Opt. Soc. Am. A 27, 1638–1646 (2010).
[CrossRef]

2009

2008

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

T. Mirani, D. Rajan, M. P. Christensen, S. C. Douglas, and S. L. Wood, “Computational imaging systems: Joint design and end-to-end optimality,” Appl. Opt. 47, B86–B103 (2008).
[CrossRef] [PubMed]

2007

W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
[CrossRef]

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

2003

2002

2000

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.

1992

E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992).
[CrossRef]

1908

G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. Théor. Appl. 7, 821–825 (1908).
[CrossRef] [PubMed]

Adams, A.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Adelson, E. H.

E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds. (MIT Press, 1991), pp. 3–20.

Agrawal, A.

A. Agrawal, A. Veeraraghavan, and R. Raskar, “Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography,” Comput. Graph. Forum 29, 763–772 (2010).
[CrossRef]

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

Antunez, E.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Athale, R.

Bajwa, W. U.

W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Barth, A.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Bergen, J. R.

E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds. (MIT Press, 1991), pp. 3–20.

Bracewell, R. N.

R. N. Bracewell, The Fourier Transform and Its Applications, 3rd ed. (McGraw-Hill, 1999).

Brédif, M.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

Cathey, W. T.

Chai, J.-X.

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.

Chan, S.-C.

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.

Chan, W.-S.

W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
[CrossRef]

Chen, H. H.

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

Christensen, M. P.

Cohen, M. F.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of ACM SIGGRAPH (1996), pp. 43–54.

Curless, B.

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Douglas, S. C.

Dowski, E. R.

Durand, F.

A. Levin, W. T. Freeman, and F. Durand, “Understanding camera trade-offs through a Bayesian analysis of light field projections,” in Proceedings of the 10th European Conference on Computer Vision (2008), pp. 88–101.

Duval, G.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

Fatemi, E.

L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992).
[CrossRef]

Freeman, W. T.

A. Levin, W. T. Freeman, and F. Durand, “Understanding camera trade-offs through a Bayesian analysis of light field projections,” in Proceedings of the 10th European Conference on Computer Vision (2008), pp. 88–101.

Georgeiv, T.

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Georgiev, T.

A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.
[CrossRef]

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts and Company Publishers, 2004).

Gortler, S. J.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of ACM SIGGRAPH (1996), pp. 43–54.

Grzeszczuk, R.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of ACM SIGGRAPH (1996), pp. 43–54.

Hanrahan, P.

M. Levoy and P. Hanrahan, “Light field rendering,” in Proceedings of ACM SIGGRAPH (1996), pp. 31–42.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

Haupt, J. D.

W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Horowitz, M.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Indebetouw, G.

Intwala, C.

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Joshi, N.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Lam, E. Y.

X. Zhang and E. Y. Lam, “Edge-preserving sectional image reconstruction in optical scanning holography,” J. Opt. Soc. Am. A 27, 1630–1637 (2010).
[CrossRef]

Z. Xu and E. Y. Lam, “Image reconstruction using spectroscopic and hyperspectral information for compressive terahertz imaging,” J. Opt. Soc. Am. A 27, 1638–1646 (2010).
[CrossRef]

E. Y. Lam, X. Zhang, H. Vo, T.-C. Poon, and G. Indebetouw, “Three-dimensional microscopy and sectional image reconstruction using optical scanning holography,” Appl. Opt. 48, H113–H119 (2009).
[CrossRef] [PubMed]

W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
[CrossRef]

E. Y. Lam, “Computational photography: Advances and challenges,” in Tribute to Joseph W. Goodman, H. J. Caulfield and H. H. Arsenault, eds., Proc. SPIE 8122, 81220O (2011).

Z. Xu and E. Y. Lam, “Light field superresolution reconstruction in computational photography,” in Signal Recovery and Synthesis, (Optical Society of America, 2011), p. SMB3.

Z. Xu and E. Y. Lam, “A spatial projection analysis of light field capture,” in Frontiers in Optics, (Optical Society of America, 2010), p. FWH2.

Levin, A.

A. Levin, W. T. Freeman, and F. Durand, “Understanding camera trade-offs through a Bayesian analysis of light field projections,” in Proceedings of the 10th European Conference on Computer Vision (2008), pp. 88–101.

Levoy, M.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

M. Levoy and P. Hanrahan, “Light field rendering,” in Proceedings of ACM SIGGRAPH (1996), pp. 31–42.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Liang, C.-K.

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

Lin, T.-H.

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

Lippmann, G.

G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. Théor. Appl. 7, 821–825 (1908).
[CrossRef] [PubMed]

Liu, C.

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

Lumsdaine, A.

A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.
[CrossRef]

Mait, J.

Mak, G. Y.

W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
[CrossRef]

Mirani, T.

Mohan, A.

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

Morgan, S.

W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).

Nayar, S.

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Ng, M. K.

W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
[CrossRef]

Ng, R.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

Nowak, R. D.

W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Osher, S.

L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992).
[CrossRef]

Poon, T.-C.

Rajan, D.

Raskar, R.

A. Agrawal, A. Veeraraghavan, and R. Raskar, “Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography,” Comput. Graph. Forum 29, 763–772 (2010).
[CrossRef]

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

Raz, G. M.

W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Rudin, L. I.

L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992).
[CrossRef]

Salesin, D.

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Shum, H.-Y.

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.

Szeliski, R.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of ACM SIGGRAPH (1996), pp. 43–54.

Talvala, E.-V.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Tong, X.

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.

Tumblin, J.

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

Vaish, V.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

van der Gracht, J.

Veeraraghavan, A.

A. Agrawal, A. Veeraraghavan, and R. Raskar, “Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography,” Comput. Graph. Forum 29, 763–772 (2010).
[CrossRef]

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

Vo, H.

Wang, J. Y.

E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

Wilburn, B.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

Wong, B.-Y.

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

Wood, S. L.

Wright, S. J.

W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Xu, Z.

Z. Xu and E. Y. Lam, “Image reconstruction using spectroscopic and hyperspectral information for compressive terahertz imaging,” J. Opt. Soc. Am. A 27, 1638–1646 (2010).
[CrossRef]

Z. Xu and E. Y. Lam, “A spatial projection analysis of light field capture,” in Frontiers in Optics, (Optical Society of America, 2010), p. FWH2.

Z. Xu and E. Y. Lam, “Light field superresolution reconstruction in computational photography,” in Signal Recovery and Synthesis, (Optical Society of America, 2011), p. SMB3.

Yang, J.

W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).

Yin, W.

W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).

Zhang, X.

Zhang, Y.

W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).

Zheng, K. C.

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Appl. Opt.

Comput. Graph. Forum

A. Agrawal, A. Veeraraghavan, and R. Raskar, “Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography,” Comput. Graph. Forum 29, 763–772 (2010).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

E. H. Adelson and J. Y. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

J. Opt. Soc. Am. A

J. Phys. Théor. Appl.

G. Lippmann, “Épreuves réversibles donnant la sensation du relief,” J. Phys. Théor. Appl. 7, 821–825 (1908).
[CrossRef] [PubMed]

Multidim. Syst. Sign. Process

W.-S. Chan, E. Y. Lam, M. K. Ng, and G. Y. Mak, “Super-resolution reconstruction in a computational compound-eye imaging system,” Multidim. Syst. Sign. Process 18, 83–101 (2007).
[CrossRef]

Opt. Express

Physica D

L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D 60, 259–268 (1992).
[CrossRef]

Proceedings of ACM SIGGRAPH

A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing,” in Proceedings of ACM SIGGRAPH 26, (2007).

J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum, “Plenoptic sampling,” in Proceedings of ACM SIGGRAPH 27 (2000), pp. 307–318.

C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH 27 (2008), pp. 1–10.
[CrossRef]

Visual Communications and Image Processing

W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing, Proc. SPIE  7744, 77440K (2010).

Other

A. Levin, W. T. Freeman, and F. Durand, “Understanding camera trade-offs through a Bayesian analysis of light field projections,” in Proceedings of the 10th European Conference on Computer Vision (2008), pp. 88–101.

Z. Xu and E. Y. Lam, “A spatial projection analysis of light field capture,” in Frontiers in Optics, (Optical Society of America, 2010), p. FWH2.

W. U. Bajwa, J. D. Haupt, G. M. Raz, S. J. Wright, and R. D. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE/SP 14th Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

“The (new) Stanford light field archive,” http://lightfield.stanford.edu/lfs.html .

A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.
[CrossRef]

R. N. Bracewell, The Fourier Transform and Its Applications, 3rd ed. (McGraw-Hill, 1999).

T. Georgeiv, K. C. Zheng, B. Curless, D. Salesin, S. Nayar, and C. Intwala, “Spatio-angular resolution tradeoff in integral photography,” in Proceedings of Eurographics Symposium on Rendering (2006), pp. 263–272.

Z. Xu and E. Y. Lam, “Light field superresolution reconstruction in computational photography,” in Signal Recovery and Synthesis, (Optical Society of America, 2011), p. SMB3.

B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, “High performance imaging using large camera arrays,” in Proceedings of ACM SIGGRAPH (2005), pp. 765–776.
[CrossRef]

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Report CTSR (2005), pp. 1–11.

E. Y. Lam, “Computational photography: Advances and challenges,” in Tribute to Joseph W. Goodman, H. J. Caulfield and H. H. Arsenault, eds., Proc. SPIE 8122, 81220O (2011).

M. Levoy and P. Hanrahan, “Light field rendering,” in Proceedings of ACM SIGGRAPH (1996), pp. 31–42.

J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts and Company Publishers, 2004).

E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” in Computational Models of Visual Processing, M. S. Landy and J. A. Movshon, eds. (MIT Press, 1991), pp. 3–20.

S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The lumigraph,” in Proceedings of ACM SIGGRAPH (1996), pp. 43–54.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

The two-plane parametrization of a 4D lightfield.

Fig. 2
Fig. 2

Schematic diagram of a regular camera, with an attenuation mask placed inside it.

Fig. 3
Fig. 3

The modulation in the lightfield-frequency domain.

Fig. 4
Fig. 4

Our proposed lightfield camera, with two attenuation masks respectively placed at the aperture stop and the optical path in the camera.

Fig. 5
Fig. 5

The corresponding lightfield-frequency domain operations in our double-mask light-field camera. (The asterisk pattern in the figure denotes the convolution.)

Fig. 6
Fig. 6

(a) The pattern of the first mask; (b)–(e) are the pattern parts of the second mask, respectively in cases of using full, 64%, 36% and 16% sensor size.

Fig. 8
Fig. 8

The reconstructed images at one selected viewpoint by using the least-norm method (left column) and the proposed method in Section 2.5 (right column): (a) ground truth, (b) and (c) full size, (d) and (e) 64% sensor size, (f) and (g) 36% sensor size, (h) and (i) 16% sensor size.

Fig. 9
Fig. 9

Reconstructions when using 36% sensor size: (a) ground truth; (b) the best quality that can be achieved by using the traditional lightfield cameras; (c) our reconstruction with the least-norm method; (d) our reconstruction with the proposed iterative method.

Fig. 10
Fig. 10

Reconstructions when using 16% sensor size: (a) ground truth; (b) the best quality that can be achieved by using the traditional lightfield cameras; (c) our reconstruction with the least-norm method; (d) our reconstruction with the proposed iterative method.

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

o ( u , s ) = ( u , s ) m ( u , s ) .
B C D E = A B A D x u s u = d z d .
x = ( 1 z d ) s + z d u .
m ( u , s ) = c [ ( 1 z d ) s + z d u ] .
o ( f u , f s ) = ( f u , f s ) * M ( f u , f s ) ,
M ( f u , f s ) = c [ ( 1 z d ) s + z d u ] exp [ j 2 π ( f u u + f s s ) ] d u d s = { c [ ( 1 z d ) s + z d u ] exp ( j 2 π f s s ) d s } exp ( j 2 π f u u ) d u .
M ( f u , f s ) = d d z { C ( d d z f s ) exp [ j 2 π ( z d z f s ) u ] } exp ( j 2 π f u u ) d u = d d z C ( d d z f s ) δ ( f u z d z f s ) ,
α = arctan z d z .
M ( f u , f s ) = C ( f u ) δ ( f s ) .
i ( s ) = ( u , s ) m 1 ( u , s ) m 2 ( u , s ) d u = ( u , s ) c 1 ( u ) c 2 [ ( 1 z d ) s + z d u ] d u ,
i = F 1 M 2 M 1 F = F 1 M F = A ,
C 1 ( f u ) = i = ( n 1 ) n 1 a i δ ( f u i Δ f u ) ,
[ a 0 a 1 a n 1 a 1 a 0 a n 2 a ( n 1 ) a ( n 2 ) a 0 ] p × n .
C 2 ( f x ) = i = ( k 1 ) / 2 ( k 1 ) / 2 δ ( f x i B s ) ,
= A i = A T ( A A T ) 1 i ,
= argmin { 1 2 A i 2 2 + λ u i u TV + μ s i s TV } ,

Metrics