Abstract

Optical phase-space functions describe spatial and angular information simultaneously; examples of optical phase-space functions include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient and do not achieve diffraction-limited resolution. Here, we propose a multiplexed method that solves these problems. We use a spatial light modulator (SLM) in the pupil plane of a microscope in order to sequentially pattern multiplexed coded apertures while capturing images in real space. Then, we reconstruct the 3D fluorescence distribution of our sample by solving an inverse problem via regularized least squares with a proximal accelerated gradient descent solver. We experimentally reconstruct a 101 Megavoxel 3D volume (1010×510×500µm with NA 0.4), demonstrating improved acquisition time, light throughput and resolution compared to scanning aperture methods. Our flexible patterning scheme further allows sparsity in the sample to be exploited for reduced data capture.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
3D imaging in volumetric scattering media using phase-space measurements

Hsiou-Yuan Liu, Eric Jonas, Lei Tian, Jingshan Zhong, Benjamin Recht, and Laura Waller
Opt. Express 23(11) 14461-14471 (2015)

Wave optics theory and 3-D deconvolution for the light field microscope

Michael Broxton, Logan Grosenick, Samuel Yang, Noy Cohen, Aaron Andalman, Karl Deisseroth, and Marc Levoy
Opt. Express 21(21) 25418-25439 (2013)

References

  • View by:
  • |
  • |
  • |

  1. I. Cox, C. Sheppard, and T. Wilson, “Super-resolution by confocal fluorescent microscopy,” Optik 60, 391–396 (1982).
  2. F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2, 932–940 (2005).
    [Crossref] [PubMed]
  3. T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
    [Crossref] [PubMed]
  4. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
    [Crossref]
  5. M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, and M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Express 21, 25418–25439 (2013).
    [Crossref] [PubMed]
  6. Y. Y. Schechner, R. Piestun, and J. Shamir, “Wave propagation with rotating intensity distributions,” Phys. Rev. E 54, R50–R53 (1996).
    [Crossref]
  7. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
    [Crossref] [PubMed]
  8. Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
    [Crossref] [PubMed]
  9. N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7, 141–147 (2010).
    [Crossref]
  10. M. Testorf, B. Hennelly, and J. Ojeda-Castañeda, Phase-Space Optics: Fundamentals and Applications (McGraw-Hill Professional, 2009).
  11. M. Bastiaans, “The Wigner distribution function applied to optical signals and systems,” Opt. Commun. 25, 26–30 (1978).
    [Crossref]
  12. A. Accardi and G. Wornell, “Quasi light fields: extending the light field to coherent radiation,” J. Opt. Soc. Am. A 26, 2055–2066 (2009).
    [Crossref]
  13. Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–10.
  14. R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).
  15. R. Ng and P. Hanrahan, “Digital correction of lens aberrations in light field photography,” Proc. SPIE 6342, 63421E (2007).
    [Crossref]
  16. N. C. Pégard, H.-Y. Liu, N. Antipa, M. Gerlock, H. Adesnik, and L. Waller, “Compressive light-field microscopy for 3D neural activity recording,” Optica 3, 517–524 (2016).
    [Crossref]
  17. D. L. Marks, R. A. Stack, and D. J. Brady, “Three-dimensional coherence imaging in the Fresnel domain,” Appl. Opt. 38, 1332–1342 (1999).
    [Crossref]
  18. J. K. Wood, K. A. Sharma, S. Cho, T. G. Brown, and M. A. Alonso, “Using shadows to measure spatial coherence,” Opt. Lett. 39, 4927–4930 (2014).
    [Crossref] [PubMed]
  19. D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
    [Crossref]
  20. L. Tian, Z. Zhang, J. C. Petruccelli, and G. Barbastathis, “Wigner function measurement using a lenslet array,” Opt. Express 21, 10511–10525 (2013).
    [Crossref] [PubMed]
  21. L. Waller, G. Situ, and J. Fleischer, “Phase-space measurement and coherence synthesis of optical beams,” Nat. Photonics 6, 474–479 (2012).
    [Crossref]
  22. H.-Y. Liu, E. Jonas, L. Tian, J. Zhong, B. Recht, and L. Waller, “3D imaging in volumetric scattering media using phase-space measurements,” Opt. Express 23, 14461–14471 (2015).
    [Crossref] [PubMed]
  23. S. Cho, M. A. Alonso, and T. G. Brown, “Measurement of spatial coherence through diffraction from a transparent mask with a phase discontinuity,” Opt. Lett. 37, 2724–2726 (2012).
    [Crossref] [PubMed]
  24. H.-Y. Liu, J. Zhong, and L. Waller, “4D phase-space multiplexing for fluorescent microscopy,” Proc. SPIE 9720, 97200A (2016).
    [Crossref]
  25. A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
    [Crossref]
  26. C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.
  27. P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
    [Crossref]
  28. J. Chang, I. Kauvar, X. Hu, and G. Wetzstein, “Variable aperture light field photography: Overcoming the diffraction-limited spatio-angular resolution tradeoff,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), pp. 3737–3745.
  29. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences 2, 183–202 (2009).
    [Crossref]
  30. A. Walther, “Radiometry and coherence,” J. Opt. Soc. Am. 58, 1256–1259 (1968).
    [Crossref]
  31. J. Goodman, Introduction to Fourier optics (Roberts & Co., 2005).
  32. M. Bertero and P. Boccacci, “A simple method for the reduction of boundary effects in the Richardson-Lucy approach to image deconvolution,” Astronomy and Astrophysics 437, 369–374 (2005).
    [Crossref]

2016 (2)

2015 (1)

2014 (2)

J. K. Wood, K. A. Sharma, S. Cho, T. G. Brown, and M. A. Alonso, “Using shadows to measure spatial coherence,” Opt. Lett. 39, 4927–4930 (2014).
[Crossref] [PubMed]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
[Crossref] [PubMed]

2013 (2)

2012 (2)

L. Waller, G. Situ, and J. Fleischer, “Phase-space measurement and coherence synthesis of optical beams,” Nat. Photonics 6, 474–479 (2012).
[Crossref]

S. Cho, M. A. Alonso, and T. G. Brown, “Measurement of spatial coherence through diffraction from a transparent mask with a phase discontinuity,” Opt. Lett. 37, 2724–2726 (2012).
[Crossref] [PubMed]

2011 (1)

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

2010 (1)

N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7, 141–147 (2010).
[Crossref]

2009 (3)

A. Accardi and G. Wornell, “Quasi light fields: extending the light field to coherent radiation,” J. Opt. Soc. Am. A 26, 2055–2066 (2009).
[Crossref]

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences 2, 183–202 (2009).
[Crossref]

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

2007 (3)

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
[Crossref]

P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
[Crossref]

R. Ng and P. Hanrahan, “Digital correction of lens aberrations in light field photography,” Proc. SPIE 6342, 63421E (2007).
[Crossref]

2006 (1)

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

2005 (2)

F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2, 932–940 (2005).
[Crossref] [PubMed]

M. Bertero and P. Boccacci, “A simple method for the reduction of boundary effects in the Richardson-Lucy approach to image deconvolution,” Astronomy and Astrophysics 437, 369–374 (2005).
[Crossref]

2001 (1)

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

1999 (1)

1996 (1)

Y. Y. Schechner, R. Piestun, and J. Shamir, “Wave propagation with rotating intensity distributions,” Phys. Rev. E 54, R50–R53 (1996).
[Crossref]

1982 (1)

I. Cox, C. Sheppard, and T. Wilson, “Super-resolution by confocal fluorescent microscopy,” Optik 60, 391–396 (1982).

1978 (1)

M. Bastiaans, “The Wigner distribution function applied to optical signals and systems,” Opt. Commun. 25, 26–30 (1978).
[Crossref]

1968 (1)

Accardi, A.

Adams, A.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

Adesnik, H.

Alonso, M. A.

Andalman, A.

Antipa, N.

Backer, A. S.

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
[Crossref] [PubMed]

Barbastathis, G.

Bastiaans, M.

M. Bastiaans, “The Wigner distribution function applied to optical signals and systems,” Opt. Commun. 25, 26–30 (1978).
[Crossref]

Beck, A.

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences 2, 183–202 (2009).
[Crossref]

Bertero, M.

M. Bertero and P. Boccacci, “A simple method for the reduction of boundary effects in the Richardson-Lucy approach to image deconvolution,” Astronomy and Astrophysics 437, 369–374 (2005).
[Crossref]

Betzig, E.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7, 141–147 (2010).
[Crossref]

Biteen, J. S.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Boccacci, P.

M. Bertero and P. Boccacci, “A simple method for the reduction of boundary effects in the Richardson-Lucy approach to image deconvolution,” Astronomy and Astrophysics 437, 369–374 (2005).
[Crossref]

Brady, D. J.

Bredif, M.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Brown, T. G.

Broxton, M.

Chang, J.

J. Chang, I. Kauvar, X. Hu, and G. Wetzstein, “Variable aperture light field photography: Overcoming the diffraction-limited spatio-angular resolution tradeoff,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), pp. 3737–3745.

Chen, H.

C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.

Cho, S.

Christodoulides, D.

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

Cohen, N.

Coskun, T.

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

Cox, I.

I. Cox, C. Sheppard, and T. Wilson, “Super-resolution by confocal fluorescent microscopy,” Optik 60, 391–396 (1982).

Davidson, M.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Deisseroth, K.

Denk, W.

F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2, 932–940 (2005).
[Crossref] [PubMed]

Durand, F.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
[Crossref]

P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
[Crossref]

Duval, G.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Eugenieva, E.

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

Fergus, R.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
[Crossref]

Fleischer, J.

L. Waller, G. Situ, and J. Fleischer, “Phase-space measurement and coherence synthesis of optical beams,” Nat. Photonics 6, 474–479 (2012).
[Crossref]

Footer, M.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

Freeman, W. T.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
[Crossref]

Galbraith, C.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Galbraith, J.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Gao, L.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Gerlock, M.

Goodman, J.

J. Goodman, Introduction to Fourier optics (Roberts & Co., 2005).

Green, P.

P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
[Crossref]

Grosenick, L.

Hanrahan, P.

R. Ng and P. Hanrahan, “Digital correction of lens aberrations in light field photography,” Proc. SPIE 6342, 63421E (2007).
[Crossref]

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Helmchen, F.

F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2, 932–940 (2005).
[Crossref] [PubMed]

Hennelly, B.

M. Testorf, B. Hennelly, and J. Ojeda-Castañeda, Phase-Space Optics: Fundamentals and Applications (McGraw-Hill Professional, 2009).

Horowitz, M.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Hu, X.

J. Chang, I. Kauvar, X. Hu, and G. Wetzstein, “Variable aperture light field photography: Overcoming the diffraction-limited spatio-angular resolution tradeoff,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), pp. 3737–3745.

Ji, N.

N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7, 141–147 (2010).
[Crossref]

Jonas, E.

Kauvar, I.

J. Chang, I. Kauvar, X. Hu, and G. Wetzstein, “Variable aperture light field photography: Overcoming the diffraction-limited spatio-angular resolution tradeoff,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), pp. 3737–3745.

Levin, A.

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
[Crossref]

Levoy, M.

M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, and M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Express 21, 25418–25439 (2013).
[Crossref] [PubMed]

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–10.

Liang, C.

C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.

Lin, T.

C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.

Liu, C.

C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.

Liu, H.-Y.

Liu, N.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Lord, S. J.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Marks, D. L.

Matusik, W.

P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
[Crossref]

Milkie, D.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Milkie, D. E.

N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7, 141–147 (2010).
[Crossref]

Mitchell, M.

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

Moerner, W.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Moerner, W. E.

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
[Crossref] [PubMed]

Ng, R.

R. Ng and P. Hanrahan, “Digital correction of lens aberrations in light field photography,” Proc. SPIE 6342, 63421E (2007).
[Crossref]

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Ojeda-Castañeda, J.

M. Testorf, B. Hennelly, and J. Ojeda-Castañeda, Phase-Space Optics: Fundamentals and Applications (McGraw-Hill Professional, 2009).

Pavani, S. R. P.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Pégard, N. C.

Petruccelli, J. C.

Piestun, R.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Y. Y. Schechner, R. Piestun, and J. Shamir, “Wave propagation with rotating intensity distributions,” Phys. Rev. E 54, R50–R53 (1996).
[Crossref]

Planchon, T.

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Recht, B.

Sahl, S. J.

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
[Crossref] [PubMed]

Schechner, Y. Y.

Y. Y. Schechner, R. Piestun, and J. Shamir, “Wave propagation with rotating intensity distributions,” Phys. Rev. E 54, R50–R53 (1996).
[Crossref]

Segev, M.

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

Shamir, J.

Y. Y. Schechner, R. Piestun, and J. Shamir, “Wave propagation with rotating intensity distributions,” Phys. Rev. E 54, R50–R53 (1996).
[Crossref]

Sharma, K. A.

Shechtman, Y.

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
[Crossref] [PubMed]

Sheppard, C.

I. Cox, C. Sheppard, and T. Wilson, “Super-resolution by confocal fluorescent microscopy,” Optik 60, 391–396 (1982).

Situ, G.

L. Waller, G. Situ, and J. Fleischer, “Phase-space measurement and coherence synthesis of optical beams,” Nat. Photonics 6, 474–479 (2012).
[Crossref]

Stack, R. A.

Sun, W.

P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
[Crossref]

Teboulle, M.

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences 2, 183–202 (2009).
[Crossref]

Testorf, M.

M. Testorf, B. Hennelly, and J. Ojeda-Castañeda, Phase-Space Optics: Fundamentals and Applications (McGraw-Hill Professional, 2009).

Thompson, M. A.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Tian, L.

Twieg, R. J.

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Waller, L.

H.-Y. Liu, J. Zhong, and L. Waller, “4D phase-space multiplexing for fluorescent microscopy,” Proc. SPIE 9720, 97200A (2016).
[Crossref]

N. C. Pégard, H.-Y. Liu, N. Antipa, M. Gerlock, H. Adesnik, and L. Waller, “Compressive light-field microscopy for 3D neural activity recording,” Optica 3, 517–524 (2016).
[Crossref]

H.-Y. Liu, E. Jonas, L. Tian, J. Zhong, B. Recht, and L. Waller, “3D imaging in volumetric scattering media using phase-space measurements,” Opt. Express 23, 14461–14471 (2015).
[Crossref] [PubMed]

L. Waller, G. Situ, and J. Fleischer, “Phase-space measurement and coherence synthesis of optical beams,” Nat. Photonics 6, 474–479 (2012).
[Crossref]

Walther, A.

Wetzstein, G.

J. Chang, I. Kauvar, X. Hu, and G. Wetzstein, “Variable aperture light field photography: Overcoming the diffraction-limited spatio-angular resolution tradeoff,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), pp. 3737–3745.

Wilson, T.

I. Cox, C. Sheppard, and T. Wilson, “Super-resolution by confocal fluorescent microscopy,” Optik 60, 391–396 (1982).

Wong, B.

C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.

Wood, J. K.

Wornell, G.

Yang, S.

Zhang, Z.

L. Tian, Z. Zhang, J. C. Petruccelli, and G. Barbastathis, “Wigner function measurement using a lenslet array,” Opt. Express 21, 10511–10525 (2013).
[Crossref] [PubMed]

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–10.

Zhong, J.

ACM Trans. Graph. (3)

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006).
[Crossref]

A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph. 26, 70 (2007).
[Crossref]

P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graph. 26, 68 (2007).
[Crossref]

Appl. Opt. (1)

Astronomy and Astrophysics (1)

M. Bertero and P. Boccacci, “A simple method for the reduction of boundary effects in the Richardson-Lucy approach to image deconvolution,” Astronomy and Astrophysics 437, 369–374 (2005).
[Crossref]

J. Opt. Soc. Am. (1)

J. Opt. Soc. Am. A (1)

Nat. Methods (3)

N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7, 141–147 (2010).
[Crossref]

F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2, 932–940 (2005).
[Crossref] [PubMed]

T. Planchon, L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using bessel beam plane illumination,” Nat. Methods 8, 417–423 (2011).
[Crossref] [PubMed]

Nat. Photonics (1)

L. Waller, G. Situ, and J. Fleischer, “Phase-space measurement and coherence synthesis of optical beams,” Nat. Photonics 6, 474–479 (2012).
[Crossref]

Opt. Commun. (1)

M. Bastiaans, “The Wigner distribution function applied to optical signals and systems,” Opt. Commun. 25, 26–30 (1978).
[Crossref]

Opt. Express (3)

Opt. Lett. (2)

Optica (1)

Optik (1)

I. Cox, C. Sheppard, and T. Wilson, “Super-resolution by confocal fluorescent microscopy,” Optik 60, 391–396 (1982).

Phys. Rev. E (2)

Y. Y. Schechner, R. Piestun, and J. Shamir, “Wave propagation with rotating intensity distributions,” Phys. Rev. E 54, R50–R53 (1996).
[Crossref]

D. Christodoulides, E. Eugenieva, T. Coskun, M. Segev, and M. Mitchell, “Equivalence of three approaches describing partially incoherent wave propagation in inertial nonlinear media,” Phys. Rev. E 63, 35601 (2001).
[Crossref]

Phys. Rev. Lett. (1)

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113, 133902 (2014).
[Crossref] [PubMed]

Proc. Natl. Acad. Sci. USA (1)

S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. USA 106, 2995–2999 (2009).
[Crossref] [PubMed]

Proc. SPIE (2)

R. Ng and P. Hanrahan, “Digital correction of lens aberrations in light field photography,” Proc. SPIE 6342, 63421E (2007).
[Crossref]

H.-Y. Liu, J. Zhong, and L. Waller, “4D phase-space multiplexing for fluorescent microscopy,” Proc. SPIE 9720, 97200A (2016).
[Crossref]

SIAM Journal on Imaging Sciences (1)

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences 2, 183–202 (2009).
[Crossref]

Other (6)

J. Goodman, Introduction to Fourier optics (Roberts & Co., 2005).

J. Chang, I. Kauvar, X. Hu, and G. Wetzstein, “Variable aperture light field photography: Overcoming the diffraction-limited spatio-angular resolution tradeoff,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2016), pp. 3737–3745.

C. Liang, T. Lin, B. Wong, C. Liu, and H. Chen, “Programmable aperture photography: Multiplexed light field acquisition,” in Proceedings of ACM SIGGRAPH (ACM, 2008), pp. 55:1–55:10.

M. Testorf, B. Hennelly, and J. Ojeda-Castañeda, Phase-Space Optics: Fundamentals and Applications (McGraw-Hill Professional, 2009).

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–10.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Tech. Rep. CTSR 2005-02, Stanford (2005).

Supplementary Material (2)

NameDescription
» Visualization 1: MOV (1138 KB)      Raw acquisition data and masks used on the SLM
» Visualization 2: MOV (3037 KB)      3D Rotation display of the reconstructed object

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Phase-space multiplexing for 3D fluorescence microscopy. The microscope (a 4 f system) uses a Spatial Light Modulator (SLM) in Fourier space to implement multiple coded apertures, while capturing 2D intensity images in real space for each. Our wave-optical forward model A relates the object c to the measured images ymeas for each pattern. The inverse problem recovers the object, subject to sparsity priors where applicable. [See Visualization 1]

Fig. 2
Fig. 2

Illustration of how the spatial spectrum of intensity measurements are related to the input complex-field for a coded-aperture measurement. (a) Widefield imaging: the frequency representation of the electric field (u1) forms an outer product with its conjugate, resulting in a diamond-shaped Mutual Intensity (MI) space, whose rotated projection gives the spectrum of the measured image Ĩ(∆u). (b) Scanning aperture imaging: the nth aperture Mn patterns the electric field and its conjugate to probe a select area of MI space. (c) Coded-aperture imaging: the quasi-random coded apertures probe different areas within a larger range of the MI space. If is from a point source, the intensity projections as the source moves through focus make up the 3D OTF Fourier-transformed along the optical axis.

Fig. 3
Fig. 3

Multiplexed phase-space measurements contain more information than scanning-aperture measurements. (Left) A sample aperture pattern, (Middle) the corresponding measured intensity image (same exposure time), and (Right) its log-scale Fourier transform. The multiplexed measurements have better light throughput and more high-frequency content.

Fig. 4
Fig. 4

3D reconstruction of a brine shrimp sample as compared to focus stack and confocal microscopy. (a) and (b) 3D renderings of the reconstructed fluorescence intensity distribution (1010×510×500µm) from different perspectives. (c)–(e) 2D widefield images at three different focus planes. (f)–(h) Slices of our reconstructed volume at the same depth planes. (i)–(k) Confocal microscopy slices at the same depth planes for comparison [See Visualization 2.

Fig. 5
Fig. 5

Image quality can be traded for capture speed (number of coded aperture images). 3D reconstructions from increasing numbers of images with different coded apertures show that this object is too dense to be accurately reconstructed by a single coded-aperture image, but gives a reasonable reconstruction with 10 or more images, due to sparsity of the sample.

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

W ( u , r ) E ˜ * ( u Δ u 2 ) E ˜ ( u + Δ u 2 ) e i 2 π ( Δ u ) r d 2 ( Δ u ) ,
MI ( u , Δ u ) E ˜ * ( u Δ u 2 ) E ˜ ( u + Δ u 2 ) ,
E ˜ n ( u 1 ; α ) = M n ( λ f u 1 ) E ˜ s ( u 1 ; α ) ,
I n ( r ; α ) = E ˜ n ( u 1 ; α ) e i 2 π u 1 r d 2 u 1 E ˜ n * ( u 2 ; α ) e i 2 π u 2 r d 2 u 2 ,
I n ( r ; α ) = E ˜ n ( u + Δ u / 2 ; α ) E ˜ n * ( u Δ u / 2 ; α ) e i 2 π Δ u r d 2 Δ u d 2 u
= MI n ( u , Δ u ; α ) e i 2 π Δ u r d 2 Δ u d 2 u
= W n ( u , r ; α ) d 2 u ,
I ˜ n ( Δ u ; α ) = MI n ( u , Δ u ; α ) d 2 u = M n * ( λ f ( u Δ u 2 ) ) M n ( λ f ( u + Δ u 2 ) ) MI s ( u , Δ u ; α ) d 2 u ,
I n ( r ) = α C ( α ) I n ( r ; α ) .
E ˜ s ( u 1 ; r s , z s , λ ) = { e i 2 π λ ( z s ) 1 λ 2 | u 1 | 2 i 2 π r s u 1 , | u 1 | < N A λ 0 , otherwise .
I n ( r ; r s , z s , λ ) = K M n , z s ( Δ u ; λ ) e i 2 π ( r r s ) Δ u d 2 Δ u ,
K M n , z s ( Δ u ; λ ) = M n * ( λ f ( u Δ u 2 ) ) M n ( λ f ( u + Δ u 2 ) ) e i 2 π z s λ ( 1 λ 2 | u Δ u 2 | 2 1 λ 2 | u + Δ u 2 | 2 ) d 2 u
I n ( r ) = ( λ S ( λ ) K M n , z s ( Δ u ; λ ) e i 2 π ( r r s ) Δ u d 2 Δ u ) C ( r s , z s ) d 2 r s d z s .
y = A c .
arg min c 0 1 2 y y meas 2 2 + μ W c 1 ,

Metrics