Abstract

We present a fully transparent and flexible light-sensing film that, based on a single thin-film luminescent concentrator layer, supports simultaneous multi-focal image reconstruction and depth estimation without additional optics. Together with the sampling of two-dimensional light fields propagated inside the film layer under various focal conditions, it allows entire focal image stacks to be computed after only one recording that can be used for depth estimation. The transparency and flexibility of our sensor unlock the potential of lensless multilayer imaging and depth sensing with arbitrary sensor shapes – enabling novel human-computer interfaces.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
    [CrossRef] [PubMed]
  2. Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
    [CrossRef] [PubMed]
  3. A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
    [CrossRef] [PubMed]
  4. A. Koppelhuber, O. Bimber, ”Towards a transparent, flexible, scalable and disposable image sensor using thin-film luminescent concentrators,” Opt. Express 21, 4796–4810 (2013).
    [CrossRef] [PubMed]
  5. T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
    [CrossRef]
  6. D. L. Donoho, Compressed sensing, IEEE T. Inform. Theory 52(4), 1289–1306 (2006).
    [CrossRef]
  7. J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
    [CrossRef] [PubMed]
  8. J. S. Batchelder, A. H. Zewail, T. Cole, ”Luminescent solar concentrators. 1: theory of operation and techniques for performance evaluation,” Appl. Optics 18(18), 3090–3110, (1979).
    [CrossRef]
  9. A. H. Andersen, A. C. Kak, ”Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm,” Ultrasonic Imaging 6(1), 81–94, (1984).
    [PubMed]
  10. A. Pentland, ”A new sense for depth of field,” IEEE T. Pattern Anal. 9, 523–531 (1987).
    [CrossRef]
  11. S. Chaudhuri, A. N. Rajagopalan, Depth From Defocus: A Real Aperture Imaging Approach. (Springer Verlag, 1999).
  12. M. Rosenblatt, ”Remarks on some nonparametric estimates of a density function,” Ann. Math. Stat. 27, 832–837 (1956).
    [CrossRef]
  13. E. Parzen, ”On estimation of a probability density function and mode,” Ann. Math. Stat. 33, 1065 (1962).
    [CrossRef]
  14. I. Tosic, P. Frossard, ”Dictionary learning,” IEEE Signal Proc. Mag. 28, 27–38 (2011).
    [CrossRef]

2013

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

A. Koppelhuber, O. Bimber, ”Towards a transparent, flexible, scalable and disposable image sensor using thin-film luminescent concentrators,” Opt. Express 21, 4796–4810 (2013).
[CrossRef] [PubMed]

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

2011

I. Tosic, P. Frossard, ”Dictionary learning,” IEEE Signal Proc. Mag. 28, 27–38 (2011).
[CrossRef]

2008

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

2006

D. L. Donoho, Compressed sensing, IEEE T. Inform. Theory 52(4), 1289–1306 (2006).
[CrossRef]

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

1987

A. Pentland, ”A new sense for depth of field,” IEEE T. Pattern Anal. 9, 523–531 (1987).
[CrossRef]

1984

A. H. Andersen, A. C. Kak, ”Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm,” Ultrasonic Imaging 6(1), 81–94, (1984).
[PubMed]

1979

J. S. Batchelder, A. H. Zewail, T. Cole, ”Luminescent solar concentrators. 1: theory of operation and techniques for performance evaluation,” Appl. Optics 18(18), 3090–3110, (1979).
[CrossRef]

1962

E. Parzen, ”On estimation of a probability density function and mode,” Ann. Math. Stat. 33, 1065 (1962).
[CrossRef]

1956

M. Rosenblatt, ”Remarks on some nonparametric estimates of a density function,” Ann. Math. Stat. 27, 832–837 (1956).
[CrossRef]

Abouraddy, A. F.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Andersen, A. H.

A. H. Andersen, A. C. Kak, ”Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm,” Ultrasonic Imaging 6(1), 81–94, (1984).
[PubMed]

Arnold, J.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Batchelder, J. S.

J. S. Batchelder, A. H. Zewail, T. Cole, ”Luminescent solar concentrators. 1: theory of operation and techniques for performance evaluation,” Appl. Optics 18(18), 3090–3110, (1979).
[CrossRef]

Bayindir, M.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Bimber, O.

Brady, D.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Chabinyc, M. L.

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

Chaudhuri, S.

S. Chaudhuri, A. N. Rajagopalan, Depth From Defocus: A Real Aperture Imaging Approach. (Springer Verlag, 1999).

Choi, K. J.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Choi, W. M.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Cole, T.

J. S. Batchelder, A. H. Zewail, T. Cole, ”Luminescent solar concentrators. 1: theory of operation and techniques for performance evaluation,” Appl. Optics 18(18), 3090–3110, (1979).
[CrossRef]

Crozier, K. B.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Donoho, D. L.

D. L. Donoho, Compressed sensing, IEEE T. Inform. Theory 52(4), 1289–1306 (2006).
[CrossRef]

Driscoll, T.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Fink, Y.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Frossard, P.

I. Tosic, P. Frossard, ”Dictionary learning,” IEEE Signal Proc. Mag. 28, 27–38 (2011).
[CrossRef]

Geddes, J. B.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Hinczewski, D. S.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Huang, Y.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Hung, Y.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Hunt, J.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Joannopoulos, J. D.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Jung, I.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Kak, A. C.

A. H. Andersen, A. C. Kak, ”Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm,” Ultrasonic Imaging 6(1), 81–94, (1984).
[PubMed]

Kim, R.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Ko, H. C.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Koppelhuber, A.

Li, R.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Lipworth, G.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Liu, Z.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Lu, C.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Malyarchuk, V.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Mrozack, A.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Ng, T. N.

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

Park, H.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Parzen, E.

E. Parzen, ”On estimation of a probability density function and mode,” Ann. Math. Stat. 33, 1065 (1962).
[CrossRef]

Pentland, A.

A. Pentland, ”A new sense for depth of field,” IEEE T. Pattern Anal. 9, 523–531 (1987).
[CrossRef]

Rajagopalan, A. N.

S. Chaudhuri, A. N. Rajagopalan, Depth From Defocus: A Real Aperture Imaging Approach. (Springer Verlag, 1999).

Reynolds, M.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Rogers, J. A.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Rosenblatt, M.

M. Rosenblatt, ”Remarks on some nonparametric estimates of a density function,” Ann. Math. Stat. 27, 832–837 (1956).
[CrossRef]

Sambandan, S.

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

Shapira, O.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Smith, D. R.

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Song, J.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Song, Y. M.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Sorin, F.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Stoykovich, M. P.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Street, R. A.

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

Tosic, I.

I. Tosic, P. Frossard, ”Dictionary learning,” IEEE Signal Proc. Mag. 28, 27–38 (2011).
[CrossRef]

Wang, S.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Wong, W. S.

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

Xiao, J.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Xie, Y.

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Yu, C. J.

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Zewail, A. H.

J. S. Batchelder, A. H. Zewail, T. Cole, ”Luminescent solar concentrators. 1: theory of operation and techniques for performance evaluation,” Appl. Optics 18(18), 3090–3110, (1979).
[CrossRef]

Ann. Math. Stat.

M. Rosenblatt, ”Remarks on some nonparametric estimates of a density function,” Ann. Math. Stat. 27, 832–837 (1956).
[CrossRef]

E. Parzen, ”On estimation of a probability density function and mode,” Ann. Math. Stat. 33, 1065 (1962).
[CrossRef]

Appl. Optics

J. S. Batchelder, A. H. Zewail, T. Cole, ”Luminescent solar concentrators. 1: theory of operation and techniques for performance evaluation,” Appl. Optics 18(18), 3090–3110, (1979).
[CrossRef]

Appl. Phys. Lett.

T. N. Ng, W. S. Wong, M. L. Chabinyc, S. Sambandan, R. A. Street, ”Flexible image sensor array with bulk heterojunction organic photodiode,” Appl. Phys. Lett. 92(21), 213303 (2008).
[CrossRef]

IEEE Signal Proc. Mag.

I. Tosic, P. Frossard, ”Dictionary learning,” IEEE Signal Proc. Mag. 28, 27–38 (2011).
[CrossRef]

IEEE T. Inform. Theory

D. L. Donoho, Compressed sensing, IEEE T. Inform. Theory 52(4), 1289–1306 (2006).
[CrossRef]

IEEE T. Pattern Anal.

A. Pentland, ”A new sense for depth of field,” IEEE T. Pattern Anal. 9, 523–531 (1987).
[CrossRef]

Nat. Mater.

A. F. Abouraddy, O. Shapira, M. Bayindir, J. Arnold, F. Sorin, D. S. Hinczewski, J. D. Joannopoulos, Y. Fink, ”Large-scale optical-field measurements with geometric fibre constructs,” Nat. Mater. 5(7), 532–536, (2006).
[CrossRef] [PubMed]

Nature

H. C. Ko, M. P. Stoykovich, J. Song, V. Malyarchuk, W. M. Choi, C. J. Yu, J. B. Geddes, J. Xiao, S. Wang, Y. Huang, J. A. Rogers, ”A hemispherical electronic eye camera based on compressible silicon optoelectronics,” Nature 454(7205), 748–753, (2008).
[CrossRef] [PubMed]

Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. J. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Hung, J. A. Rogers, ”Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99, (2013).
[CrossRef] [PubMed]

Opt. Express

Science

J. Hunt, T. Driscoll, A. Mrozack, G. Lipworth, M. Reynolds, D. Brady, D. R. Smith, ”Metamaterial apertures for computational imaging,” Science 339, 310–313 (2013).
[CrossRef] [PubMed]

Ultrasonic Imaging

A. H. Andersen, A. C. Kak, ”Simultaneous algebraic reconstruction technique (SART): a superior implementation of the ART algorithm,” Ultrasonic Imaging 6(1), 81–94, (1984).
[PubMed]

Other

S. Chaudhuri, A. N. Rajagopalan, Depth From Defocus: A Real Aperture Imaging Approach. (Springer Verlag, 1999).

Supplementary Material (5)

» Media 1: MP4 (2553 KB)     
» Media 2: MP4 (2166 KB)     
» Media 3: MP4 (23424 KB)     
» Media 4: MP4 (1139 KB)     
» Media 5: MP4 (13460 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1
Fig. 1

Thin-film sensor principle and prototype: (a) A sheet of transparent luminescent concentrator film. (b) Illustration of the sensor’s underlying principle. (c) Prototype with LC film, optical fibers, and line scan cameras. (d) Close-up of an LC edge with apertures multiplexing transported light to a two-dimensional light field. (e) Light field of a single image point forwarded by the optical fibers. Media 1 shows the light fields of a laser point when moved on the LC surface.

Fig. 2
Fig. 2

Calibration data of basis light fields transported inside the LC film, and reconstruction results of images focussed on the LC surface: (a) Each directional scanning range of one aperture (blue box in (b)) corresponds to all values in one row of the light transport matrix T. The scanning range of all apertures are visualized in Media 2. (b) The field-of-view of an aperture is the integral of all of its scanning directions and corresponds to all values in multiple rows of T. (c) Image reconstructions with transport matrices calibrated for two different resolutions. The images were focussed on the LC film with a video projector, and high-dynamic range measurements were taken. Media 3 presents real-time reconstructions of shadows cast onto the LC surface. Achieving real-time rates limited measurements to short exposure and low dynamic range. Compared to high-dynamic-range measurements, this leads to image reconstructions with a lower S/N.

Fig. 3
Fig. 3

Multi-focal image reconstruction: (a) While T0 can only be applied for reconstructing images f0 optically focussed directly on the LC surface, images focussed at fi can be recovered with Ti. (b) Combinations of focal distances and transport matrices used for image reconstruction. In contrast to measurements, the simulations do not suffer from a low S/N and present near-ideal results. Computing Ti, i > 0, from the calibrated T0 is only marginally inferior to calibrating all transport matrices individually. The images are focused on/above the LC surface with a video projector.

Fig. 4
Fig. 4

Deconvolution vs. reconstruction with light-transport matrices: The blue frame indicates the image reconstructed directly on the LC surface (i.e., pf=0 reconstructed with T0). The red frame indicates the reconstructed image at the correct focal distance to the target object. (a) The focal stack computed by deconvolving pf=0 with the Lucy-Richardson algorithm and a Gaussian PSF of increasing σ-values. (b) The focal stack recovered by our image reconstruction method using light transport matrices Tσ computed with the same σ-values.

Fig. 5
Fig. 5

Depth estimation from shadow-defocus: (a) Experimental setup for depth estimation from defocussed shadows cast onto the LC film by differently sized target objects. To prove that depth estimation is independent of their dimensions, targets are scaled such that their shadows from various distances cover the LC equally. (b) Depth sensing geometry. (c–e) Targets reconstructed sequentially at three different distances. (f–h) Two targets reconstructed simultaneously at different distances. (d,g) Image variance plots with points of steepest descending variance-gradient (PSDVG) indicating proper focal distances. (e,h) Depth reconstruction with distance estimation imprecision. Media 4 sweeps through a focal stack showing two simultaneous targets in focus at different depths. Media 5 presents interactive depth reconstructions from shadows cast by a moving target. Due to the lower S/N, the necessary high-dynamic-range measurements limited the reconstruction speed to 0.5 frames per second. The lateral reconstruction resolution was 16 × 16 pixels. The axial reconstruction resolution was 20 σ-levels (0.3 – 0.9). Note also that the shortest target-sensor distance is constrained by the lateral reconstruction resolution. If too close, the defocus of the target cannot be resolved precisely with only 16 × 16 pixels.

Fig. 6
Fig. 6

Depth reconstruction steps: (a) Per-pixel depth map before k-means clustering. (b) Per-segment depth map after k-means clustering and before background subtraction. (c) Final depth map after background subtraction.

Fig. 7
Fig. 7

Simulated and measured variances for the experiment shown in Figs. 5(f)–5(h): (a) Simulation results for bottom target. (b) Simulation results for center target. (c) Measured results for bottom target. (d) Measured results for center target. The red dashed lines indicate the PSDVGs determined by our algorithm. The black dashed lines indicate the ground-truth PSDVGs. The small error within the simulation in (b) results from remaining image reconstruction artifacts of the SART solver.

Fig. 8
Fig. 8

Simulated and measured variances for the experiment shown in Figs. 5(c)–5(e): (a) Simulation results for bottom target. (b) Simulation results for center target. (c) Simulation results for top target. (d) Measured results for bottom target. (e) Measured results for center target. (f) Measured results for top target. The red dashed lines indicate the PSDVGs determined by our algorithm. The black dashed lines indicate the ground-truth PSDVGs.

Fig. 9
Fig. 9

High-frequency imaging with 4,6, and 11 depth layers (16 × 16, 32 × 32, and 64 × 64 simulations): (a,g,m) T0-reconstructions. (b,h,n) Ground-truth depth clusters before background subtraction with indicated σ-values. (c,i,o) Reconstructed depth clusters before background subtraction. (d,j,p) Ground-truth depths after clustering and background subtraction. (e,k,q) Reconstructed depths after clustering and background subtraction. (f,l,r) Per-pixel σ-errors after clustering and background subtraction. Overlapping blur and undetectable background makes our approach more unreliable.

Fig. 10
Fig. 10

Comparison of different imaging options: (a) In case of our experiments, the PSF of a back-lit object’s shadow is defined by the properties of the light source. (b) In case of a pin-hole camera, the PSF of a front-lit object’s image is defined by the properties of the aperture. (c) A constrained depth-of-field and field-of-view can also be achieved with thin-film imaging optics, such as an additional layer of spaced microlouvers.

Fig. 11
Fig. 11

Optical design for the multiplexing of light being transported inside the LC to a two-dimensional light field l(x, ϕ): Triangles are cut into the borders of the LC foil and filled with light-blocking plasticine. The gaps between these triangles act as apertures that project the passing light in a controlled way onto the LC edges, by constraining the field-of-view per triangle and the integration direction per edge position within each triangle. A diffusor between the LC edge and the optical fibers couples the multiplexed light signal into the acceptance cone of the optical fibers that propagate it to the photo sensors. The LC edges and the ends of the fiber ribbons are protected with opaque, flexible acrylic covers.

Fig. 12
Fig. 12

Numerical stability of image reconstruction: The condition number κ for simulated transport matrices T computed for an increasing σ and various reconstruction resolutions.

Equations (18)

Equations on this page are rendered with MathJax. Learn more.

r l ( d l d p ) = r b ( σ ) d p ,
d p = r b ( σ ) d l r l + r b ( σ ) .
r b ( σ ) = s σ .
d p = σ d l ( 1 s r l + σ ) .
d l = d p 1 d p 2 ( σ 1 σ 2 ) d p 2 σ 1 d p 1 σ 2 ,
1 s r l = σ 1 ( d l d p 1 ) d p 1 .
v ( x , y , z ) = 1 ( 2 w + 1 ) 2 i w + w j w + w ( S ( x + i , y + j , z ) μ ( x , y , z ) ) 2 ,
μ ( x , y , z ) = 1 ( 2 w + 1 ) 2 i w + w j w + w S ( x + i , y + j , z ) .
α = 2 tan 1 ( r l d l ) .
β = 2 tan 1 ( r s + r l d l ) .
d c = r s tan ( β / 2 ) .
r b ( σ ) = r l d p d l d p = r a d p d p d a .
α = 2 tan 1 ( r a d a ) .
β = 2 tan 1 ( r s + r a d a ) .
α = 2 tan 1 ( s m 2 h m ) .
β = 2 tan 1 ( s m h m ) ,
β = 2 tan 1 ( 2 tan ( α 2 ) ) ,
s m h m = tan ( β 2 ) = 2 tan ( α 2 ) .

Metrics