Abstract

This paper deals with the fast-developing area of computational photography where a combination of imaging techniques and efficient image processing algorithms is done to generate a super imaging system. In recent years, three main implementations of the computational photography philosophy were intensively investigated and demonstrated: (i) multiple aperture, (ii) light field photography, and (iii) multiexposure. The paper provides a mini-review of these three approaches and shows ways to improve and combine them toward a super imaging system. As a conclusion, the paper provides preliminary guidelines for fusing the three approaches into one integrated super imaging system with optimized performance.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Kidjer, “Principles of lens design,” Proc. SPIE CR4, 30–52 (1992).
  2. J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
    [CrossRef]
  3. A. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, 1989).
  4. E. Dowski and T. Chathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34, 1859–1866 (1995).
    [CrossRef]
  5. K. H. Brenner, A. W. Lohmann, and J. Ojeda-Castaneda, “The ambiguity function as polar display of OTF,” Opt. Commun. 44, 323–326 (1983).
    [CrossRef]
  6. T. Chathey, E. R. Dowski, and A. R. Fitz Gerrrell, “Optical/digital aberration control in incoherent optical system,” Proc. SPIE 2730, 120–126 (1996).
    [CrossRef]
  7. J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
    [CrossRef]
  8. Y. Kitamura, “Reconstruction of a high resolution image on a compound eye image capturing system,” Appl. Opt. 43, 1719–1727, (2004).
    [CrossRef]
  9. K. Nitta and J. Tanida, “Image reconstruction for thin observation module by bound optics by using the iterative backprojection method,” Appl. Opt. 45, 2893–2900 (2006).
    [CrossRef]
  10. J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes—topical review,” Bioinspir. Biomim. 1, R1–R16 (2006).
  11. J. W. Duparré, P. Dannberg, P. Schreiber, A. Bräuer, P. Nussbaum, F. Heitger, and A. Tünnermann, “Ultra thin camera based on artificial apposition compound eyes,” http://www.suss-microoptics.com/media/downloads/publications/MOC-04_Duparre.pdf .
  12. D. Capel and A. Zisserman, “Computer vision applied to super-resolution,” IEEE Signal Process. Mag. 20(3), 75–86 (2003).
    [CrossRef]
  13. A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
    [CrossRef]
  14. M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
    [CrossRef]
  15. J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
    [CrossRef]
  16. P. Jorge and S. Ferreira, “Interpolation and the discrete Papoulis-Gerchberg algorithm,” IEEE Trans. Signal Process. 42, 2596–2606 (1994).
    [CrossRef]
  17. S. Mendelowitz, “Image reconstruction algorithm designed for the TOMBO system,” M. Sc. Thesis (Tel Aviv University, 2011).
  18. F. N. Lanchester, English Patent No. 16548/95 (1895).
  19. R. E. Liesegang, British J. Photography 43, 569 (1896).
  20. J. A. C. Branfill, British J. Photography 44, 142 (1897).
  21. P. E. Ives, “Parallax stereogram and process of making same,” English Patent No. 725567 (14April1903).
  22. E. H. Adelson and Y. A. Wang, “Single lens stereo with plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
    [CrossRef]
  23. R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).
  24. R. Ng, “Digital light field photography,” Ph.D. dissertation (Stanford University, 2006).
  25. R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
    [CrossRef]
  26. Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–9.
  27. A. Davis, M. Levoy, and F. Durand, “Unstructured light fields,” Comput. Graph. Forum 31, 305–314 (2012).
    [CrossRef]
  28. “Light field based commercial digital still camera,” http://www.lytro.com .
  29. R. Szeliski, Image Alignment and Stitching: A Tutorial (Now Publishers, 2006).
  30. M. Robinson and D. Stork, “Joint digital-optical design of super resolution multi frame imaging systems,” Appl. Opt. 47, B11–B20 (2008).
    [CrossRef]
  31. R. Raskar, “Coded exposure photography: motion deblurring using fluttered shutter,” in SIGGRAPH 2006 Conference Proceedings (ACM, 2006), pp. 795–804.
  32. I. Gur and D. Mendlovic, “Diffraction limited top-hat generator,” Opt. Commun. 145, 237–248 (1998).
    [CrossRef]
  33. U. Levy, E. Marom, and D. Mendlovic, “Simultaneous multicolor image formation with a single diffractive optical element,” Opt. Lett. 26, 1149–1151 (2001).
    [CrossRef]
  34. U. Levy, E. Marom, and D. Mendlovic, “Thin element approximation for the analysis of blazed grating: simplified model and validity limits,” Opt. Commun. 229, 11–21 (2004).
    [CrossRef]
  35. R. Horstmeyer, G. Euliss, R. Athale, and M. Levoy, “Flexible multimodal camera using a light field architecture,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.
  36. Z. Zalevsky and D. Mendlovic, Optical Superresolution (Springer-Verlag, 2003), pp. 183–194.

2012

A. Davis, M. Levoy, and F. Durand, “Unstructured light fields,” Comput. Graph. Forum 31, 305–314 (2012).
[CrossRef]

2008

M. Robinson and D. Stork, “Joint digital-optical design of super resolution multi frame imaging systems,” Appl. Opt. 47, B11–B20 (2008).
[CrossRef]

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
[CrossRef]

2007

A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
[CrossRef]

2006

K. Nitta and J. Tanida, “Image reconstruction for thin observation module by bound optics by using the iterative backprojection method,” Appl. Opt. 45, 2893–2900 (2006).
[CrossRef]

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes—topical review,” Bioinspir. Biomim. 1, R1–R16 (2006).

2004

Y. Kitamura, “Reconstruction of a high resolution image on a compound eye image capturing system,” Appl. Opt. 43, 1719–1727, (2004).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Thin element approximation for the analysis of blazed grating: simplified model and validity limits,” Opt. Commun. 229, 11–21 (2004).
[CrossRef]

2003

D. Capel and A. Zisserman, “Computer vision applied to super-resolution,” IEEE Signal Process. Mag. 20(3), 75–86 (2003).
[CrossRef]

J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
[CrossRef]

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

2001

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Simultaneous multicolor image formation with a single diffractive optical element,” Opt. Lett. 26, 1149–1151 (2001).
[CrossRef]

1998

I. Gur and D. Mendlovic, “Diffraction limited top-hat generator,” Opt. Commun. 145, 237–248 (1998).
[CrossRef]

1996

T. Chathey, E. R. Dowski, and A. R. Fitz Gerrrell, “Optical/digital aberration control in incoherent optical system,” Proc. SPIE 2730, 120–126 (1996).
[CrossRef]

1995

E. Dowski and T. Chathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34, 1859–1866 (1995).
[CrossRef]

1994

P. Jorge and S. Ferreira, “Interpolation and the discrete Papoulis-Gerchberg algorithm,” IEEE Trans. Signal Process. 42, 2596–2606 (1994).
[CrossRef]

1992

E. H. Adelson and Y. A. Wang, “Single lens stereo with plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

M. Kidjer, “Principles of lens design,” Proc. SPIE CR4, 30–52 (1992).

1983

K. H. Brenner, A. W. Lohmann, and J. Ojeda-Castaneda, “The ambiguity function as polar display of OTF,” Opt. Commun. 44, 323–326 (1983).
[CrossRef]

1897

J. A. C. Branfill, British J. Photography 44, 142 (1897).

1896

R. E. Liesegang, British J. Photography 43, 569 (1896).

Ackerman, J.

A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
[CrossRef]

Adelson, E. H.

E. H. Adelson and Y. A. Wang, “Single lens stereo with plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

Agrawal, A.

R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
[CrossRef]

Athale, R.

J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
[CrossRef]

R. Horstmeyer, G. Euliss, R. Athale, and M. Levoy, “Flexible multimodal camera using a light field architecture,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.

Brady, D.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Branfill, J. A. C.

J. A. C. Branfill, British J. Photography 44, 142 (1897).

Bredif, M.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

Brenner, K. H.

K. H. Brenner, A. W. Lohmann, and J. Ojeda-Castaneda, “The ambiguity function as polar display of OTF,” Opt. Commun. 44, 323–326 (1983).
[CrossRef]

Capel, D.

D. Capel and A. Zisserman, “Computer vision applied to super-resolution,” IEEE Signal Process. Mag. 20(3), 75–86 (2003).
[CrossRef]

Carrier, J.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Chathey, T.

T. Chathey, E. R. Dowski, and A. R. Fitz Gerrrell, “Optical/digital aberration control in incoherent optical system,” Proc. SPIE 2730, 120–126 (1996).
[CrossRef]

E. Dowski and T. Chathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34, 1859–1866 (1995).
[CrossRef]

Chen, C.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Davis, A.

A. Davis, M. Levoy, and F. Durand, “Unstructured light fields,” Comput. Graph. Forum 31, 305–314 (2012).
[CrossRef]

Dowski, E.

E. Dowski and T. Chathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34, 1859–1866 (1995).
[CrossRef]

Dowski, E. R.

T. Chathey, E. R. Dowski, and A. R. Fitz Gerrrell, “Optical/digital aberration control in incoherent optical system,” Proc. SPIE 2730, 120–126 (1996).
[CrossRef]

Duparré, J. W.

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes—topical review,” Bioinspir. Biomim. 1, R1–R16 (2006).

Durand, F.

A. Davis, M. Levoy, and F. Durand, “Unstructured light fields,” Comput. Graph. Forum 31, 305–314 (2012).
[CrossRef]

Duval, G.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

Euliss, G.

R. Horstmeyer, G. Euliss, R. Athale, and M. Levoy, “Flexible multimodal camera using a light field architecture,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.

Ferreira, S.

P. Jorge and S. Ferreira, “Interpolation and the discrete Papoulis-Gerchberg algorithm,” IEEE Trans. Signal Process. 42, 2596–2606 (1994).
[CrossRef]

Fitz Gerrrell, A. R.

T. Chathey, E. R. Dowski, and A. R. Fitz Gerrrell, “Optical/digital aberration control in incoherent optical system,” Proc. SPIE 2730, 120–126 (1996).
[CrossRef]

Fleet, E.

A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
[CrossRef]

Gibbons, R.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Gur, I.

I. Gur and D. Mendlovic, “Diffraction limited top-hat generator,” Opt. Commun. 145, 237–248 (1998).
[CrossRef]

Hanaran, P.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

Horowitz, M.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

Horstmeyer, R.

R. Horstmeyer, G. Euliss, R. Athale, and M. Levoy, “Flexible multimodal camera using a light field architecture,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.

Ichioka, Y.

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Ishida, K.

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Ives, P. E.

P. E. Ives, “Parallax stereogram and process of making same,” English Patent No. 725567 (14April1903).

Jain, A.

A. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, 1989).

Jorge, P.

P. Jorge and S. Ferreira, “Interpolation and the discrete Papoulis-Gerchberg algorithm,” IEEE Trans. Signal Process. 42, 2596–2606 (1994).
[CrossRef]

Kanaev, A.

A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
[CrossRef]

Kidjer, M.

M. Kidjer, “Principles of lens design,” Proc. SPIE CR4, 30–52 (1992).

Kitamura, Y.

Y. Kitamura, “Reconstruction of a high resolution image on a compound eye image capturing system,” Appl. Opt. 43, 1719–1727, (2004).
[CrossRef]

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

Kolste, R. T.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Kondou, N.

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Lanchester, F. N.

F. N. Lanchester, English Patent No. 16548/95 (1895).

Levoy, M.

A. Davis, M. Levoy, and F. Durand, “Unstructured light fields,” Comput. Graph. Forum 31, 305–314 (2012).
[CrossRef]

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

R. Horstmeyer, G. Euliss, R. Athale, and M. Levoy, “Flexible multimodal camera using a light field architecture,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–9.

Levy, U.

U. Levy, E. Marom, and D. Mendlovic, “Thin element approximation for the analysis of blazed grating: simplified model and validity limits,” Opt. Commun. 229, 11–21 (2004).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Simultaneous multicolor image formation with a single diffractive optical element,” Opt. Lett. 26, 1149–1151 (2001).
[CrossRef]

Liesegang, R. E.

R. E. Liesegang, British J. Photography 43, 569 (1896).

Lohmann, A. W.

K. H. Brenner, A. W. Lohmann, and J. Ojeda-Castaneda, “The ambiguity function as polar display of OTF,” Opt. Commun. 44, 323–326 (1983).
[CrossRef]

Mait, J.

J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
[CrossRef]

Marom, E.

U. Levy, E. Marom, and D. Mendlovic, “Thin element approximation for the analysis of blazed grating: simplified model and validity limits,” Opt. Commun. 229, 11–21 (2004).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Simultaneous multicolor image formation with a single diffractive optical element,” Opt. Lett. 26, 1149–1151 (2001).
[CrossRef]

Mendelowitz, S.

S. Mendelowitz, “Image reconstruction algorithm designed for the TOMBO system,” M. Sc. Thesis (Tel Aviv University, 2011).

Mendlovic, D.

U. Levy, E. Marom, and D. Mendlovic, “Thin element approximation for the analysis of blazed grating: simplified model and validity limits,” Opt. Commun. 229, 11–21 (2004).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Simultaneous multicolor image formation with a single diffractive optical element,” Opt. Lett. 26, 1149–1151 (2001).
[CrossRef]

I. Gur and D. Mendlovic, “Diffraction limited top-hat generator,” Opt. Commun. 145, 237–248 (1998).
[CrossRef]

Z. Zalevsky and D. Mendlovic, Optical Superresolution (Springer-Verlag, 2003), pp. 183–194.

Miyamoto, M.

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

Miyatake, S.

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Miyazaki, D.

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Morimoto, T.

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Ng, R.

R. Ng, “Digital light field photography,” Ph.D. dissertation (Stanford University, 2006).

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

Nitta, K.

K. Nitta and J. Tanida, “Image reconstruction for thin observation module by bound optics by using the iterative backprojection method,” Appl. Opt. 45, 2893–2900 (2006).
[CrossRef]

Ojeda-Castaneda, J.

K. H. Brenner, A. W. Lohmann, and J. Ojeda-Castaneda, “The ambiguity function as polar display of OTF,” Opt. Commun. 44, 323–326 (1983).
[CrossRef]

Pitslanis, N.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Prather, D.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Raskar, R.

R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
[CrossRef]

R. Raskar, “Coded exposure photography: motion deblurring using fluttered shutter,” in SIGGRAPH 2006 Conference Proceedings (ACM, 2006), pp. 795–804.

Robinson, M.

M. Robinson and D. Stork, “Joint digital-optical design of super resolution multi frame imaging systems,” Appl. Opt. 47, B11–B20 (2008).
[CrossRef]

Schulz, T.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Scribner, D.

A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
[CrossRef]

Shankar, M.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Shogenji, R.

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

Stork, D.

M. Robinson and D. Stork, “Joint digital-optical design of super resolution multi frame imaging systems,” Appl. Opt. 47, B11–B20 (2008).
[CrossRef]

Szeliski, R.

R. Szeliski, Image Alignment and Stitching: A Tutorial (Now Publishers, 2006).

Tanida, J.

K. Nitta and J. Tanida, “Image reconstruction for thin observation module by bound optics by using the iterative backprojection method,” Appl. Opt. 45, 2893–2900 (2006).
[CrossRef]

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

van der Gracht, J.

J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
[CrossRef]

Veeraraghavan, A.

R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
[CrossRef]

Wang, Y. A.

E. H. Adelson and Y. A. Wang, “Single lens stereo with plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

Willet, R.

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Wilson, C.

R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
[CrossRef]

Wippermann, F. C.

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes—topical review,” Bioinspir. Biomim. 1, R1–R16 (2006).

Yamada, K.

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Zalevsky, Z.

Z. Zalevsky and D. Mendlovic, Optical Superresolution (Springer-Verlag, 2003), pp. 183–194.

Zhang, Z.

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–9.

Zisserman, A.

D. Capel and A. Zisserman, “Computer vision applied to super-resolution,” IEEE Signal Process. Mag. 20(3), 75–86 (2003).
[CrossRef]

ACM Trans. Graph.

R. Raskar, A. Agrawal, C. Wilson, and A. Veeraraghavan, “Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses.” ACM Trans. Graph. 27, 56 (2008).
[CrossRef]

Appl. Opt.

M. Robinson and D. Stork, “Joint digital-optical design of super resolution multi frame imaging systems,” Appl. Opt. 47, B11–B20 (2008).
[CrossRef]

E. Dowski and T. Chathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34, 1859–1866 (1995).
[CrossRef]

J. Tanida, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40, 1806–1813, (2001).
[CrossRef]

Y. Kitamura, “Reconstruction of a high resolution image on a compound eye image capturing system,” Appl. Opt. 43, 1719–1727, (2004).
[CrossRef]

K. Nitta and J. Tanida, “Image reconstruction for thin observation module by bound optics by using the iterative backprojection method,” Appl. Opt. 45, 2893–2900 (2006).
[CrossRef]

M. Shankar, R. Willet, N. Pitslanis, T. Schulz, R. Gibbons, R. T. Kolste, J. Carrier, C. Chen, D. Prather, and D. Brady, “Thin infrared imaging systems through multichannel sampling,” Appl. Opt. 47, B1–B10 (2008).
[CrossRef]

Bioinspir. Biomim.

J. W. Duparré and F. C. Wippermann, “Micro-optical artificial compound eyes—topical review,” Bioinspir. Biomim. 1, R1–R16 (2006).

British J. Photography

R. E. Liesegang, British J. Photography 43, 569 (1896).

J. A. C. Branfill, British J. Photography 44, 142 (1897).

Comput. Graph. Forum

A. Davis, M. Levoy, and F. Durand, “Unstructured light fields,” Comput. Graph. Forum 31, 305–314 (2012).
[CrossRef]

IEEE Signal Process. Mag.

D. Capel and A. Zisserman, “Computer vision applied to super-resolution,” IEEE Signal Process. Mag. 20(3), 75–86 (2003).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

E. H. Adelson and Y. A. Wang, “Single lens stereo with plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

IEEE Trans. Signal Process.

P. Jorge and S. Ferreira, “Interpolation and the discrete Papoulis-Gerchberg algorithm,” IEEE Trans. Signal Process. 42, 2596–2606 (1994).
[CrossRef]

Opt. Commun.

K. H. Brenner, A. W. Lohmann, and J. Ojeda-Castaneda, “The ambiguity function as polar display of OTF,” Opt. Commun. 44, 323–326 (1983).
[CrossRef]

I. Gur and D. Mendlovic, “Diffraction limited top-hat generator,” Opt. Commun. 145, 237–248 (1998).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Thin element approximation for the analysis of blazed grating: simplified model and validity limits,” Opt. Commun. 229, 11–21 (2004).
[CrossRef]

Opt. Express

J. Mait, R. Athale, and J. van der Gracht, “Evolutionary paths in imaging and recent trends,” Opt. Express 11, 2093–2101 (2003).
[CrossRef]

J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11, 2109–2117, (2003).
[CrossRef]

Opt. Lett.

A. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “TOMBO sensor with scene-independent superresolution processing,” Opt. Lett. 32, 2855–2857 (2007).
[CrossRef]

U. Levy, E. Marom, and D. Mendlovic, “Simultaneous multicolor image formation with a single diffractive optical element,” Opt. Lett. 26, 1149–1151 (2001).
[CrossRef]

Proc. SPIE

M. Kidjer, “Principles of lens design,” Proc. SPIE CR4, 30–52 (1992).

T. Chathey, E. R. Dowski, and A. R. Fitz Gerrrell, “Optical/digital aberration control in incoherent optical system,” Proc. SPIE 2730, 120–126 (1996).
[CrossRef]

Other

A. Jain, Fundamentals of Digital Image Processing (Prentice-Hall, 1989).

J. W. Duparré, P. Dannberg, P. Schreiber, A. Bräuer, P. Nussbaum, F. Heitger, and A. Tünnermann, “Ultra thin camera based on artificial apposition compound eyes,” http://www.suss-microoptics.com/media/downloads/publications/MOC-04_Duparre.pdf .

S. Mendelowitz, “Image reconstruction algorithm designed for the TOMBO system,” M. Sc. Thesis (Tel Aviv University, 2011).

F. N. Lanchester, English Patent No. 16548/95 (1895).

P. E. Ives, “Parallax stereogram and process of making same,” English Patent No. 725567 (14April1903).

R. Horstmeyer, G. Euliss, R. Athale, and M. Levoy, “Flexible multimodal camera using a light field architecture,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–8.

Z. Zalevsky and D. Mendlovic, Optical Superresolution (Springer-Verlag, 2003), pp. 183–194.

R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanaran, “Light field photography with a hand-held plenoptic camera,” Stanford Tech Report CTSR 2005-02, (Stanford University, 2005).

R. Ng, “Digital light field photography,” Ph.D. dissertation (Stanford University, 2006).

Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in Proceedings of the IEEE International Conference on Computational Photography (IEEE, 2009), pp. 1–9.

“Light field based commercial digital still camera,” http://www.lytro.com .

R. Szeliski, Image Alignment and Stitching: A Tutorial (Now Publishers, 2006).

R. Raskar, “Coded exposure photography: motion deblurring using fluttered shutter,” in SIGGRAPH 2006 Conference Proceedings (ACM, 2006), pp. 795–804.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1.
Fig. 1.

Block diagram for a hybrid image acquisition system compared with a conventional one.

Fig. 2.
Fig. 2.

Compound optics generic configuration: (A) is the microlens array, (B) is the rectangular aperture channel array, (C) is a specific “subchannel,” and (D) is the detection imager.

Fig. 3.
Fig. 3.

“Plenoptic function:” (a) general 5D case and (b) free-space–two-plane 4D case.

Fig. 4.
Fig. 4.

Creation of the 4D “plenoptic function.” Conventional imaging system (upper setup) vs. light field imaging system (lower setup).

Fig. 5.
Fig. 5.

Illustration of the multiexposure concept of fusing few exposures to create better quality image.

Fig. 6.
Fig. 6.

Activities toward a super imaging camera.

Fig. 7.
Fig. 7.

Schematic construction of the super imaging camera.

Fig. 8.
Fig. 8.

Simulations of three different images using Tanida’s method versus the method in [17]. Left to right: pixel rearrangement (Tanida’s), fast fusion [17], LMS-IBP (Tanida’s), and IBP with MMTV regulator [17].

Fig. 9.
Fig. 9.

Light field camera (left side). The conventional arrangement of a single lens out of the lenslet array (upper scheme) and a proposed DOE-based configuration (lower scheme).

Fig. 10.
Fig. 10.

Hyperspace of the integrated camera. The vertical axis is the number of apertures and the other two are the numbers of fields and exposures. An indication of the three-parameter working range is provided.

Metrics