Abstract

Abstract: Integral microscopes (IMic) have been recently developed in order to capture the spatial and the angular information of 3D microscopic samples with a single exposure. Computational post-processing of this information permits to carry out a 3D reconstruction of the sample. By applying conventional algorithms, both depth and also view reconstructions are possible. However, the main drawback of IMic is that the resolution of the reconstructed images is low and axially heterogeneous. In this paper, we propose a new configuration of the IMic by placing the lens array not at the image plane, but at the pupil (or Fourier) plane of the microscope objective. With this novel system, the spatial resolution is increased by factor 1.4, and the depth of field is substantially enlarged. Our experiments show the feasibility of the proposed method.

© 2016 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Resolution enhancement in integral microscopy by physical interpolation

Anabel Llavador, Emilio Sánchez-Ortiga, Juan Carlos Barreiro, Genaro Saavedra, and Manuel Martínez-Corral
Biomed. Opt. Express 6(8) 2854-2863 (2015)

Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems

Manuel Martínez-Corral and Bahram Javidi
Adv. Opt. Photon. 10(3) 512-566 (2018)

FIMic: design for ultimate 3D-integral microscopy of in-vivo biological samples

G. Scrofani, J. Sola-Pikabea, A. Llavador, E. Sanchez-Ortiga, J. C. Barreiro, G. Saavedra, J. Garcia-Sucerquia, and M. Martínez-Corral
Biomed. Opt. Express 9(1) 335-346 (2018)

References

  • View by:
  • |
  • |
  • |

  1. J. Huisken and D. Y. R. Stainier, “Selective plane illumination microscopy techniques in developmental biology,” Development 136(12), 1963–1975 (2009).
    [Crossref] [PubMed]
  2. E. H. K. Stelzer, K. Greger, and E. G. Reynaud, Light Sheet Based Fluorescence Microscopy: Principles and Practice (Wiley-Blackwell, 2014).
  3. M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997).
    [Crossref] [PubMed]
  4. J. B. Pawley, Handbook of Biological Confocal Microscopy, 3rd ed. (Plenum, 2006).
  5. E. Sánchez-Ortiga, A. Doblas, G. Saavedra, M. Martínez-Corral, and J. Garcia-Sucerquia, “Off-axis digital holographic microscopy: practical design parameters for operating at diffraction limit,” Appl. Opt. 53(10), 2058–2066 (2014).
    [Crossref] [PubMed]
  6. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).
  7. H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. 21(3), 171–176 (1931).
    [Crossref]
  8. H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001).
    [Crossref] [PubMed]
  9. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12(3), 483–491 (2004).
    [Crossref] [PubMed]
  10. Y. S. Hwang and E. S. Kim, “Perspective-variant reconstruction of a three-dimensional object along the depth direction with virtually generated elemental images by ray-based pixel mapping in integral-imaging,” Opt. Lasers Eng. 51(7), 797–807 (2013).
    [Crossref]
  11. S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12(19), 4579–4588 (2004).
    [Crossref] [PubMed]
  12. M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
    [Crossref]
  13. D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
    [Crossref]
  14. J.-Y. Jang, J. I. Ser, S. Cha, and S. H. Shin, “Depth extraction by using the correlation of the periodic function with an elemental image in integral imaging,” Appl. Opt. 51(16), 3279–3286 (2012).
    [Crossref] [PubMed]
  15. J. Y. Jang, D. Shin, B. G. Lee, S. P. Hong, and E. S. Kim, “3D image correlator using computational integral imaging reconstruction based on modified convolution property of periodic functions,” J. Opt. Soc. Korea 18(4), 388–394 (2014).
    [Crossref]
  16. A. Llavador, E. Sánchez-Ortiga, G. Saavedra, B. Javidi, and M. Martínez-Corral, “Free-depths reconstruction with synthetic impulse response in integral imaging,” Opt. Express 23(23), 30127–30135 (2015).
    [Crossref] [PubMed]
  17. J. S. Jang and B. Javidi, “Three-dimensional integral imaging of micro-objects,” Opt. Lett. 29(11), 1230–1232 (2004).
    [Crossref] [PubMed]
  18. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
    [Crossref]
  19. Y. T. Lim, J. H. Park, K. Ch. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009).
    [Crossref] [PubMed]
  20. K.-Ch. Kwon, J.-S. Jeong, M.-U. Erdenebat, Y.-L. Piao, K.-H. Yoo, and N. Kim, “Resolution-enhancement for an orthographic-view image display in an integral imaging microscope system,” Biomed. Opt. Express 6(3), 736–746 (2015).
    [Crossref] [PubMed]
  21. A. Llavador, E. Sánchez-Ortiga, J. C. Barreiro, G. Saavedra, and M. Martínez-Corral, “Resolution enhancement in integral microscopy by physical interpolation,” Biomed. Opt. Express 6(8), 2854–2863 (2015).
    [Crossref] [PubMed]
  22. M. Martinez-Corral, G. Saavedra, E. Sánchez-Ortiga, A. Llavador, and J. Sola-Pikabea, “Microscopio integral, usos del mismo y sistema de microscopía integral,” Spanish patent P201531938.
  23. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Tech. Rep. CSTR. 2 (2005).
  24. H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
    [Crossref]
  25. M. Martinez-Corral, A. Dorado, A. Llavador, G. Saavedra, and B. Javidi, “Three-dimensional integral imaging and display” in Multi-Dimensional Imaging, B. Javidi, E. Tajahuerce, and P. Andres, eds. (John Willey and Sons, 2014), Chap. 11.
  26. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002).
    [Crossref] [PubMed]
  27. B. Javidi, I. Moon, and S. Yeom, “Three-dimensional identification of biological microorganism using integral imaging,” Opt. Express 14(25), 12096–12108 (2006).
    [Crossref] [PubMed]

2015 (3)

2014 (2)

2013 (2)

Y. S. Hwang and E. S. Kim, “Perspective-variant reconstruction of a three-dimensional object along the depth direction with virtually generated elemental images by ray-based pixel mapping in integral-imaging,” Opt. Lasers Eng. 51(7), 797–807 (2013).
[Crossref]

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

2012 (1)

2009 (4)

J. Huisken and D. Y. R. Stainier, “Selective plane illumination microscopy techniques in developmental biology,” Development 136(12), 1963–1975 (2009).
[Crossref] [PubMed]

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Y. T. Lim, J. H. Park, K. Ch. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009).
[Crossref] [PubMed]

2006 (2)

2004 (3)

2002 (1)

2001 (1)

1997 (1)

1931 (1)

1908 (1)

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Adams, A.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
[Crossref]

Arimoto, H.

Barreiro, J. C.

Cha, S.

Cho, M.

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

Doblas, A.

Dorado, A.

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

Erdenebat, M.-U.

Footer, M.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
[Crossref]

Garcia-Sucerquia, J.

Hong, S. P.

Hong, S.-H.

Horowitz, M.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
[Crossref]

Huisken, J.

J. Huisken and D. Y. R. Stainier, “Selective plane illumination microscopy techniques in developmental biology,” Development 136(12), 1963–1975 (2009).
[Crossref] [PubMed]

Hwang, Y. S.

Y. S. Hwang and E. S. Kim, “Perspective-variant reconstruction of a three-dimensional object along the depth direction with virtually generated elemental images by ray-based pixel mapping in integral-imaging,” Opt. Lasers Eng. 51(7), 797–807 (2013).
[Crossref]

Ives, H. E.

Jang, J. S.

Jang, J. Y.

Jang, J.-S.

Jang, J.-Y.

Javidi, B.

A. Llavador, E. Sánchez-Ortiga, G. Saavedra, B. Javidi, and M. Martínez-Corral, “Free-depths reconstruction with synthetic impulse response in integral imaging,” Opt. Express 23(23), 30127–30135 (2015).
[Crossref] [PubMed]

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

B. Javidi, I. Moon, and S. Yeom, “Three-dimensional identification of biological microorganism using integral imaging,” Opt. Express 14(25), 12096–12108 (2006).
[Crossref] [PubMed]

S.-H. Hong and B. Javidi, “Improved resolution 3D object reconstruction using computational integral imaging with time multiplexing,” Opt. Express 12(19), 4579–4588 (2004).
[Crossref] [PubMed]

S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12(3), 483–491 (2004).
[Crossref] [PubMed]

J. S. Jang and B. Javidi, “Three-dimensional integral imaging of micro-objects,” Opt. Lett. 29(11), 1230–1232 (2004).
[Crossref] [PubMed]

J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27(13), 1144–1146 (2002).
[Crossref] [PubMed]

H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001).
[Crossref] [PubMed]

Jeong, J.-S.

Juskaitis, R.

Kim, E. S.

J. Y. Jang, D. Shin, B. G. Lee, S. P. Hong, and E. S. Kim, “3D image correlator using computational integral imaging reconstruction based on modified convolution property of periodic functions,” J. Opt. Soc. Korea 18(4), 388–394 (2014).
[Crossref]

Y. S. Hwang and E. S. Kim, “Perspective-variant reconstruction of a three-dimensional object along the depth direction with virtually generated elemental images by ray-based pixel mapping in integral-imaging,” Opt. Lasers Eng. 51(7), 797–807 (2013).
[Crossref]

Kim, N.

Kwon, K. Ch.

Kwon, K.-Ch.

Lee, B. G.

Levoy, M.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
[Crossref]

Lim, Y. T.

Lippmann, G.

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Llavador, A.

Martinez-Corral, M.

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

Martínez-Corral, M.

Moon, I.

Navarro, H.

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

Neil, M. A. A.

Ng, R.

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
[Crossref]

Park, J. H.

Piao, Y.-L.

Saavedra, G.

Sánchez-Ortiga, E.

Ser, J. I.

Shin, D.

Shin, D. H.

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Shin, S. H.

Stainier, D. Y. R.

J. Huisken and D. Y. R. Stainier, “Selective plane illumination microscopy techniques in developmental biology,” Development 136(12), 1963–1975 (2009).
[Crossref] [PubMed]

Wilson, T.

Yeom, S.

Yoo, H.

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Yoo, K.-H.

ACM Trans. Graph. (1)

M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006).
[Crossref]

Appl. Opt. (2)

Biomed. Opt. Express (2)

Development (1)

J. Huisken and D. Y. R. Stainier, “Selective plane illumination microscopy techniques in developmental biology,” Development 136(12), 1963–1975 (2009).
[Crossref] [PubMed]

J. Disp. Technol. (2)

M. Cho and B. Javidi, “Computational reconstruction of three-dimensional integral imaging by rearrangement of elemental image pixels,” J. Disp. Technol. 5(2), 61–65 (2009).
[Crossref]

H. Navarro, E. Sánchez-Ortiga, G. Saavedra, A. Llavador, A. Dorado, M. Martinez-Corral, and B. Javidi, “Non-homogeneity of lateral resolution in integral imaging,” J. Disp. Technol. 9(1), 37–43 (2013).
[Crossref]

J. Opt. Soc. Am. (1)

J. Opt. Soc. Korea (1)

J. Phys. (1)

G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908).

Opt. Commun. (1)

D. H. Shin and H. Yoo, “Computational integral imaging reconstruction method of 3D images using pixel-to-pixel mapping and image interpolation,” Opt. Commun. 282(14), 2760–2767 (2009).
[Crossref]

Opt. Express (5)

Opt. Lasers Eng. (1)

Y. S. Hwang and E. S. Kim, “Perspective-variant reconstruction of a three-dimensional object along the depth direction with virtually generated elemental images by ray-based pixel mapping in integral-imaging,” Opt. Lasers Eng. 51(7), 797–807 (2013).
[Crossref]

Opt. Lett. (4)

Other (5)

M. Martinez-Corral, A. Dorado, A. Llavador, G. Saavedra, and B. Javidi, “Three-dimensional integral imaging and display” in Multi-Dimensional Imaging, B. Javidi, E. Tajahuerce, and P. Andres, eds. (John Willey and Sons, 2014), Chap. 11.

M. Martinez-Corral, G. Saavedra, E. Sánchez-Ortiga, A. Llavador, and J. Sola-Pikabea, “Microscopio integral, usos del mismo y sistema de microscopía integral,” Spanish patent P201531938.

R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Tech. Rep. CSTR. 2 (2005).

J. B. Pawley, Handbook of Biological Confocal Microscopy, 3rd ed. (Plenum, 2006).

E. H. K. Stelzer, K. Greger, and E. G. Reynaud, Light Sheet Based Fluorescence Microscopy: Principles and Practice (Wiley-Blackwell, 2014).

Supplementary Material (6)

NameDescription
» Visualization 1: AVI (726 KB)      Visualization 1
» Visualization 2: AVI (1511 KB)      Visualization 2
» Visualization 3: AVI (443 KB)      Visualization 3
» Visualization 4: AVI (1116 KB)      Visualization 4
» Visualization 5: AVI (319 KB)      Visualization 5
» Visualization 6: AVI (458 KB)      Visualization 6

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1 Scheme of an Integral Microscope. The MLA is placed at the image plane of a conventional microscope and the CCD is shifted a distance f ' ML from the microlens array
Fig. 2
Fig. 2 Integral image acquired with our IMic of Fig. 1. The image is composed by 72x72 microimages with 27x27 pixels each. The structure of the microimages can be seen at the inset.
Fig. 3
Fig. 3 a) Single frame extracted from the depth reconstruction of the 3D sample obtained after applying conventional algorithm of refocusing (Visualization 1). In (b) and (c) we can see two perspective views of the microscopic object. A movie with continuous variation of horizontal and vertical perspective is shown in Visualization 2.
Fig. 4
Fig. 4 Capture and reconstruction of a 1951 high resolution USAF test. (a) and (c) show the integral image captured with the object placed at a distance of z 0 =0 mm and z 0 =0.03 mm from the FFP of the microscope objective. In (b) and (d) we show the depth reconstruction from (a) and (c), respectively. Additionally, in Visualization 3 we show a movie with the reconstructed images corresponding to object positions ranging from z 0 =0.02 mm to z 0 =0.06 mm.
Fig. 5
Fig. 5 (a) Scheme of a FiMic. (b) Microlens arrangement at the pupil plane used in the experiment. (c) Experimental configuration of the FiMic with a relay system.
Fig. 6
Fig. 6 Integral image captured with the FiMic. The integral image is composed by 3x3 EIs, with 720x720 pixels each.
Fig. 7
Fig. 7 Reconstructions of the 3D sample. In (a) and (b) we show two frames extracted from the depth reconstruction of the 3D sample obtained after applying conventional algorithm of refocusing (Visualization 4). In (c) and (d) we show two perspective views extracted from the capture shown in Fig. 6. A movie with variation of horizontal and vertical perspective is shown in Visualization 5.
Fig. 8
Fig. 8 Depth reconstruction of the USAF test chart captured with the FiMic when it is placed at a distance of (a) z 0 =0 mm and (b) z 0 =0.03 mm from the FFP of the microscope objective.

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

N A MO M = p 2f ' ML ,
( x 1 θ 1 )=( 0 f ' TL 1/f ' TL 0 )( x 0 θ 0 ).

Metrics