Abstract

Controlling photographic illumination in a structured fashion is a common practice in computational photography and image-based rendering. Here we introduce an incoherent photographic imaging approach, termed Fourier ptychographic photography, that uses nonuniform structured light for super-resolution imaging. In this approach, frequency mixing between the object and the structured light shifts the high-frequency object information to the passband of the photographic lens. Therefore, the recorded intensity images contain object information that is beyond the cutoff frequency of the collection optics. Based on multiple images acquired under different structured light patterns, we used the Fourier ptychographic algorithm to recover the super-resolution object image and the unknown illumination pattern. We demonstrated the reported approach by imaging various objects, including a resolution target, a quick response code, a dollar bill, an insect, and a color leaf. The reported approach may find applications in photographic imaging settings, remote sensing, and imaging radar. It may also provide new insights for high-resolution imaging by shifting the focus from the collection optics to the generation of structured light.

© 2015 Chinese Laser Press

Full Article  |  PDF Article
OSA Recommended Articles
High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography

Siyuan Dong, Pariksheet Nanda, Radhika Shiradkar, Kaikai Guo, and Guoan Zheng
Opt. Express 22(17) 20856-20870 (2014)

Recovering higher dimensional image data using multiplexed structured illumination

Siyuan Dong, Kaikai Guo, Shaowei Jiang, and Guoan Zheng
Opt. Express 23(23) 30393-30398 (2015)

Resolution doubling with a reduced number of image acquisitions

Siyuan Dong, Jun Liao, Kaikai Guo, Liheng Bian, Jinli Suo, and Guoan Zheng
Biomed. Opt. Express 6(8) 2946-2952 (2015)

References

  • View by:
  • |
  • |
  • |

  1. K.-C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1, I-519–I-526 (2001).
  2. A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
    [Crossref]
  3. B. Horn, Robot Vision (MIT, 1986).
  4. M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009).
    [Crossref]
  5. M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).
  6. P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).
  7. M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198, 82–87 (2000).
    [Crossref]
  8. D. R. Gerwe and M. A. Plonus, “Superresolved image reconstruction of images taken through the turbulent atmosphere,” J. Opt. Soc. Am. A 15, 2620–2628 (1998).
    [Crossref]
  9. A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65–75 (2004).
  10. X. Ou, R. Horstmeyer, C. Yang, and G. Zheng, “Quantitative phase imaging via Fourier ptychographic microscopy,” Opt. Lett. 38, 4845–4848 (2013).
    [Crossref]
  11. G. Zheng, R. Horstmeyer, and C. Yang, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nat. Photonics 7, 739–745 (2013).
    [Crossref]
  12. S. Dong, Z. Bian, R. Shiradkar, and G. Zheng, “Sparsely sampled Fourier ptychography,” Opt. Express 22, 5455–5464 (2014).
    [Crossref]
  13. S. Dong, R. Horstmeyer, R. Shiradkar, K. Guo, X. Ou, Z. Bian, H. Xin, and G. Zheng, “Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging,” Opt. Express 22, 13586–13599 (2014).
    [Crossref]
  14. S. Dong, R. Shiradkar, P. Nanda, and G. Zheng, “Spectral multiplexing and coherent-state decomposition in Fourier ptychographic imaging,” Biomed. Opt. Express 5, 1757–1767 (2014).
    [Crossref]
  15. X. Ou, G. Zheng, and C. Yang, “Embedded pupil function recovery for Fourier ptychographic microscopy,” Opt. Express 22, 4960–4972 (2014).
    [Crossref]
  16. S. Dong, P. Nanda, R. Shiradkar, K. Guo, and G. Zheng, “High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography,” Opt. Express 22, 20856–20870 (2014).
    [Crossref]
  17. R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–250 (1972).
  18. H. Faulkner and J. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004).
    [Crossref]
  19. J. R. Fienup, “Reconstruction of an object from the modulus of its Fourier transform,” Opt. Lett. 3, 27–29 (1978).
    [Crossref]
  20. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21, 2758–2769 (1982).
    [Crossref]
  21. F. Zhang, I. Peterson, J. Vila-Comamala, A. Diaz, F. Berenguer, R. Bean, B. Chen, A. Menzel, I. K. Robinson, and J. M. Rodenburg, “Translation position determination in ptychographic coherent diffraction imaging,” Opt. Express 21, 13592–13606 (2013).
    [Crossref]
  22. A. M. Maiden and J. M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109, 1256–1262 (2009).
    [Crossref]

2014 (5)

2013 (3)

2009 (2)

A. M. Maiden and J. M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109, 1256–1262 (2009).
[Crossref]

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009).
[Crossref]

2005 (1)

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

2004 (3)

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

H. Faulkner and J. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004).
[Crossref]

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65–75 (2004).

2001 (2)

K.-C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1, I-519–I-526 (2001).

A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[Crossref]

2000 (1)

M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198, 82–87 (2000).
[Crossref]

1998 (1)

1982 (1)

1978 (1)

1972 (1)

R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–250 (1972).

Bean, R.

Belhumeur, P. N.

A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[Crossref]

Berenguer, F.

Bian, Z.

Bolas, M.

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

Chen, B.

F. Zhang, I. Peterson, J. Vila-Comamala, A. Diaz, F. Berenguer, R. Bean, B. Chen, A. Menzel, I. K. Robinson, and J. M. Rodenburg, “Translation position determination in ptychographic coherent diffraction imaging,” Opt. Express 21, 13592–13606 (2013).
[Crossref]

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

Diaz, A.

Dong, S.

Faulkner, H.

H. Faulkner and J. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004).
[Crossref]

Fienup, J. R.

Fraser, D.

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65–75 (2004).

Garg, G.

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

Georghiades, A. S.

A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[Crossref]

Gerchberg, R. W.

R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–250 (1972).

Gerwe, D. R.

Guo, K.

Gustafsson, M. G.

M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198, 82–87 (2000).
[Crossref]

Ho, J.

K.-C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1, I-519–I-526 (2001).

Horn, B.

B. Horn, Robot Vision (MIT, 1986).

Horowitz, M.

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

Horstmeyer, R.

Kriegman, D.

A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[Crossref]

K.-C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1, I-519–I-526 (2001).

Lambert, A. J.

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65–75 (2004).

Lee, K.-C.

K.-C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1, I-519–I-526 (2001).

Lensch, H.

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

Levoy, M.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009).
[Crossref]

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

Maiden, A. M.

A. M. Maiden and J. M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109, 1256–1262 (2009).
[Crossref]

Marschner, S. R.

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

McDowall, I.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009).
[Crossref]

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

Menzel, A.

Nanda, P.

Ou, X.

Peterson, I.

Plonus, M. A.

Robinson, I. K.

Rodenburg, J.

H. Faulkner and J. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004).
[Crossref]

Rodenburg, J. M.

Saxton, W. O.

R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–250 (1972).

Sen, P.

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

Shiradkar, R.

Vaish, V.

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

Vila-Comamala, J.

Xin, H.

Yang, C.

Zhang, F.

Zhang, Z.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009).
[Crossref]

Zheng, G.

ACM Trans. Graph. (2)

M. Levoy, B. Chen, V. Vaish, M. Horowitz, I. McDowall, and M. Bolas, “Synthetic aperture confocal imaging,” ACM Trans. Graph. 23, 825–834 (2004).

P. Sen, B. Chen, G. Garg, S. R. Marschner, M. Horowitz, M. Levoy, and H. Lensch, “Dual photography,” ACM Trans. Graph. 24, 745–755 (2005).

Appl. Opt. (1)

Biomed. Opt. Express (1)

IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. (1)

K.-C. Lee, J. Ho, and D. Kriegman, “Nine points of light: acquiring subspaces for face recognition under variable lighting,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1, I-519–I-526 (2001).

IEEE Trans. Pattern Anal. Mach. Intell. (1)

A. S. Georghiades, P. N. Belhumeur, and D. Kriegman, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001).
[Crossref]

J. Microsc. (2)

M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198, 82–87 (2000).
[Crossref]

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009).
[Crossref]

J. Opt. Soc. Am. A (1)

Nat. Photonics (1)

G. Zheng, R. Horstmeyer, and C. Yang, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nat. Photonics 7, 739–745 (2013).
[Crossref]

Opt. Express (5)

Opt. Lett. (2)

Optik (1)

R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–250 (1972).

Phys. Rev. Lett. (1)

H. Faulkner and J. Rodenburg, “Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm,” Phys. Rev. Lett. 93, 023903 (2004).
[Crossref]

Proc. SPIE (1)

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65–75 (2004).

Ultramicroscopy (1)

A. M. Maiden and J. M. Rodenburg, “An improved ptychographical phase retrieval algorithm for diffractive imaging,” Ultramicroscopy 109, 1256–1262 (2009).
[Crossref]

Other (1)

B. Horn, Robot Vision (MIT, 1986).

Supplementary Material (2)

» Media 1: AVI (1924 KB)     
» Media 2: AVI (2007 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1.

Schematic of the FPP setup. In the illumination path, a diffused LED was used for incoherent illumination (red arrow on the left). A semitransparent diffuser was placed in front of the diffused LED, and its image was projected onto the object. In the detection path, a photographic lens (Nikon, 50 mm) was used to collect the reflected light from the object.

Fig. 2.
Fig. 2.

Recovery procedures of the coherent and incoherent FP approaches. For the case of incoherent FP, the updating processes in steps 3 and 4 can be expressed as Eqs. (1)–(3).

Fig. 3.
Fig. 3.

Imaging performance of the reported FPP platform. (a) The reference image captured under uniform illumination, (b1) the captured raw image under pattern illumination, (b2) the recovered image using 100 raw images, (b3) the recovered illumination pattern, (b4) line traces of (a) and (b2). Also refer to Media 1.

Fig. 4.
Fig. 4.

Image reconstruction using different numbers of raw images. The solution converges with the 16 raw images. We used 15–20 loops in this experiment.

Fig. 5.
Fig. 5.

Demonstration of the reported platform for different objects: a dollar bill, a quick response code, and an insect. (a1)–(c1) The reference images under uniform illumination, (a2)–(c2) the recovered images using the reported platform. We used 100 raw images and 15 loops for the reconstruction. Also refer to Media 2.

Fig. 6.
Fig. 6.

Imaging a color object using the reported platform. (a1)–(a3) Reference images using uniform R/G/B illumination, (b) combined reference color image, (c1)–(c3) recovered super-resolution images using the reported platform, (d) combined super-resolution color image.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

F(Itn)updated=F(Itn)+OTF·(F(In)OTF·F(Itn)),
Iobjupdated=Iobj+Pn(max(Pn))2·(ItnupdatedIobj·Pn).
Punknownupdated=Punknown+Iobj(max(Iobj))2·(ItnupdatedIobj·Punknown(xxn)),

Metrics