Abstract

Extended depth of field (EDF) microscopy, achieved through computational optics, allows for real-time 3D imaging of live cell dynamics. EDF is achieved through a combination of point spread function engineering and digital image processing. A linear Wiener filter has been conventionally used to deconvolve the image, but it suffers from high frequency noise amplification and processing artifacts. A nonlinear processing scheme is proposed which extends the depth of field while minimizing background noise. The nonlinear filter is generated via a training algorithm and an iterative optimizer. Biological microscope images processed with the nonlinear filter show a significant improvement in image quality and signal-to-noise ratio over the conventional linear filter.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13, 191–219 (1984).
    [CrossRef]
  2. J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
    [CrossRef]
  3. J. B. Pawley, “Limitations on optical sectioning in live-cell confocal microscopy,” Scanning 24, 241–246 (2002).
    [CrossRef]
  4. M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope,” J. Microsc. 235, 144–162 (2009).
    [CrossRef]
  5. E. R. Dowski and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34, 1859–1866 (1995).
    [CrossRef]
  6. S. C. Tucker, W. T. Cathey, and E. Dowski, “Extended depth of field and aberration control for inexpensive digital microscope systems,” Opt. Express 4, 467–474 (1999).
    [CrossRef]
  7. M. R. Arnison, C. J. Cogswell, C. J. R. Sheppard, and P. Török, in Optical Imaging and Microscopy: Techniques and Advanced Systems (Springer-Verlag, 2003).
  8. R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Simultaneous quantitative depth mapping and extended depth of field for 4D microscopy through PSF engineering,” Proc. SPIE 8227, 822705 (2012).
    [CrossRef]
  9. I. E. Beckers, R. H. Cormack, and C. J. Cogswell, “Real-time extended-depth DIC microscopy,” Proc. SPIE 7570, 757013 (2010).
    [CrossRef]
  10. I. N. Bankman, in Handbook of Medical Imaging (Academic, 2000), Chap. 1.
  11. S. Yuan and C. Preza, “Point-spread function engineering to reduce the impact of spherical aberration on 3D computational fluorescence microscopy imaging,” Opt. Express 19, 298–314 (2011).
    [CrossRef]
  12. A. D. Hillery and R. T. Chin, “Iterative Wiener filters for image resoration,” IEEE Trans. Signal Process. 39, 1892–1899 (1991).
    [CrossRef]
  13. D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT, 1986).
  14. R. Hecht-Nielsen, “Theory of the backpropagation neural network,” in Proceedings of IEEE International Joint Conference on Neural Networks (IEEE, 1989), pp. 593–605.
  15. R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Reducing noise in extended depth of field microscope images by optical manipulation of the point spread function,” Proc. SPIE 7904, 79041E (2011).
    [CrossRef]
  16. A. Hasegawa, K. Itoh, and Y. Ichioka, “Generalization of shift invariant neural networks: image processing of corneal endothelium,” Neural Networks 9, 345–356 (1996).
    [CrossRef]
  17. W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
    [CrossRef]
  18. A. S. Miller, B. H. Blott, and T. K. Hames, “Review of neural network applications in medical imaging and signal processing,” Med. Biol. Eng. Comput. 30, 449–464 (1992).
    [CrossRef]
  19. X. P. Zhang, “Thresholding neural network for adaptive noise reduction,” Neural Networks 12, 567–584 (2001).
    [CrossRef]
  20. S. Tamura, “An analysis of a noise reduction neural network,” in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (IEEE, 1989), pp. 2001–2004.
  21. J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of IEEE International Conference on Neural Networks (IEEE, 1995), pp. 1942–1948.
  22. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
    [CrossRef]
  23. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
    [CrossRef]
  24. K. Jain, Fundamentals of Digital Images Processing(Prentice-Hall, 1989).
  25. J. T. Bushberg, J. A. Seibert, E. M. Leidholt, and J. M. Boone, The Essential Physics of Medical Imaging (Lippincott Williams & Wilkins, 2006).

2012 (1)

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Simultaneous quantitative depth mapping and extended depth of field for 4D microscopy through PSF engineering,” Proc. SPIE 8227, 822705 (2012).
[CrossRef]

2011 (2)

S. Yuan and C. Preza, “Point-spread function engineering to reduce the impact of spherical aberration on 3D computational fluorescence microscopy imaging,” Opt. Express 19, 298–314 (2011).
[CrossRef]

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Reducing noise in extended depth of field microscope images by optical manipulation of the point spread function,” Proc. SPIE 7904, 79041E (2011).
[CrossRef]

2010 (1)

I. E. Beckers, R. H. Cormack, and C. J. Cogswell, “Real-time extended-depth DIC microscopy,” Proc. SPIE 7570, 757013 (2010).
[CrossRef]

2009 (1)

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope,” J. Microsc. 235, 144–162 (2009).
[CrossRef]

2002 (1)

J. B. Pawley, “Limitations on optical sectioning in live-cell confocal microscopy,” Scanning 24, 241–246 (2002).
[CrossRef]

2001 (1)

X. P. Zhang, “Thresholding neural network for adaptive noise reduction,” Neural Networks 12, 567–584 (2001).
[CrossRef]

1999 (2)

J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
[CrossRef]

S. C. Tucker, W. T. Cathey, and E. Dowski, “Extended depth of field and aberration control for inexpensive digital microscope systems,” Opt. Express 4, 467–474 (1999).
[CrossRef]

1996 (2)

A. Hasegawa, K. Itoh, and Y. Ichioka, “Generalization of shift invariant neural networks: image processing of corneal endothelium,” Neural Networks 9, 345–356 (1996).
[CrossRef]

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

1995 (1)

1992 (1)

A. S. Miller, B. H. Blott, and T. K. Hames, “Review of neural network applications in medical imaging and signal processing,” Med. Biol. Eng. Comput. 30, 449–464 (1992).
[CrossRef]

1991 (1)

A. D. Hillery and R. T. Chin, “Iterative Wiener filters for image resoration,” IEEE Trans. Signal Process. 39, 1892–1899 (1991).
[CrossRef]

1984 (1)

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13, 191–219 (1984).
[CrossRef]

1983 (1)

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef]

1953 (1)

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Agard, D. A.

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13, 191–219 (1984).
[CrossRef]

Arnison, M. R.

M. R. Arnison, C. J. Cogswell, C. J. R. Sheppard, and P. Török, in Optical Imaging and Microscopy: Techniques and Advanced Systems (Springer-Verlag, 2003).

Bankman, I. N.

I. N. Bankman, in Handbook of Medical Imaging (Academic, 2000), Chap. 1.

Beckers, I. E.

I. E. Beckers, R. H. Cormack, and C. J. Cogswell, “Real-time extended-depth DIC microscopy,” Proc. SPIE 7570, 757013 (2010).
[CrossRef]

Blott, B. H.

A. S. Miller, B. H. Blott, and T. K. Hames, “Review of neural network applications in medical imaging and signal processing,” Med. Biol. Eng. Comput. 30, 449–464 (1992).
[CrossRef]

Boone, J. M.

J. T. Bushberg, J. A. Seibert, E. M. Leidholt, and J. M. Boone, The Essential Physics of Medical Imaging (Lippincott Williams & Wilkins, 2006).

Bushberg, J. T.

J. T. Bushberg, J. A. Seibert, E. M. Leidholt, and J. M. Boone, The Essential Physics of Medical Imaging (Lippincott Williams & Wilkins, 2006).

Cathey, W. T.

Chin, R. T.

A. D. Hillery and R. T. Chin, “Iterative Wiener filters for image resoration,” IEEE Trans. Signal Process. 39, 1892–1899 (1991).
[CrossRef]

Cogswell, C. J.

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Simultaneous quantitative depth mapping and extended depth of field for 4D microscopy through PSF engineering,” Proc. SPIE 8227, 822705 (2012).
[CrossRef]

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Reducing noise in extended depth of field microscope images by optical manipulation of the point spread function,” Proc. SPIE 7904, 79041E (2011).
[CrossRef]

I. E. Beckers, R. H. Cormack, and C. J. Cogswell, “Real-time extended-depth DIC microscopy,” Proc. SPIE 7570, 757013 (2010).
[CrossRef]

M. R. Arnison, C. J. Cogswell, C. J. R. Sheppard, and P. Török, in Optical Imaging and Microscopy: Techniques and Advanced Systems (Springer-Verlag, 2003).

Conchello, J. A.

J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
[CrossRef]

Cooper, J.

J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
[CrossRef]

Cormack, R. H.

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Simultaneous quantitative depth mapping and extended depth of field for 4D microscopy through PSF engineering,” Proc. SPIE 8227, 822705 (2012).
[CrossRef]

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Reducing noise in extended depth of field microscope images by optical manipulation of the point spread function,” Proc. SPIE 7904, 79041E (2011).
[CrossRef]

I. E. Beckers, R. H. Cormack, and C. J. Cogswell, “Real-time extended-depth DIC microscopy,” Proc. SPIE 7570, 757013 (2010).
[CrossRef]

Doi, K.

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

Dowski, E.

Dowski, E. R.

Eberhart, R.

J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of IEEE International Conference on Neural Networks (IEEE, 1995), pp. 1942–1948.

Gelatt, C. D.

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef]

Giger, M. L.

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

Hames, T. K.

A. S. Miller, B. H. Blott, and T. K. Hames, “Review of neural network applications in medical imaging and signal processing,” Med. Biol. Eng. Comput. 30, 449–464 (1992).
[CrossRef]

Hasegawa, A.

A. Hasegawa, K. Itoh, and Y. Ichioka, “Generalization of shift invariant neural networks: image processing of corneal endothelium,” Neural Networks 9, 345–356 (1996).
[CrossRef]

Hecht-Nielsen, R.

R. Hecht-Nielsen, “Theory of the backpropagation neural network,” in Proceedings of IEEE International Joint Conference on Neural Networks (IEEE, 1989), pp. 593–605.

Hillery, A. D.

A. D. Hillery and R. T. Chin, “Iterative Wiener filters for image resoration,” IEEE Trans. Signal Process. 39, 1892–1899 (1991).
[CrossRef]

Ichioka, Y.

A. Hasegawa, K. Itoh, and Y. Ichioka, “Generalization of shift invariant neural networks: image processing of corneal endothelium,” Neural Networks 9, 345–356 (1996).
[CrossRef]

Itoh, K.

A. Hasegawa, K. Itoh, and Y. Ichioka, “Generalization of shift invariant neural networks: image processing of corneal endothelium,” Neural Networks 9, 345–356 (1996).
[CrossRef]

Jain, K.

K. Jain, Fundamentals of Digital Images Processing(Prentice-Hall, 1989).

Karpova, T.

J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
[CrossRef]

Kennedy, J.

J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of IEEE International Conference on Neural Networks (IEEE, 1995), pp. 1942–1948.

Kirkpatrick, S.

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef]

Leidholt, E. M.

J. T. Bushberg, J. A. Seibert, E. M. Leidholt, and J. M. Boone, The Essential Physics of Medical Imaging (Lippincott Williams & Wilkins, 2006).

Levoy, M.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope,” J. Microsc. 235, 144–162 (2009).
[CrossRef]

McClelland, J. L.

D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT, 1986).

McDowall, I.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope,” J. Microsc. 235, 144–162 (2009).
[CrossRef]

McNally, J. G.

J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
[CrossRef]

Metropolis, N.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Miller, A. S.

A. S. Miller, B. H. Blott, and T. K. Hames, “Review of neural network applications in medical imaging and signal processing,” Med. Biol. Eng. Comput. 30, 449–464 (1992).
[CrossRef]

Nishikawa, R. M.

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

Pawley, J. B.

J. B. Pawley, “Limitations on optical sectioning in live-cell confocal microscopy,” Scanning 24, 241–246 (2002).
[CrossRef]

Preza, C.

S. Yuan and C. Preza, “Point-spread function engineering to reduce the impact of spherical aberration on 3D computational fluorescence microscopy imaging,” Opt. Express 19, 298–314 (2011).
[CrossRef]

Rosenbluth, A. W.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Rosenbluth, M. N.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Rumelhart, D. E.

D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT, 1986).

Schmidt, R. A.

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

Seibert, J. A.

J. T. Bushberg, J. A. Seibert, E. M. Leidholt, and J. M. Boone, The Essential Physics of Medical Imaging (Lippincott Williams & Wilkins, 2006).

Sheppard, C. J. R.

M. R. Arnison, C. J. Cogswell, C. J. R. Sheppard, and P. Török, in Optical Imaging and Microscopy: Techniques and Advanced Systems (Springer-Verlag, 2003).

Tamura, S.

S. Tamura, “An analysis of a noise reduction neural network,” in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (IEEE, 1989), pp. 2001–2004.

Teller, A. H.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Teller, E.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

Török, P.

M. R. Arnison, C. J. Cogswell, C. J. R. Sheppard, and P. Török, in Optical Imaging and Microscopy: Techniques and Advanced Systems (Springer-Verlag, 2003).

Tucker, S. C.

Vecchi, M. P.

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef]

Yuan, S.

S. Yuan and C. Preza, “Point-spread function engineering to reduce the impact of spherical aberration on 3D computational fluorescence microscopy imaging,” Opt. Express 19, 298–314 (2011).
[CrossRef]

Zahreddine, R. N.

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Simultaneous quantitative depth mapping and extended depth of field for 4D microscopy through PSF engineering,” Proc. SPIE 8227, 822705 (2012).
[CrossRef]

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Reducing noise in extended depth of field microscope images by optical manipulation of the point spread function,” Proc. SPIE 7904, 79041E (2011).
[CrossRef]

Zhang, W.

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

Zhang, X. P.

X. P. Zhang, “Thresholding neural network for adaptive noise reduction,” Neural Networks 12, 567–584 (2001).
[CrossRef]

Zhang, Z.

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope,” J. Microsc. 235, 144–162 (2009).
[CrossRef]

Annu. Rev. Biophys. Bioeng. (1)

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13, 191–219 (1984).
[CrossRef]

Appl. Opt. (1)

IEEE Trans. Signal Process. (1)

A. D. Hillery and R. T. Chin, “Iterative Wiener filters for image resoration,” IEEE Trans. Signal Process. 39, 1892–1899 (1991).
[CrossRef]

J. Chem. Phys. (1)

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1092 (1953).
[CrossRef]

J. Microsc. (1)

M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope,” J. Microsc. 235, 144–162 (2009).
[CrossRef]

Med. Biol. Eng. Comput. (1)

A. S. Miller, B. H. Blott, and T. K. Hames, “Review of neural network applications in medical imaging and signal processing,” Med. Biol. Eng. Comput. 30, 449–464 (1992).
[CrossRef]

Med. Phys. (1)

W. Zhang, K. Doi, M. L. Giger, R. M. Nishikawa, and R. A. Schmidt, “An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms,” Med. Phys. 23, 595–601 (1996).
[CrossRef]

Methods (1)

J. G. McNally, T. Karpova, J. Cooper, and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999).
[CrossRef]

Neural Networks (2)

X. P. Zhang, “Thresholding neural network for adaptive noise reduction,” Neural Networks 12, 567–584 (2001).
[CrossRef]

A. Hasegawa, K. Itoh, and Y. Ichioka, “Generalization of shift invariant neural networks: image processing of corneal endothelium,” Neural Networks 9, 345–356 (1996).
[CrossRef]

Opt. Express (2)

S. Yuan and C. Preza, “Point-spread function engineering to reduce the impact of spherical aberration on 3D computational fluorescence microscopy imaging,” Opt. Express 19, 298–314 (2011).
[CrossRef]

S. C. Tucker, W. T. Cathey, and E. Dowski, “Extended depth of field and aberration control for inexpensive digital microscope systems,” Opt. Express 4, 467–474 (1999).
[CrossRef]

Proc. SPIE (3)

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Simultaneous quantitative depth mapping and extended depth of field for 4D microscopy through PSF engineering,” Proc. SPIE 8227, 822705 (2012).
[CrossRef]

I. E. Beckers, R. H. Cormack, and C. J. Cogswell, “Real-time extended-depth DIC microscopy,” Proc. SPIE 7570, 757013 (2010).
[CrossRef]

R. N. Zahreddine, R. H. Cormack, and C. J. Cogswell, “Reducing noise in extended depth of field microscope images by optical manipulation of the point spread function,” Proc. SPIE 7904, 79041E (2011).
[CrossRef]

Scanning (1)

J. B. Pawley, “Limitations on optical sectioning in live-cell confocal microscopy,” Scanning 24, 241–246 (2002).
[CrossRef]

Science (1)

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef]

Other (8)

K. Jain, Fundamentals of Digital Images Processing(Prentice-Hall, 1989).

J. T. Bushberg, J. A. Seibert, E. M. Leidholt, and J. M. Boone, The Essential Physics of Medical Imaging (Lippincott Williams & Wilkins, 2006).

S. Tamura, “An analysis of a noise reduction neural network,” in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (IEEE, 1989), pp. 2001–2004.

J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of IEEE International Conference on Neural Networks (IEEE, 1995), pp. 1942–1948.

D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT, 1986).

R. Hecht-Nielsen, “Theory of the backpropagation neural network,” in Proceedings of IEEE International Joint Conference on Neural Networks (IEEE, 1989), pp. 593–605.

I. N. Bankman, in Handbook of Medical Imaging (Academic, 2000), Chap. 1.

M. R. Arnison, C. J. Cogswell, C. J. R. Sheppard, and P. Török, in Optical Imaging and Microscopy: Techniques and Advanced Systems (Springer-Verlag, 2003).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1.
Fig. 1.

Diagram of an EDF microscope. A phase plate is placed at the exit pupil of the imaging objective. The phase plate creates a focus invariant PSF that blurs the signal through an extended depth. Resolution is restored through a deconvolution filter, creating an image with up to 10 times the DOF.

Fig. 2.
Fig. 2.

Illustration of EDF microscopy with a cubic phase plate. The top row (a) shows a simulated traditional PSF with best focus at the left. As defocus increases, the diffraction-limited spot quickly blurs. The simulated cubic PSF in the middle row (b) is largely invariant with increasing defocus. In the absence of noise, a single linear deconvolution filter can then be applied to all of the images of the engineered PSFs to remove the effect of the cubic phase plate. The bottom row (c) shows the filtered PSFs from the varying focal planes that have all been restored to a sharp focus, thus extending the optical system’s DOF.

Fig. 3.
Fig. 3.

Comparison of MTFs through focus. The solid lines show the MTFs of a traditional microscope objective over 5 μm of defocus. The MTF starts out near diffraction-limited performance, but then quickly falls off as the focus is shifted [see Fig. 2(a) for the corresponding PSFs of these MTF plots]. The defocus results in the complete loss of certain spatial frequencies (i.e., zeros in the transfer function). The dashed line represents MTFs generated by the cubic phase plate over the same through focal region [corresponding PSFs shown in Fig. 2(b)]. The cubic MTFs are largely invariant over the 5 μm span, and retain the full spatial frequency information out to the cutoff (i.e., no zeros in the transfer function). These properties—focus invariance and no zeros—make it possible to use a single digital filter to restore the cubic MTFs back to diffraction-limited resolution.

Fig. 4.
Fig. 4.

Noise amplification from the linear filtering process for a cubic PSF. When no noise (a) is present, the cubic SFR is perfectly restored to a diffraction-limited (ideal SFR) condition. As the noise increases to 2.5% (b), the reconstruction becomes less reliable. At (c), 5% noise, the linearly filtered MTF has severely reduced the resolution and contrast of the EDF image. The filter can be optimized to correct this (in theory) via the Wiener parameter, but it requires complete knowledge of the noise and signal characteristics in frequency space. Since the goal of this system is to view objects that extend beyond the traditional depth of focus in real time, there is no way obtain the necessary object and noise information to properly determine the Wiener parameter. Linear filtering will always be a suboptimal solution because of these restrictions.

Fig. 5.
Fig. 5.

Amplification and processing artifacts in a test image from Wiener filtering. (a) Original MIT “camera man” image. (b) Image was then convolved with a cubic PSF and corrupted with 5% AWGN to simulate an experimentally obtained EDF image. (c) Encoded image was filtered (deconvolved) to remove blur from the cubic PSF. However, the filter has introduced significant background noise and ringing artifacts despite being optimized.

Fig. 6.
Fig. 6.

Diagram of the three-layer, shift invariant neural network. This simplified example shows pixels from an input image, Xi, as they are weighted, summed, and operated on to create the nonlinearly filtered pixels of the output image, Zk. The entire input image, X, is convolved with the input 2D weighting array, WIN, where the elements of the array are represented by lower case letters (a, b, c, etc.). The weighted input image pixels, Mj, are then operated on by a sigmoidal activation function, g(*). The output of the activation function, Y, is then convolved with the output 2D weighting array, WOUT, where the elements of the array are represented by the lower case letters (e, d, f, etc.). This generates the final nonlinearly filtered image, Z.

Fig. 7.
Fig. 7.

Training images used to determine the optimal value for the weighting arrays in the nonlinear filter. The original, diffraction-limited resolution target (a) is convolved with a cubic PSF and background noise is added to simulate an encoded EDF image. An example is shown with 7.5% white Gaussian background noise (b). The nonlinear filter is applied and an optimizer adjusts the filter weights to minimize the difference from the original. The optimized filter is then applied to test image (c). The filtered image is not a perfect match to the original, but can be improved in theory through modification of the merit function. A measured cubic PSF from a 20x/0.5NA objective was used for training in this example.

Fig. 8.
Fig. 8.

Input weighting array, WIN (a) and output weighting array, WOUT (b) for the shift-invariant neural network used in the example experiment. Intensity of each square corresponds to the numerical value of that element in the 2D array, shown in the grayscale bars at the right.

Fig. 9.
Fig. 9.

Comparison between the linear and nonlinear filter on a simulated resolution target in the noiseless case. The diffraction-limited bar chart (a) was convolved with a cubic PSF. The target was then both linearly (b) and nonlinearly (c) filtered. Both filters have been matched in bandwidth. The nonlinear filter shows higher contrast than the linear filter in the ideal imaging case. Target bar spacing decreases to the diffraction limit of 1.49cycles/μm, which correlates to that of a 20x/0.5NA lens. The scale bar shown in (c) is 5 μm.

Fig. 10.
Fig. 10.

Comparison of the linear and nonlinear filter with 7.5% additive Gaussian white noise. The linearly filtered image (a) exhibits a large amount of amplified noise. The nonlinearly filtered image (b) maintains resolution with improved contrast for high spatial frequencies. Note that the two filters still have identical bandwidth. All intensity variations at the location of the highest spatial frequency bar are solely due to the amplified noise and not from any increase in resolution of the signal.

Fig. 11.
Fig. 11.

Comparison of conventional versus EDF microscopy of Oscillatoria algae imaged in fluorescence. (a) Algae under traditional imaging. The limited DOF causes significant blur. (b) Algae is then imaged with a cubic phase plate. The DOF has been extended, but there is still an overall uniform blur imparted by the cubic phase mask. The image is then filtered linearly (c) and nonlinearly (d). Both filters deconvolve the cubic PSF, but the linear filter has significant background noise and ringing artifacts. The nonlinearly filtered image extends the DOF without amplifying the noise and with minimum artifacts. Note that a portion of the algae is still beyond the EDF so is slightly blurred.

Fig. 12.
Fig. 12.

Comparison of conventional versus EDF microscopy of Oscillatoria algae imaged in brightfield with a 20x/0.5NA objective. Figures 12(a)12(d) follow the same progression as Fig. 11 using the same weighting arrays. Again, the nonlinearly filtered image shows improved SNR and fewer artifacts when compared to the linear filter.

Fig. 13.
Fig. 13.

Comparison between the full bandwidth linear filter (a), the truncated linear filter (b), and the nonlinear filter (c) in a fluorescence image of the same Oscillatoria algae. The bandwidth of the linear filter was iteratively shortened until the intensity of the ringing artifacts was less than 10% of the peak intensity value. Truncating the linear filter mitigates ringing artifacts and background noise, but at the cost of degraded resolution. The fine detail has been lost in the linearly filtered image, while the nonlinearly filtered image maintains resolution without the ringing artifacts or noise. Imaged with a 20x/0.5NA objective.

Tables (1)

Tables Icon

Table 1. Comparison of SNR between the Linear and Nonlinear Filter

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

φ(x,y)(x3+y3).
F(fx,fy)=H*(fx,fy)|H(fx,fy)|2+N(fx,fy)S(fx,fy).
Z=WOUTg(WINX),
Yj=g(Mj)=11+e2r(Mjc),

Metrics