Abstract

Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy and demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing localization microscopy methods for spectral classification require additional optical elements in the emission path, e.g., spectral filters, prisms, or phase masks, our neural net correctly identifies static and mobile emitters with high efficiency using a standard, unmodified single-channel configuration. Furthermore, we show how deep learning can be used to design new phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species, including simultaneously differentiating four species in a single image.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Astigmatic multifocus microscopy enables deep 3D super-resolved imaging

Laura Oudjedi, Jean-Bernard Fiche, Sara Abrahamsson, Laurent Mazenq, Aurélie Lecestre, Pierre-François Calmon, Aline Cerf, and Marcelo Nöllmann
Biomed. Opt. Express 7(6) 2163-2173 (2016)

Deep-STORM: super-resolution single-molecule microscopy by deep learning

Elias Nehme, Lucien E. Weiss, Tomer Michaeli, and Yoav Shechtman
Optica 5(4) 458-464 (2018)

4Pi fluorescence detection and 3D particle localization with a single objective

J. Schnitzbauer, R. McGorty, and B. Huang
Opt. Express 21(17) 19701-19708 (2013)

References

  • View by:
  • |
  • |
  • |

  1. S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006).
    [Crossref] [PubMed]
  2. E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
    [Crossref] [PubMed]
  3. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
    [Crossref] [PubMed]
  4. N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
    [Crossref]
  5. T. A. Klar and S. W. Hell, “Subdiffraction resolution in far-field fluorescence microscopy,” Opt. Lett. 24(14), 954–956 (1999).
    [Crossref] [PubMed]
  6. T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
    [Crossref] [PubMed]
  7. J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
    [Crossref] [PubMed]
  8. S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. 19(11), 780–782 (1994).
    [Crossref] [PubMed]
  9. G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
    [Crossref] [PubMed]
  10. T. Kim, S. Moon, and K. Xu, “Information-rich localization microscopy through machine learning,” https://www.biorxiv.org/content/early/2018/08/07/373878
  11. C. Franke, M. Sauer, and S. van de Linde, “Photometry unlocks 3D information from 2D localization microscopy data,” Nat. Methods 14(1), 41–44 (2017).
    [Crossref] [PubMed]
  12. A. P. Bartko and R. M. Dickson, “Imaging three-dimensional single molecule orientations,” J. Phys. Chem. B 103(51), 11237–11241 (1999).
    [Crossref]
  13. D. J. Rowland and J. S. Biteen, “Top-hat and asymmetric Gaussian-based fitting functions for quantifying directional single-molecule motion,” ChemPhysChem 15(4), 712–720 (2014).
    [Crossref] [PubMed]
  14. M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
    [Crossref] [PubMed]
  15. S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
    [Crossref] [PubMed]
  16. T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
    [Crossref] [PubMed]
  17. A. von Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2(11), 985–993 (2015).
    [Crossref] [PubMed]
  18. Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
    [Crossref] [PubMed]
  19. C. Smith, M. Huisman, M. Siemons, D. Grünwald, and S. Stallinga, “Simultaneous measurement of emission color and 3D position of single molecules,” Opt. Express 24(5), 4996–5013 (2016).
    [Crossref] [PubMed]
  20. A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
    [Crossref] [PubMed]
  21. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
    [Crossref] [PubMed]
  22. A. Chakrabarti, “Learning Sensor Multiplexing Design through Back-propagation,” https://arxiv.org/abs/1605.07078
  23. R. Horstmeyer, R. Y. Chen, B. Kappes, and B. Judkewitz, “Convolutional neural networks that teach microscopes how to image,” https://arxiv.org/abs/1709.07223
  24. S. Elmalem, R. Giryes, and E. Marom, “Learned phase coded aperture for the benefit of depth of field extension,” Opt. Express 26(12), 15316–15331 (2018).
    [Crossref] [PubMed]
  25. E. Nehme, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Deep-STORM: super-resolution single-molecule microscopy by deep learning,” Optica 5(4), 458–464 (2018).
    [Crossref]
  26. W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
    [Crossref] [PubMed]
  27. P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
    [Crossref] [PubMed]
  28. C. Cabriel, N. Bourg, P. Jouchet, G. Dupuis, C. Leterrier, A. Baron, M.-A. Badet-Denisot, B. Vauzeilles, E. Fort, and S. Lévêque-Fort, “Combining 3D single molecule localization strategies for reproducible bioimaging,” https://www.biorxiv.org/content/early/2018/11/16/385799
  29. L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res. 9, 2579–2605 (2008).
  30. A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Colored point spread function engineering for parallel confocal microscopy,” Opt. Express 24(24), 27395–27402 (2016).
    [Crossref] [PubMed]
  31. M. Siemons, C. N. Hulleman, R. Ø. Thorsen, C. S. Smith, and S. Stallinga, “High precision wavefront control in point spread function engineering for single emitter localization,” Opt. Express 26(7), 8397–8416 (2018).
    [Crossref] [PubMed]
  32. Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
    [Crossref] [PubMed]
  33. Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
    [Crossref] [PubMed]
  34. W. Xie, J. A. Noble, and A. Zisserman, “Microscopy cell counting and detection with fully convolutional regression networks,” https://arxiv.org/abs/1708.03307
    [Crossref]
  35. Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
    [Crossref]
  36. P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
    [Crossref] [PubMed]
  37. L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
    [Crossref] [PubMed]
  38. S. J. Holden, S. Uphoff, and A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011).
    [Crossref] [PubMed]
  39. F. Huang, S. L. Schwartz, J. M. Byars, and K. A. Lidke, “Simultaneous multiple-emitter fitting for single molecule super-resolution imaging,” Biomed. Opt. Express 2(5), 1377–1393 (2011).
    [Crossref] [PubMed]
  40. S. Gazagnes, E. Soubies, and L. Blanc-Féraud, “High density molecule localization for super-resolution microscopy using CEL0 based sparse approximation,” in IEEE International Symposium on Biomedical Imaging, (ISBI, 2017), pp. 28–31.
    [Crossref]
  41. J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
    [Crossref] [PubMed]
  42. J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
    [Crossref] [PubMed]
  43. M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
    [Crossref] [PubMed]
  44. A. Vedaldi and K. Lenc, “MatConvNet - Convolutional Neural Networks for MATLAB,” in Proceedings of the 23rd ACM International Conference on Multimedia, (ACM, 2015), pp. 689–692.
    [Crossref]
  45. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” https://arxiv.org/abs/1609.04802
    [Crossref]
  46. D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” https://arxiv.org/abs/1412.6980
  47. S. Hochreiter, “The vanishing gradient problem during learning recurrent neural nets and problem solutions,” Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6(02), 107–116 (1998).
    [Crossref]
  48. T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
    [Crossref] [PubMed]

2018 (6)

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

M. Siemons, C. N. Hulleman, R. Ø. Thorsen, C. S. Smith, and S. Stallinga, “High precision wavefront control in point spread function engineering for single emitter localization,” Opt. Express 26(7), 8397–8416 (2018).
[Crossref] [PubMed]

E. Nehme, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Deep-STORM: super-resolution single-molecule microscopy by deep learning,” Optica 5(4), 458–464 (2018).
[Crossref]

S. Elmalem, R. Giryes, and E. Marom, “Learned phase coded aperture for the benefit of depth of field extension,” Opt. Express 26(12), 15316–15331 (2018).
[Crossref] [PubMed]

2017 (4)

C. Franke, M. Sauer, and S. van de Linde, “Photometry unlocks 3D information from 2D localization microscopy data,” Nat. Methods 14(1), 41–44 (2017).
[Crossref] [PubMed]

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref] [PubMed]

Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
[Crossref]

2016 (3)

2015 (6)

S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
[Crossref] [PubMed]

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

A. von Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2(11), 985–993 (2015).
[Crossref] [PubMed]

2014 (4)

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref] [PubMed]

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref] [PubMed]

D. J. Rowland and J. S. Biteen, “Top-hat and asymmetric Gaussian-based fitting functions for quantifying directional single-molecule motion,” ChemPhysChem 15(4), 712–720 (2014).
[Crossref] [PubMed]

2012 (2)

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
[Crossref] [PubMed]

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

2011 (3)

S. J. Holden, S. Uphoff, and A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011).
[Crossref] [PubMed]

F. Huang, S. L. Schwartz, J. M. Byars, and K. A. Lidke, “Simultaneous multiple-emitter fitting for single molecule super-resolution imaging,” Biomed. Opt. Express 2(5), 1377–1393 (2011).
[Crossref] [PubMed]

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

2009 (1)

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

2008 (2)

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res. 9, 2579–2605 (2008).

2007 (1)

M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
[Crossref] [PubMed]

2006 (3)

S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006).
[Crossref] [PubMed]

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref] [PubMed]

1999 (2)

A. P. Bartko and R. M. Dickson, “Imaging three-dimensional single molecule orientations,” J. Phys. Chem. B 103(51), 11237–11241 (1999).
[Crossref]

T. A. Klar and S. W. Hell, “Subdiffraction resolution in far-field fluorescence microscopy,” Opt. Lett. 24(14), 954–956 (1999).
[Crossref] [PubMed]

1998 (1)

S. Hochreiter, “The vanishing gradient problem during learning recurrent neural nets and problem solutions,” Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6(02), 107–116 (1998).
[Crossref]

1994 (1)

Arganda-Carreras, I.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Aristov, A.

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

Backer, A. S.

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref] [PubMed]

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref] [PubMed]

Barroca, T.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Bartko, A. P.

A. P. Bartko and R. M. Dickson, “Imaging three-dimensional single molecule orientations,” J. Phys. Chem. B 103(51), 11237–11241 (1999).
[Crossref]

Bates, M.

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
[Crossref] [PubMed]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref] [PubMed]

Bengio, Y.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
[Crossref] [PubMed]

Bernet, S.

Betzig, E.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Biteen, J. S.

D. J. Rowland and J. S. Biteen, “Top-hat and asymmetric Gaussian-based fitting functions for quantifying directional single-molecule motion,” ChemPhysChem 15(4), 712–720 (2014).
[Crossref] [PubMed]

Bittel, A.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Blanc-Féraud, L.

S. Gazagnes, E. Soubies, and L. Blanc-Féraud, “High density molecule localization for super-resolution microscopy using CEL0 based sparse approximation,” in IEEE International Symposium on Biomedical Imaging, (ISBI, 2017), pp. 28–31.
[Crossref]

Bock, H.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Bon, P.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Bonifacino, J. S.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Borkovec, J.

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Bossi, M.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Bourg, N.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Byars, J. M.

Cardona, A.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Carlini, L.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Chaurasia, A.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

Chen, K. H.

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

Colyer, R.

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

Culurciello, E.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

Davidson, M. W.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Dempsey, G. T.

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
[Crossref] [PubMed]

Dertinger, T.

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

Dickson, R. M.

A. P. Bartko and R. M. Dickson, “Imaging three-dimensional single molecule orientations,” J. Phys. Chem. B 103(51), 11237–11241 (1999).
[Crossref]

Dupuis, G.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Eggeling, C.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Eliceiri, K.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Elmalem, S.

Elnatan, D.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
[Crossref] [PubMed]

Enderlein, J.

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

Erdélyi, M.

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

Fölling, J.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Fort, E.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Franke, C.

C. Franke, M. Sauer, and S. van de Linde, “Photometry unlocks 3D information from 2D localization microscopy data,” Nat. Methods 14(1), 41–44 (2017).
[Crossref] [PubMed]

Frise, E.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Gajdos, T.

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

Gazagnes, S.

S. Gazagnes, E. Soubies, and L. Blanc-Féraud, “High density molecule localization for super-resolution microscopy using CEL0 based sparse approximation,” in IEEE International Symposium on Biomedical Imaging, (ISBI, 2017), pp. 28–31.
[Crossref]

Gibbs, S. L.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Girirajan, T. P. K.

S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006).
[Crossref] [PubMed]

Giryes, R.

Göröcs, Z.

Gray, J. W.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Grünwald, D.

Günaydin, H.

Hagen, G. M.

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Hao, X.

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

Hartenstein, V.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Hein, B.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Hell, S. W.

Heruth, D. P.

S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
[Crossref] [PubMed]

Hess, H. F.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Hess, S. T.

S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006).
[Crossref] [PubMed]

Hinton, G.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
[Crossref] [PubMed]

L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res. 9, 2579–2605 (2008).

Hochreiter, S.

S. Hochreiter, “The vanishing gradient problem during learning recurrent neural nets and problem solutions,” Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6(02), 107–116 (1998).
[Crossref]

Holden, S.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Holden, S. J.

S. J. Holden, S. Uphoff, and A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011).
[Crossref] [PubMed]

Huang, B.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
[Crossref] [PubMed]

M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
[Crossref] [PubMed]

Huang, F.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

F. Huang, S. L. Schwartz, J. M. Byars, and K. A. Lidke, “Simultaneous multiple-emitter fitting for single molecule super-resolution imaging,” Biomed. Opt. Express 2(5), 1377–1393 (2011).
[Crossref] [PubMed]

Huang, T.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Huisman, M.

Hulleman, C. N.

Iyer, G.

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

Jacques, S.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Jakobs, S.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Jesacher, A.

Kapanidis, A. N.

S. J. Holden, S. Uphoff, and A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011).
[Crossref] [PubMed]

Kaynig, V.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Kirshner, H.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Klar, T. A.

Krížek, P.

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Lécart, S.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

LeCun, Y.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
[Crossref] [PubMed]

Lee, M. Y.

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref] [PubMed]

A. von Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2(11), 985–993 (2015).
[Crossref] [PubMed]

Lelek, M.

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

Lenc, K.

A. Vedaldi and K. Lenc, “MatConvNet - Convolutional Neural Networks for MATLAB,” in Proceedings of the 23rd ACM International Conference on Multimedia, (ACM, 2015), pp. 689–692.
[Crossref]

Lévêque-Fort, S.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Lew, M. D.

Lidke, K. A.

Lin, L. J.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Lindwasser, O. W.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Lippincott-Schwartz, J.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Liu, S.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

Longair, M.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Ma, D.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

Manley, S.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Marom, E.

Mason, M. D.

S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006).
[Crossref] [PubMed]

Mayet, C.

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Medda, R.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Michaeli, T.

Min, J.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Mlodzianoski, M. J.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

Moerner, W. E.

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

A. von Diezmann, M. Y. Lee, M. D. Lew, and W. E. Moerner, “Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy,” Optica 2(11), 985–993 (2015).
[Crossref] [PubMed]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref] [PubMed]

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref] [PubMed]

Nan, X.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Nehme, E.

Novák, T.

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

Olenych, S.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Olivier, N.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Ouyang, W.

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

Ovesný, M.

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Ozcan, A.

Patterson, G. H.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Petrov, P. N.

Phelps, C.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Pietzsch, T.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Preibisch, S.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Riordan, S. M.

S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
[Crossref] [PubMed]

Ritsch-Marte, M.

Rivenson, Y.

Rowland, D. J.

D. J. Rowland and J. S. Biteen, “Top-hat and asymmetric Gaussian-based fitting functions for quantifying directional single-molecule motion,” ChemPhysChem 15(4), 712–720 (2014).
[Crossref] [PubMed]

Rueden, C.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Rust, M. J.

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref] [PubMed]

Saalfeld, S.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Sahl, S. J.

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref] [PubMed]

Sauer, M.

C. Franke, M. Sauer, and S. van de Linde, “Photometry unlocks 3D information from 2D localization microscopy data,” Nat. Methods 14(1), 41–44 (2017).
[Crossref] [PubMed]

Schindelin, J.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Schmid, B.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Schwartz, S. L.

Scott, Z.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Shechtman, Y.

E. Nehme, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Deep-STORM: super-resolution single-molecule microscopy by deep learning,” Optica 5(4), 458–464 (2018).
[Crossref]

P. N. Petrov, Y. Shechtman, and W. E. Moerner, “Measurement-based estimation of global pupil functions in 3D localization microscopy,” Opt. Express 25(7), 7945–7959 (2017).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref] [PubMed]

Siemons, M.

Sinkó, J.

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

Smith, C.

Smith, C. S.

Soubies, E.

S. Gazagnes, E. Soubies, and L. Blanc-Féraud, “High density molecule localization for super-resolution microscopy using CEL0 based sparse approximation,” in IEEE International Symposium on Biomedical Imaging, (ISBI, 2017), pp. 28–31.
[Crossref]

Sougrat, R.

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

Stallinga, S.

Svindrych, Z.

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Szabó, G.

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

Thorsen, R. Ø.

Tinevez, J. Y.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Tomancak, P.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Unser, M.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Uphoff, S.

S. J. Holden, S. Uphoff, and A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011).
[Crossref] [PubMed]

van de Linde, S.

C. Franke, M. Sauer, and S. van de Linde, “Photometry unlocks 3D information from 2D localization microscopy data,” Nat. Methods 14(1), 41–44 (2017).
[Crossref] [PubMed]

van der Maaten, L.

L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res. 9, 2579–2605 (2008).

Vaughan, J. C.

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

Vedaldi, A.

A. Vedaldi and K. Lenc, “MatConvNet - Convolutional Neural Networks for MATLAB,” in Proceedings of the 23rd ACM International Conference on Multimedia, (ACM, 2015), pp. 689–692.
[Crossref]

von Diezmann, A.

Vonesch, C.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Wang, H.

Wang, J.

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Weiss, L. E.

E. Nehme, L. E. Weiss, T. Michaeli, and Y. Shechtman, “Deep-STORM: super-resolution single-molecule microscopy by deep learning,” Optica 5(4), 458–464 (2018).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref] [PubMed]

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

Weiss, S.

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

White, D. J.

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

Wichmann, J.

Wurm, C. A.

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

Ye, J. C.

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

Ye, S. Q.

S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
[Crossref] [PubMed]

Zhang, L. Q.

S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
[Crossref] [PubMed]

Zhang, P.

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

Zhang, W.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
[Crossref] [PubMed]

Zhang, Y.

Zhu, L.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
[Crossref] [PubMed]

Zhuang, X.

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
[Crossref] [PubMed]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref] [PubMed]

Zimmer, C.

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

Bioinformatics (1)

M. Ovesný, P. Křížek, J. Borkovec, Z. Svindrych, and G. M. Hagen, “ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging,” Bioinformatics 30(16), 2389–2390 (2014).
[Crossref] [PubMed]

Biomed. Opt. Express (1)

Biophys. J. (2)

S. T. Hess, T. P. K. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006).
[Crossref] [PubMed]

T. Huang, C. Phelps, J. Wang, L. J. Lin, A. Bittel, Z. Scott, S. Jacques, S. L. Gibbs, J. W. Gray, and X. Nan, “Simultaneous multicolor single-molecule tracking with single-laser excitation via spectral imaging,” Biophys. J. 114(2), 301–310 (2018).
[Crossref] [PubMed]

Cell Biosci. (1)

S. M. Riordan, D. P. Heruth, L. Q. Zhang, and S. Q. Ye, “Application of CRISPR/Cas9 for biomedical discoveries,” Cell Biosci. 5(1), 33 (2015).
[Crossref] [PubMed]

ChemPhysChem (1)

D. J. Rowland and J. S. Biteen, “Top-hat and asymmetric Gaussian-based fitting functions for quantifying directional single-molecule motion,” ChemPhysChem 15(4), 712–720 (2014).
[Crossref] [PubMed]

Int. J. Uncertain. Fuzziness Knowl. Based Syst. (1)

S. Hochreiter, “The vanishing gradient problem during learning recurrent neural nets and problem solutions,” Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6(02), 107–116 (1998).
[Crossref]

J. Mach. Learn. Res. (1)

L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res. 9, 2579–2605 (2008).

J. Phys. Chem. B (2)

A. S. Backer and W. E. Moerner, “Extending single-molecule microscopy using optical fourier processing,” J. Phys. Chem. B 118(28), 8313–8329 (2014).
[Crossref] [PubMed]

A. P. Bartko and R. M. Dickson, “Imaging three-dimensional single molecule orientations,” J. Phys. Chem. B 103(51), 11237–11241 (1999).
[Crossref]

Nano Lett. (1)

Y. Shechtman, L. E. Weiss, A. S. Backer, S. J. Sahl, and W. E. Moerner, “Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions,” Nano Lett. 15(6), 4194–4199 (2015).
[Crossref] [PubMed]

Nat. Biotechnol. (1)

W. Ouyang, A. Aristov, M. Lelek, X. Hao, and C. Zimmer, “Deep learning massively accelerates super-resolution localization microscopy,” Nat. Biotechnol. 36(5), 460–468 (2018).
[Crossref] [PubMed]

Nat. Methods (8)

P. Zhang, S. Liu, A. Chaurasia, D. Ma, M. J. Mlodzianoski, E. Culurciello, and F. Huang, “Analyzing complex single-molecule emission patterns with deep learning,” Nat. Methods 15(11), 913–916 (2018).
[Crossref] [PubMed]

C. Franke, M. Sauer, and S. van de Linde, “Photometry unlocks 3D information from 2D localization microscopy data,” Nat. Methods 14(1), 41–44 (2017).
[Crossref] [PubMed]

G. T. Dempsey, J. C. Vaughan, K. H. Chen, M. Bates, and X. Zhuang, “Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging,” Nat. Methods 8(12), 1027–1036 (2011).
[Crossref] [PubMed]

J. Fölling, M. Bossi, H. Bock, R. Medda, C. A. Wurm, B. Hein, S. Jakobs, C. Eggeling, and S. W. Hell, “Fluorescence nanoscopy by ground-state depletion and single-molecule return,” Nat. Methods 5(11), 943–945 (2008).
[Crossref] [PubMed]

M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006).
[Crossref] [PubMed]

J. Schindelin, I. Arganda-Carreras, E. Frise, V. Kaynig, M. Longair, T. Pietzsch, S. Preibisch, C. Rueden, S. Saalfeld, B. Schmid, J. Y. Tinevez, D. J. White, V. Hartenstein, K. Eliceiri, P. Tomancak, and A. Cardona, “Fiji: an open-source platform for biological-image analysis,” Nat. Methods 9(7), 676–682 (2012).
[Crossref] [PubMed]

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, “Faster STORM using compressed sensing,” Nat. Methods 9(7), 721–723 (2012).
[Crossref] [PubMed]

S. J. Holden, S. Uphoff, and A. N. Kapanidis, “DAOSTORM: an algorithm for high- density super-resolution microscopy,” Nat. Methods 8(4), 279–280 (2011).
[Crossref] [PubMed]

Nat. Photonics (2)

N. Bourg, C. Mayet, G. Dupuis, T. Barroca, P. Bon, S. Lécart, E. Fort, and S. Lévêque-Fort, “Direct optical nanoscopy with axially localized detection,” Nat. Photonics 9(9), 587–593 (2015).
[Crossref]

Y. Shechtman, L. E. Weiss, A. S. Backer, M. Y. Lee, and W. E. Moerner, “Multicolour localization microscopy by point-spread-function engineering,” Nat. Photonics 10(9), 590–594 (2016).
[Crossref] [PubMed]

Nature (1)

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015).
[Crossref] [PubMed]

Opt. Express (5)

Opt. Lett. (2)

Optica (3)

Phys. Rev. Lett. (1)

Y. Shechtman, S. J. Sahl, A. S. Backer, and W. E. Moerner, “Optimal point spread function design for 3D imaging,” Phys. Rev. Lett. 113(13), 133902 (2014).
[Crossref] [PubMed]

Proc. Natl. Acad. Sci. U.S.A. (1)

T. Dertinger, R. Colyer, G. Iyer, S. Weiss, and J. Enderlein, “Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI),” Proc. Natl. Acad. Sci. U.S.A. 106(52), 22287–22292 (2009).
[Crossref] [PubMed]

Sci. Rep. (2)

J. Min, C. Vonesch, H. Kirshner, L. Carlini, N. Olivier, S. Holden, S. Manley, J. C. Ye, and M. Unser, “FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data,” Sci. Rep. 4(1), 4577 (2015).
[Crossref] [PubMed]

T. Novák, T. Gajdos, J. Sinkó, G. Szabó, and M. Erdélyi, “TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy,” Sci. Rep. 7(1), 951 (2017).
[Crossref] [PubMed]

Science (2)

E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, and H. F. Hess, “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006).
[Crossref] [PubMed]

M. Bates, B. Huang, G. T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with photo-switchable fluorescent probes,” Science 317(5845), 1749–1753 (2007).
[Crossref] [PubMed]

Other (9)

T. Kim, S. Moon, and K. Xu, “Information-rich localization microscopy through machine learning,” https://www.biorxiv.org/content/early/2018/08/07/373878

C. Cabriel, N. Bourg, P. Jouchet, G. Dupuis, C. Leterrier, A. Baron, M.-A. Badet-Denisot, B. Vauzeilles, E. Fort, and S. Lévêque-Fort, “Combining 3D single molecule localization strategies for reproducible bioimaging,” https://www.biorxiv.org/content/early/2018/11/16/385799

W. Xie, J. A. Noble, and A. Zisserman, “Microscopy cell counting and detection with fully convolutional regression networks,” https://arxiv.org/abs/1708.03307
[Crossref]

A. Chakrabarti, “Learning Sensor Multiplexing Design through Back-propagation,” https://arxiv.org/abs/1605.07078

R. Horstmeyer, R. Y. Chen, B. Kappes, and B. Judkewitz, “Convolutional neural networks that teach microscopes how to image,” https://arxiv.org/abs/1709.07223

A. Vedaldi and K. Lenc, “MatConvNet - Convolutional Neural Networks for MATLAB,” in Proceedings of the 23rd ACM International Conference on Multimedia, (ACM, 2015), pp. 689–692.
[Crossref]

C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” https://arxiv.org/abs/1609.04802
[Crossref]

D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” https://arxiv.org/abs/1412.6980

S. Gazagnes, E. Soubies, and L. Blanc-Féraud, “High density molecule localization for super-resolution microscopy using CEL0 based sparse approximation,” in IEEE International Symposium on Biomedical Imaging, (ISBI, 2017), pp. 28–31.
[Crossref]

Supplementary Material (4)

NameDescription
» Visualization 1       Visualization 1. Net classification of ROIs containing red and green Qdots. Images are ordered by the net’s classification ranking from Green to Red.
» Visualization 2       Visualization 2. SLM-optimizer convergence. In each iteration, the SLM optimizer tests a voltage pattern (top left), which imparts a wavelength-dependent phase delay (top), producing increasingly-useful emission patterns on the detector (bottom).
» Visualization 3       Visualization 3. Net classification of ROIs containing red and green Qdots with a 2-color optimized mask. Images are ordered by the net’s classification ranking from Green to Red.
» Visualization 4       Visualization 4. Diffusing microsphere classification. Left: diffusing microspheres. Right: classification by the net matched the ground truth in all cases except highlighted the highlighted bead in purple, where the true color is green.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (21)

Fig. 1
Fig. 1 Color classification with neural nets. Patches containing red and green Qdots are extracted from a grayscale image and classified by a neural net. An example of a red emitter classification is depicted.
Fig. 2
Fig. 2 Qdot color determination using neural nets. (a) An epi-illumination microscope was used to examine Qdots on a glass coverslip. (b) A grayscale image of red and green Qdots. (c) A color image of the same sample obtained by imaging with two spectral filters. (d) The color-classified image from the neural net. (e) Average PSFs for red and green Qdots. (f) A histogram of the signal photons of the two Qdots. (g) A 3D scatter plot of the red and green Qdots showing the fitted parameters from an astigmatic Gaussian with two shape parameters (σ1 and σ2) and a variable angle (θ). (h) Classification percentage for emitters by various methods such as Nearest Neighbors, parameter thresholds, and matched filtering.
Fig. 3
Fig. 3 Design of an optimized SLM pattern using neural networks. (a) An SLM imparts a chromatically-dependent phase delay as a function of applied voltage. (b) A schematic depicting the process for creating an optimized phase mask consisting of 1. an SLM optimizer, used to generate the resulting PSFs for a particular SLM voltage pattern, and 2. a reconstruction net, which decodes the generated images. (c) The optimized SLM voltage pattern for color determination by a neural net. (d) The phase delay imparted to 565, 625, 705 and 800 nm light. (e) Simulated PSFs.
Fig. 4
Fig. 4 Four-color classification of emitters using the optimized phase mask. (a) Experimental images of four types of Qdots from a larger FOV (b). (c) Classification of Qdots in the same field of view (circles) overlaying an image of emitters which appeared in the raw data, artificially colored according to their ground-truth wavelengths. Closely-spaced Qdots, such as in the lower right, were compared to the brighter of the two emitters in the GT image. (d) Performance of color determination for the normal and optimized PSFs (N = 60, 120, 156, 29, respectively).
Fig. 5
Fig. 5 Color determination of moving microspheres. (a) Illustration of the diffusion chamber. (b) Schematic of the neural net classifying sequential groups of frames belonging to the same emitter as red or green. (c) The performance of the net as a function of the number of frames used for classification.
Fig. 6
Fig. 6 Microscope schematic using an extended imaging path.
Fig. 7
Fig. 7 Architecture of the color determination nets. (a) Two-color determination net, where the number of feature maps (n) and stride (s) of each convolutional layer are written for each experiment. Black denotes the Qdot net for the standard PSF and the super-resolution net. Purple text describes the net for moving beads. (b) The four-color determination net.
Fig. 8
Fig. 8 Optimizing voltage patterns on an SLM using neural nets. (a) The network architecture for SLM-optimizer. (b) The Reconstruction network architecture.
Fig. 9
Fig. 9 Optimized 2-color PSF. (a) The optimized SLM-voltage pattern for color determination by a neural net. (b) The phase delay imparted to 565 and 705 nm light. (c) Simulated PSFs. (d) The experimental PSFs measured with Qdots. (e) Pixel values of the cross section from experimentally measured PSFs. (f) Performance of color determination for the normal and optimized PSFs.
Fig. 10
Fig. 10 Optimized voltage masks for 2, 4 & 5-color classification.
Fig. 11
Fig. 11 t-SNE projections showing separability of 2-color Qdot data. Red and green spots represent the colors of the Qdots. (a) Analysis of the asymmetric 2D-Gaussian fit parameters shown in Fig. 2(g). (b) t-SNE projection of 11 × 11 pixel2 patches containing red and green Qdot emitters and (c) 11 × 11 pixel2 using the optimized PSFs. For both b & c, ten principle components were used; however, a wide range of selected parameter-values similarly showed separability of the two types of Qdots.
Fig. 12
Fig. 12 Localization precision in various noise conditions for the standard and net-optimized PSFs.
Fig. 13
Fig. 13 Two-color PSFs used for quantitative comparison.
Fig. 14
Fig. 14 Simulated size of the PSF with defocus.
Fig. 15
Fig. 15 Evaluating performance in various conditions. (a) Performance over different emitter densities (b) An example of a generated image and analysis with a density of 8.9 [emitters/μm2]. (c) Performance with various emitter brightnesses. (d) An example image and analysis of 800 signal photons per emitter and 200 background photons per pixel.
Fig. 16
Fig. 16 Evaluating performance over emitters’ wavelengths proximity. (a) Detection performance. (b) Detection & determination performance.
Fig. 17
Fig. 17 Evaluating performance over different numbers of colors. (a) Detection performance. (b) Color determination performance. (c) False alarms. (d) Localization performance. `
Fig. 18
Fig. 18 Optimized SLM results at low-signal conditions. (a) The optimized voltage pattern. (b) The red and green phase patterns. (c) The simulated red and green PSFs. (d) An example of a generated PSFs image with an extremely low SNR. (e) The ground truth of d. (f) Classification by the net.
Fig. 19
Fig. 19 Optimized SLM pattern for the moving-beads optical system. (a) The optimized voltage pattern. (b) The red and green phase patterns. (c) The simulated red and green PSFs.
Fig. 20
Fig. 20 Single-molecule classification using neural nets. (a) Simulated PSFs for Alexa 647 and Alexa 565. (b) Pixelated PSFs. (c) Ground truth image of two species-specific objects containing 25K fluorophores, each. (d) Representative single frame. (e) Classification success was calculated by the fraction of emitters successfully identified. (f) Pixels colored by the net according to density of emitters. (g-i) A fluorescently labeled HeLa cell. (g) Combined spectrally-filtered, diffraction-limited image. (h) Super-resolution reconstruction. (i) Reconstructed image, with pixels colored according to the net-classifications of individual localizations.
Fig. 21
Fig. 21 Chromatic focal shift for a Nikon Apo TIRF 100x/1.49 Oil objective. (a) Focal position versus Gaussian-fit size parameter, sigma. Image data for both Qdots was acquired simultaneously before identifying the color of objects in the FOV by adding a bandpass filter to the image path. (b-e) Intensity-normalized image data showing the standard PSF at two different focal positions for two Qdot emitters.

Tables (5)

Tables Icon

Table 1 Training parameters for the two-color nets

Tables Icon

Table 2 Training parameters for the four-color nets

Tables Icon

Table 3 Results of different classifiers

Tables Icon

Table 4 Comparison of PSFs in focus

Tables Icon

Table 5 Comparison of PSFs over a 500 nm z-range

Equations (11)

Equations on this page are rendered with MathJax. Learn more.

LOSS= 1 10N n [ GTlog(Z+ε)+(1GT)log(1Z+ε) ]
α(t)=1+γ t 2
PS F G = | F 1 { circl e G e i ϕ G } | 2
D=2f NA M
A G = λ G f pixe l HR
PS F R = | F 1 { circl e R e iPadding( ϕ R ) } | 2
Gra y HR = PS F R Source s R +PS F G Source s G 2
Image=Poissrnd{ λ Poisson }=Poissrnd{Gra y LR +background}
LOSS= 1 10N h,w,d,n [ MaskGTlog(Z+ε)+Mask(1GT)log(1Z+ε) ]
Δ λ A =5 nm ( λ 1 =565 nm, λ 2 =570 nm) Δ λ B =40 nm ( λ 1 =565 nm, λ 2 =605 nm) Δ λ C =140 nm ( λ 1 =565 nm, λ 2 =705 nm)
gra y HR = 1 N types n=1 N types PS F i Source s i

Metrics