Abstract

We present a camera embedded data processing method for localization microscopy (LM) with faster detectors such as scientific complementary metal-oxide semiconductor (sCMOS) cameras. Based on the natural sparsity of single molecule images, this method utilizes the field programmable gate array chip inside a camera to identify and export only the regions containing active molecules instead of raw data. Through numerical simulation and experimental analysis, we found that this method can greatly reduce data volume (<10%) with negligible loss of useful information (<0.2%) at molecular densities <0.2molecules/μm2, thus significantly reducing the challenges of data transfer, storage, and analysis in LM.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
    [CrossRef]
  2. M. Baker, Nat. Methods 8, 1005 (2011).
    [CrossRef]
  3. J. W. Lichtman and W. Denk, Science 334, 618 (2011).
    [CrossRef]
  4. T. W. Quan, P. C. Li, F. Long, S. Q. Zeng, Q. M. Luo, P. N. Hedde, G. U. Nienhaus, and Z. L. Huang, Opt. Express 18, 11867 (2010).
    [CrossRef]
  5. R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
    [CrossRef]
  6. S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
    [CrossRef]
  7. H. Q. Ma, F. Long, S. Q. Zeng, and Z. L. Huang, Opt. Lett. 37, 2481 (2012).
    [CrossRef]
  8. R. J. Ober, S. Ram, and E. S. Ward, Biophys. J. 86, 1185 (2004).
    [CrossRef]
  9. L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
    [CrossRef]
  10. http://bigwww.epfl.ch/palm/?p=tubulin-af647 .
  11. http://bigwww.epfl.ch/smlm/challenge/index.html?p=datasets-real .

2012

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
[CrossRef]

H. Q. Ma, F. Long, S. Q. Zeng, and Z. L. Huang, Opt. Lett. 37, 2481 (2012).
[CrossRef]

2011

M. Baker, Nat. Methods 8, 1005 (2011).
[CrossRef]

J. W. Lichtman and W. Denk, Science 334, 618 (2011).
[CrossRef]

2010

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
[CrossRef]

T. W. Quan, P. C. Li, F. Long, S. Q. Zeng, Q. M. Luo, P. N. Hedde, G. U. Nienhaus, and Z. L. Huang, Opt. Express 18, 11867 (2010).
[CrossRef]

2004

R. J. Ober, S. Ram, and E. S. Ward, Biophys. J. 86, 1185 (2004).
[CrossRef]

Aufmkolk, S.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Baker, M.

M. Baker, Nat. Methods 8, 1005 (2011).
[CrossRef]

Dabauvalle, M. C.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Davidson, M.

G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
[CrossRef]

Denk, W.

J. W. Lichtman and W. Denk, Science 334, 618 (2011).
[CrossRef]

Elnatan, D.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
[CrossRef]

Fornasiero, E. F.

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

Hedde, P. N.

Henriques, R.

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

Holm, T.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Huang, B.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
[CrossRef]

Huang, Z. L.

Lelek, M.

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

Li, P. C.

Lichtman, J. W.

J. W. Lichtman and W. Denk, Science 334, 618 (2011).
[CrossRef]

Lippincott-Schwartz, J.

G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
[CrossRef]

Long, F.

Loschberger, A.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Luo, Q. M.

Ma, H. Q.

Manley, S.

G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
[CrossRef]

Mhlanga, M. M.

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

Nienhaus, G. U.

Ober, R. J.

R. J. Ober, S. Ram, and E. S. Ward, Biophys. J. 86, 1185 (2004).
[CrossRef]

Patterson, G.

G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
[CrossRef]

Quan, T. W.

Ram, S.

R. J. Ober, S. Ram, and E. S. Ward, Biophys. J. 86, 1185 (2004).
[CrossRef]

Sauer, M.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Valtorta, F.

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

van de Linde, S.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Ward, E. S.

R. J. Ober, S. Ram, and E. S. Ward, Biophys. J. 86, 1185 (2004).
[CrossRef]

Wolter, S.

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

Zeng, S. Q.

Zhang, W.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
[CrossRef]

Zhu, L.

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
[CrossRef]

Zimmer, C.

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

Annu. Rev. Phys. Chem.

G. Patterson, M. Davidson, S. Manley, and J. Lippincott-Schwartz, Annu. Rev. Phys. Chem. 61, 345 (2010).
[CrossRef]

Biophys. J.

R. J. Ober, S. Ram, and E. S. Ward, Biophys. J. 86, 1185 (2004).
[CrossRef]

Nat. Methods

L. Zhu, W. Zhang, D. Elnatan, and B. Huang, Nat. Methods 9, 721 (2012).
[CrossRef]

R. Henriques, M. Lelek, E. F. Fornasiero, F. Valtorta, C. Zimmer, and M. M. Mhlanga, Nat. Methods 7, 339 (2010).
[CrossRef]

S. Wolter, A. Loschberger, T. Holm, S. Aufmkolk, M. C. Dabauvalle, S. van de Linde, and M. Sauer, Nat. Methods 9, 1040 (2012).
[CrossRef]

M. Baker, Nat. Methods 8, 1005 (2011).
[CrossRef]

Opt. Express

Opt. Lett.

Science

J. W. Lichtman and W. Denk, Science 334, 618 (2011).
[CrossRef]

Other

http://bigwww.epfl.ch/palm/?p=tubulin-af647 .

http://bigwww.epfl.ch/smlm/challenge/index.html?p=datasets-real .

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1.
Fig. 1.

FPGA-based implementation for EDP. (a) General description of LM imaging using the EDP method. (b) Architecture of the EDP method inside FPGA. Here, the row buffers for pixel caching are built from the internal memory of FPGA, and the logic elements are used for implementing the function modules.

Fig. 2.
Fig. 2.

Performance of the EDP method in simulation data under different molecular densities. Here, the NOR is defined as the number of well separated single molecules divided by the total number of molecules. Arrow 1 shows the molecular density where NOR degrades to 50%, and Arrow 2 points out the molecule density where DRR exceed 100%. Note that the image size is 512×512 pixel.

Fig. 3.
Fig. 3.

Evaluating the performance of the EDP method with experimental data. (a) Overlay of a total number of 9990 raw image frames. (b) Overlay of all the extracted spots from the 9990 image frames. (c) Raw high density image frame. (d) Overlay of the extracted spot regions from the raw image shown in (c). (e) Intensity profiles along the dotted lines in (c) and (d). Note that the box size for spot region extraction is 9×9 pixels and the background in (d) was set to be 300 photons per pixel.

Metrics