Abstract

Deep neural networks have been used to map multi-modal, multi-photon microscopy measurements of a label-free tissue sample to its corresponding histologically stained brightfield microscope colour image. It is shown that the extra structural and functional contrasts provided by using two source modes, namely two-photon excitation microscopy and fluorescence lifetime imaging, result in a more faithful reconstruction of the target haematoxylin and eosin stained mode. This modal mapping procedure can aid histopathologists, since it provides access to unobserved imaging modalities, and translates the high-dimensional numerical data generated by multi-modal, multi-photon microscopy into traditionally accepted visual forms. Furthermore, by combining the strengths of traditional chemical staining and modern multi-photon microscopy techniques, modal mapping enables label-free, non-invasive studies of in vivo tissue samples or intravital microscopic imaging inside living animals. The results show that modal co-registration and the inclusion of spatial variations increase the visual accuracy of the mapped results.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Fourier ptychographic microscopy reconstruction with multiscale deep residual network

Jizhou Zhang, Tingfa Xu, Ziyi Shen, Yifan Qiao, and Yizhou Zhang
Opt. Express 27(6) 8612-8625 (2019)

Tri-modal microscopy with multiphoton and optical coherence microscopy/tomography for multi-scale and multi-contrast imaging

Shau Poh Chong, Tom Lai, Yifeng Zhou, and Shuo Tang
Biomed. Opt. Express 4(9) 1584-1594 (2013)

Lensless computational imaging through deep learning

Ayan Sinha, Justin Lee, Shuai Li, and George Barbastathis
Optica 4(9) 1117-1125 (2017)

References

  • View by:
  • |
  • |
  • |

  1. S. K. Suvarna, C. Layton, and J. D. Bancroft, Bancroft’s theory and practice of histological techniques, 7th Edition (Churchill Livingstone Elsevier, 2013).
  2. C. Lefort, “A review of biomedical multiphoton microscopy and its laser sources,” J. Phys. D Appl. Phys. 50, 423001 (2017).
    [Crossref]
  3. E. E. Hoover and J. A. Squier, “Advances in multiphoton microscopy technology,” Nat. Photonics 7, 93–101 (2013).
    [Crossref] [PubMed]
  4. P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
    [Crossref]
  5. P. J. Campagnola and L. M. Loew, “Second harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms,” Nat. Biotechnol. 21, 1356–1360 (2003).
    [Crossref] [PubMed]
  6. L. G. Rodriguez, S. J. Lockett, and G. R. Holtom, “Coherent anti-Stokes Raman scattering microscopy: A biological review,” Cytom. Part A 69A, 779–791 (2006).
    [Crossref]
  7. W. Becker, “Fluorescence lifetime imaging – techniques and applications,” J. Microsc. 247(2), 119–136 (2012).
    [Crossref] [PubMed]
  8. H. G. Breunig, H. Studier, and K. König, “Multiphoton excitation characteristics of cellular fluorophores of human skin in vivo,” Opt. Express 18(8), 7857–7871 (2010).
    [Crossref] [PubMed]
  9. E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
    [Crossref] [PubMed]
  10. Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
    [Crossref] [PubMed]
  11. H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
    [Crossref] [PubMed]
  12. A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
    [Crossref]
  13. A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
    [Crossref] [PubMed]
  14. J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
    [Crossref]
  15. D.S. Gareau, “Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology,” J. Biomed. Opt. 14(3), 034050 (2009).
    [Crossref] [PubMed]
  16. J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
    [Crossref] [PubMed]
  17. M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
    [Crossref] [PubMed]
  18. H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE T. Med. Imaging 35(5), 1153–1159 (2016).
    [Crossref]
  19. G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
    [Crossref] [PubMed]
  20. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
    [Crossref] [PubMed]
  21. Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
    [Crossref]
  22. S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
    [Crossref]
  23. D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).
  24. P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
    [Crossref] [PubMed]
  25. Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).
  26. J. B. A. Maintz and M. A. Viergever, “A survey of medical image registration,” Med. Image Anal. 2(1), 1–36 (1998).
    [Crossref]
  27. M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
    [Crossref] [PubMed]
  28. Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
    [Crossref] [PubMed]
  29. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
    [Crossref]
  30. M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
    [Crossref]
  31. G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
    [Crossref]
  32. M. E. Celebi, “Improving the performance of k-means for colour quantization,” Image Vision Comput. 29(4), 260–271 (2011).
    [Crossref]
  33. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
    [Crossref]
  34. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv:1409.1556 (2014).
  35. D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

2018 (1)

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

2017 (5)

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
[Crossref]

C. Lefort, “A review of biomedical multiphoton microscopy and its laser sources,” J. Phys. D Appl. Phys. 50, 423001 (2017).
[Crossref]

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

2016 (5)

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE T. Med. Imaging 35(5), 1153–1159 (2016).
[Crossref]

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
[Crossref]

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

2015 (2)

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

2014 (1)

2013 (2)

E. E. Hoover and J. A. Squier, “Advances in multiphoton microscopy technology,” Nat. Photonics 7, 93–101 (2013).
[Crossref] [PubMed]

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

2012 (2)

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

W. Becker, “Fluorescence lifetime imaging – techniques and applications,” J. Microsc. 247(2), 119–136 (2012).
[Crossref] [PubMed]

2011 (3)

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

M. E. Celebi, “Improving the performance of k-means for colour quantization,” Image Vision Comput. 29(4), 260–271 (2011).
[Crossref]

2010 (1)

2009 (1)

D.S. Gareau, “Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology,” J. Biomed. Opt. 14(3), 034050 (2009).
[Crossref] [PubMed]

2006 (1)

L. G. Rodriguez, S. J. Lockett, and G. R. Holtom, “Coherent anti-Stokes Raman scattering microscopy: A biological review,” Cytom. Part A 69A, 779–791 (2006).
[Crossref]

2004 (2)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
[Crossref]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

2003 (1)

P. J. Campagnola and L. M. Loew, “Second harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms,” Nat. Biotechnol. 21, 1356–1360 (2003).
[Crossref] [PubMed]

2000 (1)

P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
[Crossref]

1998 (1)

J. B. A. Maintz and M. A. Viergever, “A survey of medical image registration,” Med. Image Anal. 2(1), 1–36 (1998).
[Crossref]

1981 (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

Alves, V.

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
[Crossref]

Amgad, M.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Antoniadou, E.

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Ba, J.

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

Bancroft, J. D.

S. K. Suvarna, C. Layton, and J. D. Bancroft, Bancroft’s theory and practice of histological techniques, 7th Edition (Churchill Livingstone Elsevier, 2013).

Barnholtz-Sloan, J. S.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Beck, A. H.

D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).

Becker, W.

W. Becker, “Fluorescence lifetime imaging – techniques and applications,” J. Microsc. 247(2), 119–136 (2012).
[Crossref] [PubMed]

Bejnordi, B. E.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Bellini, V.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Benati, E.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Bengio, Y.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Benveniste, A. P.

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

Berland, K. M.

P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
[Crossref]

Bini, J.

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

Bjartell, A.

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

Boppart, M. D.

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Boppart, S. A.

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Borsari, S.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Bovik, A. C.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
[Crossref]

Bower, A. J.

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

Brat, D. J.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Breunig, H. G.

Campagnola, P. J.

P. J. Campagnola and L. M. Loew, “Second harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms,” Nat. Biotechnol. 21, 1356–1360 (2003).
[Crossref] [PubMed]

Celebi, M. E.

M. E. Celebi, “Improving the performance of k-means for colour quantization,” Image Vision Comput. 29(4), 260–271 (2011).
[Crossref]

Chaney, E. J.

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Chidester, B.

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

Ciompi, F.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Connolly, J. L.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Cooper, L. A. D.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Dantus, M.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

DeVolder, R.

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

DiMarzio, C.

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

Do, M. N.

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

Dobbs, J.

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

Dong, C. Y.

P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
[Crossref]

Dunsby, C.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Edsjö, A.

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

Faulkner-Jones, B. E.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Ferrari, C.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

French, P.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Fujimoto, J. G.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Gareau, D.S.

D.S. Gareau, “Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology,” J. Biomed. Opt. 14(3), 034050 (2009).
[Crossref] [PubMed]

Gargeya, R.

D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).

Ghafoorian, M.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Giacomelli, M. G.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Göröcs, Z.

Graf, B. W.

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Greenspan, H.

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE T. Med. Imaging 35(5), 1153–1159 (2016).
[Crossref]

Guanti, M.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Guardoli, D.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Günaydin, H.

Günayfin, H.

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

Gutman, D. A.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Hazelwood, V.

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

Helczynski, L.

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

Hinton, G.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Holtom, G. R.

L. G. Rodriguez, S. J. Lockett, and G. R. Holtom, “Coherent anti-Stokes Raman scattering microscopy: A biological review,” Cytom. Part A 69A, 779–791 (2006).
[Crossref]

Hoover, E. E.

E. E. Hoover and J. A. Squier, “Advances in multiphoton microscopy technology,” Nat. Photonics 7, 93–101 (2013).
[Crossref] [PubMed]

Hornegger, J.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Husvogt, L.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Irshad, H.

D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).

Khosla, A.

D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).

Kingma, D. P.

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

Klein, S.

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

Koenig, K.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Kong, H.

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

König, K.

Kooi, T.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Krishnamurthy, S.

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

Kyrish, M.

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

Lægsgaard, J.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Layton, C.

S. K. Suvarna, C. Layton, and J. D. Bancroft, Bancroft’s theory and practice of histological techniques, 7th Edition (Churchill Livingstone Elsevier, 2013).

LeCun, Y.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Lefort, C.

C. Lefort, “A review of biomedical multiphoton microscopy and its laser sources,” J. Phys. D Appl. Phys. 50, 423001 (2017).
[Crossref]

Li, J.

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

Lippolis, G.

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

Litjens, G.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Liu, Y.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Lockett, S. J.

L. G. Rodriguez, S. J. Lockett, and G. R. Holtom, “Coherent anti-Stokes Raman scattering microscopy: A biological review,” Cytom. Part A 69A, 779–791 (2006).
[Crossref]

Loew, L. M.

P. J. Campagnola and L. M. Loew, “Second harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms,” Nat. Biotechnol. 21, 1356–1360 (2003).
[Crossref] [PubMed]

Lowe, D. G.

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

Lyngsø, J. K.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Mahmassani, Z.

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Maintz, J. B. A.

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

J. B. A. Maintz and M. A. Viergever, “A survey of medical image registration,” Med. Image Anal. 2(1), 1–36 (1998).
[Crossref]

Marjanovic, M.

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Masters, B. R.

P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
[Crossref]

Mobadersany, P.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Murphy, K.

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

Nehal, K.

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

Overgaard, N. C.

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

Ozcan, A.

Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
[Crossref]

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

Pellacani, G.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Pereira, S.

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
[Crossref]

Pinto, A.

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
[Crossref]

Pluim, J. P. W.

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

Ponti, G.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Rajadhyaksha, M.

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

Richards-Kortum, R.

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

Rivenson, Y.

Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
[Crossref]

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

Rodriguez, L. G.

L. G. Rodriguez, S. J. Lockett, and G. R. Holtom, “Coherent anti-Stokes Raman scattering microscopy: A biological review,” Cytom. Part A 69A, 779–791 (2006).
[Crossref]

Sanchez, C. I.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Schianchi, S.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Seidenari, S.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Setio, A. A. A.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Sheikh, H. R.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
[Crossref]

Silva, C. A.

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
[Crossref]

Simoncelli, E. P.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
[Crossref]

Simonyan, K.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv:1409.1556 (2014).

So, P. T. C.

P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
[Crossref]

Spain, J.

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

Squier, J. A.

E. E. Hoover and J. A. Squier, “Advances in multiphoton microscopy technology,” Nat. Photonics 7, 93–101 (2013).
[Crossref] [PubMed]

Staring, M.

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

Studier, H.

Summers, R. M.

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE T. Med. Imaging 35(5), 1153–1159 (2016).
[Crossref]

Suvarna, S. K.

S. K. Suvarna, C. Layton, and J. D. Bancroft, Bancroft’s theory and practice of histological techniques, 7th Edition (Churchill Livingstone Elsevier, 2013).

Talbot, C.

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Tu, H.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Turchinovich, D.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

van der Laak, J. A. W. M.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

van Ginneken, B.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE T. Med. Imaging 35(5), 1153–1159 (2016).
[Crossref]

Vardeh, H.

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

Vega, J. E. V.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Viergever, M. A.

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

J. B. A. Maintz and M. A. Viergever, “A survey of medical image registration,” Med. Image Anal. 2(1), 1–36 (1998).
[Crossref]

Wang, D.

D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).

Wang, H.

Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
[Crossref]

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

Wang, Z.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
[Crossref]

Wei, Z.

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

Wilson, W. L.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Xu, B.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Yang, W.

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

You, S.

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Yousefi, S.

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Zhang, Y.

Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4(11), 1437–1443 (2017).
[Crossref]

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

Zhao, Y.

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Y. Zhao, M. Marjanovic, E. J. Chaney, B. W. Graf, Z. Mahmassani, M. D. Boppart, and S. A. Boppart, “Longitudinal label-free tracking of cell death dynamics in living engineered human skin tissue with a multimodal microscope,” Biomed. Opt. Express 5(10), 3699–3716 (2014).
[Crossref] [PubMed]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

Zisserman, A.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv:1409.1556 (2014).

Annu. Rev. Biomed. Eng. (1)

P. T. C. So, C. Y. Dong, B. R. Masters, and K. M. Berland, “Two-photon excitation fluorescence microscopy,” Annu. Rev. Biomed. Eng. 2, 399–429 (2000).
[Crossref]

Biomed. Opt. Express (1)

BMC Cancer (1)

G. Lippolis, A. Edsjö, L. Helczynski, A. Bjartell, and N. C. Overgaard, “Automatic registration of multi-modal microscopy images for integrative analysis of prostrate tissue sections,” BMC Cancer 13, 408 (2013).
[Crossref]

Breast Cancer Res. Treat. (1)

J. Dobbs, S. Krishnamurthy, M. Kyrish, A. P. Benveniste, W. Yang, and R. Richards-Kortum, “Confocal fluorescence microscopy for rapid evaluation of invasive tumor cellularity of inflammatory breast carcinoma core needle biopsies,” Breast Cancer Res. Treat. 149(1), 303–310 (2015).
[Crossref]

Commun. ACM (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography,” Commun. ACM 24(6), 381–395 (1981).
[Crossref]

Cytom. Part A (1)

L. G. Rodriguez, S. J. Lockett, and G. R. Holtom, “Coherent anti-Stokes Raman scattering microscopy: A biological review,” Cytom. Part A 69A, 779–791 (2006).
[Crossref]

IEEE T. Image Process. (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE T. Image Process. 13(4), 600–612 (2004).
[Crossref]

IEEE T. Med. Imaging (2)

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE T. Med. Imaging 35(5), 1153–1159 (2016).
[Crossref]

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumour segmentation using convolutional neural networks in MRI images,” IEEE T. Med. Imaging 35(5), 1240–1251 (2016).
[Crossref]

Image Vision Comput. (1)

M. E. Celebi, “Improving the performance of k-means for colour quantization,” Image Vision Comput. 29(4), 260–271 (2011).
[Crossref]

Int. J. Comput. Vis. (1)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

J. Biomed. Opt. (2)

D.S. Gareau, “Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology,” J. Biomed. Opt. 14(3), 034050 (2009).
[Crossref] [PubMed]

J. Bini, J. Spain, K. Nehal, V. Hazelwood, C. DiMarzio, and M. Rajadhyaksha, “Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance,” J. Biomed. Opt. 16(7), 076008 (2011).
[Crossref] [PubMed]

J. Biophotonics (2)

A. J. Bower, M. Marjanovic, Y. Zhao, J. Li, E. J. Chaney, and S. A. Boppart, “Label-free in vivo cellular-level detection and imaging of apoptosis,” J. Biophotonics 10(1), 143–150 (2017).
[Crossref]

Y. Zhao, B. W. Graf, E. J. Chaney, Z. Mahmassani, E. Antoniadou, R. DeVolder, H. Kong, M. D. Boppart, and S. A. Boppart, “Integrated multimodal optical microscopy for structural and functional imaging of engineered and natural skin,” J. Biophotonics 5(5–6), 437–448 (2012).
[Crossref] [PubMed]

J. Microsc. (1)

W. Becker, “Fluorescence lifetime imaging – techniques and applications,” J. Microsc. 247(2), 119–136 (2012).
[Crossref] [PubMed]

J. Phys. D Appl. Phys. (1)

C. Lefort, “A review of biomedical multiphoton microscopy and its laser sources,” J. Phys. D Appl. Phys. 50, 423001 (2017).
[Crossref]

Med. Image Anal. (3)

J. B. A. Maintz and M. A. Viergever, “A survey of medical image registration,” Med. Image Anal. 2(1), 1–36 (1998).
[Crossref]

M. A. Viergever, J. B. A. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. W. Pluim, “A survey of medical image registration - under review,” Med. Image Anal. 33, 140–144 (2016).
[Crossref] [PubMed]

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sanchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017).
[Crossref] [PubMed]

Nat. Biotechnol. (1)

P. J. Campagnola and L. M. Loew, “Second harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms,” Nat. Biotechnol. 21, 1356–1360 (2003).
[Crossref] [PubMed]

Nat. Photonics (2)

E. E. Hoover and J. A. Squier, “Advances in multiphoton microscopy technology,” Nat. Photonics 7, 93–101 (2013).
[Crossref] [PubMed]

H. Tu, Y. Liu, D. Turchinovich, M. Marjanovic, J. K. Lyngsø, J. Lægsgaard, E. J. Chaney, Y. Zhao, S. You, W. L. Wilson, B. Xu, M. Dantus, and S. A. Boppart, “Stain-free histopathology by programmable supercontinuum pulses,” Nat. Photonics 10, 534–541 (2016).
[Crossref] [PubMed]

Nature (1)

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
[Crossref] [PubMed]

Opt. Express (1)

Optica (1)

PLoS ONE (1)

M. G. Giacomelli, L. Husvogt, H. Vardeh, B. E. Faulkner-Jones, J. Hornegger, J. L. Connolly, and J. G. Fujimoto, “Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging,” PLoS ONE 11(8), e0159337 (2016).
[Crossref] [PubMed]

PNAS (1)

P. Mobadersany, S. Yousefi, M. Amgad, D. A. Gutman, J. S. Barnholtz-Sloan, J. E. V. Vega, D. J. Brat, and L. A. D. Cooper, “Predicting cancer outcomes from histology and genomics using convolutional networks,” PNAS 115(13), E2970–E2979 (2018).
[Crossref] [PubMed]

Quant. Imaging Med. Surg. (1)

A. J. Bower, B. Chidester, J. Li, Y. Zhao, M. Marjanovic, E. J. Chaney, M. N. Do, and S. A. Boppart, “A quantitative framework for the analysis of multimodal optical microscopy images,” Quant. Imaging Med. Surg. 7(1), 24–37 (2017).
[Crossref] [PubMed]

Skin Res. Technol. (1)

E. Benati, V. Bellini, S. Borsari, C. Dunsby, C. Ferrari, P. French, M. Guanti, D. Guardoli, K. Koenig, G. Pellacani, G. Ponti, S. Schianchi, C. Talbot, and S. Seidenari, “Quantitative evaluation of healthy epidermis by means of multiphoton microscopy and fluorescence lifetime imaging microscopy,” Skin Res. Technol. 17(3), 295–303 (2011).
[Crossref] [PubMed]

Other (5)

Y. Rivenson, H. Wang, Z. Wei, Y. Zhang, H. Günayfin, and A. Ozcan, “Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue,” arXiv:1803.11293 (2018).

D. Wang, A. Khosla, R. Gargeya, H. Irshad, and A. H. Beck, “Deep learning for identifying metastatic breast cancer,” arXiv:1606.05718 (2016).

S. K. Suvarna, C. Layton, and J. D. Bancroft, Bancroft’s theory and practice of histological techniques, 7th Edition (Churchill Livingstone Elsevier, 2013).

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv:1409.1556 (2014).

D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” arXiv:1412.6980 (2014).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1 Schematic of the modal mapping process to transfer the combined TPEF and FLIM observations, of a label-free rat liver tissue sample, to their corresponding H&E-stained brightfield microscope image. To provide contrast in this figure, the 17 MPM channels have been min-max scaled. The length-scale bar in the stained image corresponds to 30 μm.
Fig. 2
Fig. 2 Schematic of the multi-modal image co-registration scheme used to spatially warp the H&E-stained brightfield images onto the MPM modalities.
Fig. 3
Fig. 3 Application of colour reduction on the H&E-stained images, showing the (a) variation of the NMSE and SSIM between the reduced and (b) original 24-bit colour ground-truth images with increasing number of colours; and (c) the resulting 16 colour reduced image.
Fig. 4
Fig. 4 Schematics of the DNN architectures used to map the multiple source MPM modalities onto the target reduced colour H&E-stained image: (a) FCNN-p2p comprising a fully connected classifier, and (b) VGG-a2p with a CNN spatial feature detector frontend and a fully connected classifier backend.
Fig. 5
Fig. 5 Reconstructed H&E-stained images generated by different DNNs when applied to unseen test source datasets. The NMSE and SSIM accuracies are relative to the 24-bit colour ground-truth image. As can be seen, the reconstruction fidelity increases when multiple source modes (TPEF+FLIM) are used, and when spatial variations are learnt (VGG-a2p).
Fig. 6
Fig. 6 Modal mapping results when DNNs, trained to reconstruct H&E-stained images from multi-modal MPM observations of a rat liver tissue sample, are applied to (a) mouse ovary, and (b) rat mammary tissues. The length-scale bars in the ground-truth stained images correspond to 30 μm.

Metrics