Abstract

The application of machine learning in wavefront reconstruction has brought great benefits to real-time, non-invasive, deep tissue imaging in biomedical research. However, due to the diversity and heterogeneity of biological tissues, it is difficult to train the dataset with a unified model. In general, the utilization of some unified models will result in the specific sample falling outside the training set, leading to low accuracy of the machine learning model in some real applications. This paper proposes a sensorless wavefront reconstruction method based on transfer learning to overcome the domain shift introduced by the difference between the training set and the target test set. We build a weights-sharing two-stream convolutional neural network (CNN) framework for the prediction of Zernike coefficient, in which a large number of labeled randomly generated samples serve as the source-domain data and the unlabeled specific samples serve as the target-domain data at the same time. By training on massive labeled simulated data with domain adaptation to unlabeled target-domain data, the network shows better performance on the target tissue samples. Experimental results show that the accuracy of the proposed method is 18.5% higher than that of conventional CNN-based method and the peak intensities of the point spread function (PSF) are more than 20% higher with almost the same training time and processing time. The better compensation performance on target sample could have more advantages when handling complex aberrations, especially the aberrations caused by various histological characteristics, such as refractive index inhomogeneity and biological motion in biological tissues.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
    [Crossref]
  2. M. J. Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light: Sci. Appl. 3(4), e165 (2014).
    [Crossref]
  3. J.-W. Cha, J. Ballesta, and P. T. So, “Shack-Hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010).
    [Crossref]
  4. J. W. Evans, R. J. Zawadzki, S. M. Jones, S. S. Olivier, and J. S. Werner, “Error budget analysis for an adaptive optics optical coherence tomography system,” Opt. Express 17(16), 13768–13784 (2009).
    [Crossref]
  5. H. Hofer, N. Sredar, H. Queener, C. Li, and J. Porter, “Wavefront sensorless adaptive optics ophthalmoscopy in the human eye,” Opt. Express 19(15), 14160–14171 (2011).
    [Crossref]
  6. D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.
  7. S. Bonora and R. Zawadzki, “Wavefront sensorless modal deformable mirror correction in adaptive optics: optical coherence tomography,” Opt. Lett. 38(22), 4801–4804 (2013).
    [Crossref]
  8. Z. Xu, P. Yang, K. Hu, B. Xu, and H. Li, “Deep learning control model for adaptive optics systems,” Appl. Opt. 58(8), 1998–2009 (2019).
    [Crossref]
  9. S. W. Paine and J. R. Fienup, “Machine learning for improved image-based wavefront sensing,” Opt. Lett. 43(6), 1235–1238 (2018).
    [Crossref]
  10. G. Ju, X. Qi, H. Ma, and C. Yan, “Feature-based phase retrieval wavefront sensing approach using machine learning,” Opt. Express 26(24), 31767–31783 (2018).
    [Crossref]
  11. Y. Nishizaki, M. Valdivia, R. Horisaki, K. Kitaguchi, M. Saito, J. Tanida, and E. Vera, “Deep learning wavefront sensing,” Opt. Express 27(1), 240–251 (2019).
    [Crossref]
  12. Y. Jin, Y. Zhang, L. Hu, H. Huang, Q. Xu, X. Zhu, L. Huang, Y. Zheng, H.-L. Shen, W. Gong, and K. Si, “Machine learning guided rapid focusing with sensor-less aberration corrections,” Opt. Express 26(23), 30162–30171 (2018).
    [Crossref]
  13. Y. Zhang, C. Wu, Y. Song, K. Si, Y. Zheng, L. Hu, J. Chen, L. Tang, and W. Gong, “Machine learning based adaptive optics for doughnut-shaped beam,” Opt. Express 27(12), 16871–16881 (2019).
    [Crossref]
  14. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010).
    [Crossref]
  15. G. Csurka, “Domain adaptation for visual applications: A comprehensive survey,” arXiv preprint arXiv:1702.05374 (2017).
  16. M. A. Neil, M. J. Booth, and T. Wilson, “Closed-loop aberration correction by use of a modal Zernike wave-front sensor,” Opt. Lett. 25(15), 1083–1085 (2000).
    [Crossref]
  17. M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
    [Crossref]
  18. J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning (The MIT Press, 2009).
  19. B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in European conference on computer vision, (Springer, 2016), 443–450.
  20. M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
    [Crossref]

2019 (3)

2018 (3)

2017 (1)

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
[Crossref]

2014 (1)

M. J. Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light: Sci. Appl. 3(4), e165 (2014).
[Crossref]

2013 (1)

2011 (1)

2010 (2)

J.-W. Cha, J. Ballesta, and P. T. So, “Shack-Hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010).
[Crossref]

S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010).
[Crossref]

2009 (1)

2007 (1)

M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
[Crossref]

2004 (1)

M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
[Crossref]

2000 (1)

Ballesta, J.

J.-W. Cha, J. Ballesta, and P. T. So, “Shack-Hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010).
[Crossref]

Bifano, T. G.

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

Biss, D. P.

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

Bonora, S.

Booth, M. J.

M. J. Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light: Sci. Appl. 3(4), e165 (2014).
[Crossref]

M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
[Crossref]

M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
[Crossref]

M. A. Neil, M. J. Booth, and T. Wilson, “Closed-loop aberration correction by use of a modal Zernike wave-front sensor,” Opt. Lett. 25(15), 1083–1085 (2000).
[Crossref]

Cha, J.-W.

J.-W. Cha, J. Ballesta, and P. T. So, “Shack-Hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010).
[Crossref]

Chen, J.

Csurka, G.

G. Csurka, “Domain adaptation for visual applications: A comprehensive survey,” arXiv preprint arXiv:1702.05374 (2017).

Evans, J. W.

Fienup, J. R.

Gong, W.

Hofer, H.

Horisaki, R.

Hu, K.

Hu, L.

Huang, H.

Huang, L.

Ji, N.

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
[Crossref]

Jin, Y.

Jones, S. M.

Ju, G.

Kitaguchi, K.

Lawrence, N. D.

J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning (The MIT Press, 2009).

Li, C.

Li, H.

Lin, C. P.

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

Ma, H.

Neil, M. A.

M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
[Crossref]

M. A. Neil, M. J. Booth, and T. Wilson, “Closed-loop aberration correction by use of a modal Zernike wave-front sensor,” Opt. Lett. 25(15), 1083–1085 (2000).
[Crossref]

Nishizaki, Y.

Olivier, S. S.

Paine, S. W.

Pan, S. J.

S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010).
[Crossref]

Porter, J.

Qi, X.

Queener, H.

Quionero-Candela, J.

J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning (The MIT Press, 2009).

Saenko, K.

B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in European conference on computer vision, (Springer, 2016), 443–450.

Saito, M.

Schwaighofer, A.

J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning (The MIT Press, 2009).

Schwertner, M.

M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
[Crossref]

Shen, H.-L.

Si, K.

So, P. T.

J.-W. Cha, J. Ballesta, and P. T. So, “Shack-Hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010).
[Crossref]

Song, Y.

Sredar, N.

Sugiyama, M.

J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning (The MIT Press, 2009).

Sun, B.

B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in European conference on computer vision, (Springer, 2016), 443–450.

Tang, L.

Tanida, J.

Valdivia, M.

Vera, E.

Webb, R. H.

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

Werner, J. S.

Wilson, T.

M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
[Crossref]

M. A. Neil, M. J. Booth, and T. Wilson, “Closed-loop aberration correction by use of a modal Zernike wave-front sensor,” Opt. Lett. 25(15), 1083–1085 (2000).
[Crossref]

Wu, C.

Xu, B.

Xu, Q.

Xu, Z.

Yan, C.

Yang, P.

Yang, Q.

S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010).
[Crossref]

Zamiri, P.

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

Zawadzki, R.

Zawadzki, R. J.

Zhang, Y.

Zheng, Y.

Zhou, Y.

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

Zhu, X.

Appl. Opt. (1)

IEEE Trans. Knowl. Data Eng. (1)

S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010).
[Crossref]

J. Biomed. Opt. (1)

J.-W. Cha, J. Ballesta, and P. T. So, “Shack-Hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010).
[Crossref]

J. Microsc. (1)

M. Schwertner, M. J. Booth, M. A. Neil, and T. Wilson, “Measurement of specimen-induced aberrations of biological samples using phase stepping interferometry,” J. Microsc. 213(1), 11–19 (2004).
[Crossref]

Light: Sci. Appl. (1)

M. J. Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light: Sci. Appl. 3(4), e165 (2014).
[Crossref]

Nat. Methods (1)

N. Ji, “Adaptive optical fluorescence microscopy,” Nat. Methods 14(4), 374–380 (2017).
[Crossref]

Opt. Express (6)

Opt. Lett. (3)

Philos. Trans. R. Soc., A (1)

M. J. Booth, “Adaptive optics in microscopy,” Philos. Trans. R. Soc., A 365(1861), 2829–2843 (2007).
[Crossref]

Other (4)

D. P. Biss, R. H. Webb, Y. Zhou, T. G. Bifano, P. Zamiri, and C. P. Lin, “An adaptive optics biomicroscope for mouse retinal imaging,” in MEMS Adaptive Optics, (International Society for Optics and Photonics, 2007), 646703.

J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning (The MIT Press, 2009).

B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in European conference on computer vision, (Springer, 2016), 443–450.

G. Csurka, “Domain adaptation for visual applications: A comprehensive survey,” arXiv preprint arXiv:1702.05374 (2017).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1. The schematic diagram of the AO system based on transfer learning. P, linear polarizer plate; BS, non-polarizing beam splitter; SLM, spatial light modulator; L1 and L2, relay lenses; M, mirror; OBJ1 and OBJ2, objective lenses; L3, focusing lens; CMOS, complementary metal oxide semiconductor camera.
Fig. 2.
Fig. 2. The training architecture of the weights-sharing two-stream CNN framework. The source domain stream and the target domain stream have the same network structure, and the parameters are shared during training. The labeled randomly generated simulation dataset is input into the source domain network, and the unlabeled PSFs obtained with target samples are input into the target domain network. The CORAL is calculated from the features of the adaptation layer and forms the training loss function with the source domain’s prediction error MSE.
Fig. 3.
Fig. 3. Comparison of wavefront compensation between AlexNet [10] and the proposed network based on DAC on the randomly generated data of the first 22 Zernike coefficients. (a) Two groups of PSF results. PSF patterns and their corresponding phases after compensation are presented. The scale bar in the PSF pattern is 100 µm. (b) Comparison of central intensity profiles of PSFs in (a). (c) Comparison of the differences in detected amplitudes of Zernike mode coefficients in (a).
Fig. 4.
Fig. 4. Comparison of wavefront compensation between AlexNet [10] and the proposed network based on DAC on 1-mm-thick phantoms medium. (a) Three groups of PSF results. PSF patterns and their corresponding phases after compensation are presented. The scale bar in the PSF pattern is 100 µm. (b) Comparison of central intensity profiles of PSFs in (a).
Fig. 5.
Fig. 5. The T-SNE visualization. (a) Feature domain of AlexNet. (b) Feature domain of the proposed network based on DAC. 300 data from the source domain are presented with blue points and 300 data from the target domain are presented with red points.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

l = MSE + λ l C O R A L
MSE = 1 n s i = 1 n s ( y i s y ^ i s ) 2
l CORAL = 1 4 d 2 | | C s C t | | F 2