Abstract

Deep neural networks (DNNs) are used to classify and reconstruct the input images from the intensity of the speckle patterns that result after the inputs are propagated through multimode fiber (MMF). We were able to demonstrate this result for fibers up to 1 km long by training the DNNs with a database of 16,000 handwritten digits. Better recognition accuracy was obtained when the DNNs were trained to first reconstruct the input and then classify based on the recovered image. We observed remarkable robustness against environmental instabilities and tolerance to deviations of the input pattern from the patterns with which the DNN was originally trained.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Lensless computational imaging through deep learning

Ayan Sinha, Justin Lee, Shuai Li, and George Barbastathis
Optica 4(9) 1117-1125 (2017)

Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media

Yunzhe Li, Yujia Xue, and Lei Tian
Optica 5(10) 1181-1190 (2018)

References

  • View by:
  • |
  • |
  • |

  1. D. J. Richardson, J. M. Fini, and L. E. Nelson, “Space-division multiplexing in optical fibres,” Nat. Photonics 7, 354–362 (2013).
    [Crossref]
  2. B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
    [Crossref]
  3. E. Spitz and A. Wertz, “Transmission des images à travers une fibre optique,” Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B 264, 1015 (1967).
  4. A. Gover, C. P. Lee, and A. Yariv, “Direct transmission of pictorial information in multimode optical fibers,” J. Opt. Soc. Am. 66, 306–311 (1976).
    [Crossref]
  5. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
    [Crossref]
  6. A. M. Caravaca-Aguirre, E. Niv, D. B. Conkey, and R. Piestun, “Real-time resilient focusing through a bending multimode fiber,” Opt. Express 21, 12881–12887 (2013).
    [Crossref]
  7. D. B. Conkey, N. Stasio, E. E. Morales-Delgado, M. Romito, C. Moser, and D. Psaltis, “Lensless two-photon imaging through a multicore fiber with coherence-gated digital phase conjugation,” J. Biomed. Opt. 21, 045002 (2016).
    [Crossref]
  8. D. B. Conkey, E. Kakkava, T. Lanvin, D. Loterie, N. Stasio, E. Morales-Delgado, C. Moser, and D. Psaltis, “High power, ultrashort pulse control through a multi-core fiber for ablation,” Opt. Express 25, 11491–11502 (2017).
    [Crossref]
  9. I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “High-resolution, lensless endoscope based on digital scanning through a multimode optical fiber,” Biomed. Opt. Express 4, 260–270 (2013).
    [Crossref]
  10. T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun. 3, 1027 (2012).
    [Crossref]
  11. E. R. Andresen, G. Bouwmans, S. Monneret, and H. Rigneault, “Toward endoscopes with no distal optics: video-rate scanning microscopy through a fiber bundle,” Opt. Lett. 38, 609–611 (2013).
    [Crossref]
  12. S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
    [Crossref]
  13. R. N. Mahalati, D. Askarov, J. P. Wilde, and J. M. Kahn, “Adaptive control of input field to achieve desired output intensity profile in multimode fiber with random mode coupling,” Opt. Express 20, 14321–14337 (2012).
    [Crossref]
  14. K. Aoki, A. Okamoto, Y. Wakayama, A. Tomita, and S. Honma, “Selective multimode excitation using volume holographic mode multiplexer,” Opt. Lett. 38, 769–771 (2013).
    [Crossref]
  15. R. Di Leonardo and S. Bianchi, “Hologram transmission through multi-mode optical fibers,” Opt. Express 19, 247–254 (2011).
    [Crossref]
  16. I. M. Vellekoop, “Feedback-based wavefront shaping,” Opt. Express 23, 12189–12206 (2015).
    [Crossref]
  17. S. Aisawa, K. Noguchi, and T. Matsumoto, “Remote image classification through multimode optical fiber using a neural network,” Opt. Lett. 16, 645–647 (1991).
    [Crossref]
  18. R. K. Marusarz and M. R. Sayeh, “Neural network-based multimode fiber-optic information transmission,” Appl. Opt. 40, 219–227 (2001).
    [Crossref]
  19. D. Psaltis, D. Brady, and K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
    [Crossref]
  20. Y. Owechko, G. J. Dunning, E. Marom, and B. H. Soffer, “Holographic associative memory with nonlinearities in the correlation domain,” Appl. Opt. 26, 1900–1910 (1987).
    [Crossref]
  21. G. Van der Sande, D. Brunner, and M. C. Soriano, “Advances in photonic reservoir computing,” Nanophotonics 6, 561–576 (2017).
    [Crossref]
  22. A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
    [Crossref]
  23. Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
    [Crossref]
  24. U. S. Kamilov, I. N. Papadopoulos, M. H. Shoreh, A. Goy, C. Vonesch, M. Unser, and D. Psaltis, “Learning approach to optical tomography,” Optica 2, 517–522 (2015).
    [Crossref]
  25. A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless computational imaging through deep learning,” Optica 4, 1117–1125 (2017).
    [Crossref]
  26. Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” Optica 4, 1437–1443 (2017).
    [Crossref]
  27. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations (ICLR) (2015).
  28. O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds. (Springer, 2015), Vol. 9351, pp. 234–241.
  29. A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.
  30. D. Psaltis, A. Sideris, and A. A. Yamamura, “A multilayered neural network controller,” IEEE Control Syst. Mag. 8(2), 17–21 (1988).
    [Crossref]
  31. R. Guenard, K. Krupa, R. Dupiol, M. Fabert, A. Bendahmane, V. Kermene, A. Desfarges-Berthelemot, J. L. Auguste, A. Tonello, A. Barthélémy, G. Millot, S. Wabnitz, and V. Couderc, “Nonlinear beam self-cleaning in a coupled cavity composite laser based on multimode fiber,” Opt. Express 25, 22219–22227 (2017).
    [Crossref]
  32. Z. Liu, L. G. Wright, D. N. Christodoulides, and F. W. Wise, “Kerr self-cleaning of femtosecond-pulsed beams in graded-index multimode fiber,” Opt. Lett. 41, 3675–3678 (2016).
    [Crossref]

2017 (7)

2016 (2)

Z. Liu, L. G. Wright, D. N. Christodoulides, and F. W. Wise, “Kerr self-cleaning of femtosecond-pulsed beams in graded-index multimode fiber,” Opt. Lett. 41, 3675–3678 (2016).
[Crossref]

D. B. Conkey, N. Stasio, E. E. Morales-Delgado, M. Romito, C. Moser, and D. Psaltis, “Lensless two-photon imaging through a multicore fiber with coherence-gated digital phase conjugation,” J. Biomed. Opt. 21, 045002 (2016).
[Crossref]

2015 (2)

2013 (5)

2012 (3)

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun. 3, 1027 (2012).
[Crossref]

R. N. Mahalati, D. Askarov, J. P. Wilde, and J. M. Kahn, “Adaptive control of input field to achieve desired output intensity profile in multimode fiber with random mode coupling,” Opt. Express 20, 14321–14337 (2012).
[Crossref]

2011 (2)

R. Di Leonardo and S. Bianchi, “Hologram transmission through multi-mode optical fibers,” Opt. Express 19, 247–254 (2011).
[Crossref]

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

2005 (1)

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

2001 (1)

1991 (1)

1988 (2)

D. Psaltis, D. Brady, and K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
[Crossref]

D. Psaltis, A. Sideris, and A. A. Yamamura, “A multilayered neural network controller,” IEEE Control Syst. Mag. 8(2), 17–21 (1988).
[Crossref]

1987 (1)

1976 (1)

1967 (1)

E. Spitz and A. Wertz, “Transmission des images à travers une fibre optique,” Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B 264, 1015 (1967).

Aisawa, S.

Andresen, E. R.

Aoki, K.

Askarov, D.

Auguste, J. L.

Baehr-Jones, T.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Barbastathis, G.

Barthélémy, A.

Bendahmane, A.

Bianchi, S.

Boccara, A. C.

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

Bouwmans, G.

Brady, D.

Brox, T.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds. (Springer, 2015), Vol. 9351, pp. 234–241.

Brunner, D.

G. Van der Sande, D. Brunner, and M. C. Soriano, “Advances in photonic reservoir computing,” Nanophotonics 6, 561–576 (2017).
[Crossref]

Caltagirone, F.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Caravaca-Aguirre, A. M.

Carron, I.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Cheung, E. L. M.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Choi, W.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Choi, Y.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Christodoulides, D. N.

Cižmár, T.

T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun. 3, 1027 (2012).
[Crossref]

Cocker, E. D.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Conkey, D. B.

Couderc, V.

Dasari, R. R.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Daudet, L.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

de Lima, T. F.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Desfarges-Berthelemot, A.

Dholakia, K.

T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun. 3, 1027 (2012).
[Crossref]

Di Leonardo, R.

Drémeau, A.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Dunning, G. J.

Dupiol, R.

Englund, D.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Fabert, M.

Fang-Yen, C.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Farahi, S.

Fini, J. M.

D. J. Richardson, J. M. Fini, and L. E. Nelson, “Space-division multiplexing in optical fibres,” Nat. Photonics 7, 354–362 (2013).
[Crossref]

Fink, M.

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

Fischer, P.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds. (Springer, 2015), Vol. 9351, pp. 234–241.

Flusberg, B. A.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Gigan, S.

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Göröcs, Z.

Gover, A.

Goy, A.

Guenard, R.

Günaydin, H.

Harris, N. C.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Hochberg, M.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Honma, S.

Jung, J. C.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Kahn, J. M.

Kakkava, E.

Kamilov, U. S.

Kermene, V.

Kim, M.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Krupa, K.

Krzakala, F.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Lanvin, T.

Larochelle, H.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Lee, C. P.

Lee, J.

Lee, K. J.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Lerosey, G.

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

Li, S.

Liu, Z.

Loterie, D.

Mahalati, R. N.

Marom, E.

Marusarz, R. K.

Matsumoto, T.

Millot, G.

Monneret, S.

Morales-Delgado, E.

Morales-Delgado, E. E.

D. B. Conkey, N. Stasio, E. E. Morales-Delgado, M. Romito, C. Moser, and D. Psaltis, “Lensless two-photon imaging through a multicore fiber with coherence-gated digital phase conjugation,” J. Biomed. Opt. 21, 045002 (2016).
[Crossref]

Moser, C.

Nahmias, M. A.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Nelson, L. E.

D. J. Richardson, J. M. Fini, and L. E. Nelson, “Space-division multiplexing in optical fibres,” Nat. Photonics 7, 354–362 (2013).
[Crossref]

Niv, E.

Noguchi, K.

Okamoto, A.

Owechko, Y.

Ozcan, A.

Papadopoulos, I. N.

Piestun, R.

Piyawattanametha, W.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Popoff, S. M.

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

Prabhu, M.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Prucnal, P. R.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Psaltis, D.

Richardson, D. J.

D. J. Richardson, J. M. Fini, and L. E. Nelson, “Space-division multiplexing in optical fibres,” Nat. Photonics 7, 354–362 (2013).
[Crossref]

Rigneault, H.

Rivenson, Y.

Romito, M.

D. B. Conkey, N. Stasio, E. E. Morales-Delgado, M. Romito, C. Moser, and D. Psaltis, “Lensless two-photon imaging through a multicore fiber with coherence-gated digital phase conjugation,” J. Biomed. Opt. 21, 045002 (2016).
[Crossref]

Ronneberger, O.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds. (Springer, 2015), Vol. 9351, pp. 234–241.

Saade, A.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Sayeh, M. R.

Schnitzer, M. J.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Shastri, B. J.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Shen, Y.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Shoreh, M. H.

Sideris, A.

D. Psaltis, A. Sideris, and A. A. Yamamura, “A multilayered neural network controller,” IEEE Control Syst. Mag. 8(2), 17–21 (1988).
[Crossref]

Simonyan, K.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations (ICLR) (2015).

Sinha, A.

Skirlo, S.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Soffer, B. H.

Soljacic, M.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Soriano, M. C.

G. Van der Sande, D. Brunner, and M. C. Soriano, “Advances in photonic reservoir computing,” Nanophotonics 6, 561–576 (2017).
[Crossref]

Spitz, E.

E. Spitz and A. Wertz, “Transmission des images à travers une fibre optique,” Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B 264, 1015 (1967).

Stasio, N.

D. B. Conkey, E. Kakkava, T. Lanvin, D. Loterie, N. Stasio, E. Morales-Delgado, C. Moser, and D. Psaltis, “High power, ultrashort pulse control through a multi-core fiber for ablation,” Opt. Express 25, 11491–11502 (2017).
[Crossref]

D. B. Conkey, N. Stasio, E. E. Morales-Delgado, M. Romito, C. Moser, and D. Psaltis, “Lensless two-photon imaging through a multicore fiber with coherence-gated digital phase conjugation,” J. Biomed. Opt. 21, 045002 (2016).
[Crossref]

Sun, X.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Tait, A. N.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Tomita, A.

Tonello, A.

Unser, M.

Van der Sande, G.

G. Van der Sande, D. Brunner, and M. C. Soriano, “Advances in photonic reservoir computing,” Nanophotonics 6, 561–576 (2017).
[Crossref]

Vellekoop, I. M.

Vonesch, C.

Wabnitz, S.

Wagner, K.

Wakayama, Y.

Wang, H.

Wertz, A.

E. Spitz and A. Wertz, “Transmission des images à travers une fibre optique,” Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B 264, 1015 (1967).

Wilde, J. P.

Wise, F. W.

Wright, L. G.

Wu, A. X.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Yamamura, A. A.

D. Psaltis, A. Sideris, and A. A. Yamamura, “A multilayered neural network controller,” IEEE Control Syst. Mag. 8(2), 17–21 (1988).
[Crossref]

Yang, T. D.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Yariv, A.

Yoon, C.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Zhang, Y.

Zhao, S.

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

Zhou, E.

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Zisserman, A.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations (ICLR) (2015).

Appl. Opt. (3)

Biomed. Opt. Express (1)

Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B (1)

E. Spitz and A. Wertz, “Transmission des images à travers une fibre optique,” Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B 264, 1015 (1967).

IEEE Control Syst. Mag. (1)

D. Psaltis, A. Sideris, and A. A. Yamamura, “A multilayered neural network controller,” IEEE Control Syst. Mag. 8(2), 17–21 (1988).
[Crossref]

J. Biomed. Opt. (1)

D. B. Conkey, N. Stasio, E. E. Morales-Delgado, M. Romito, C. Moser, and D. Psaltis, “Lensless two-photon imaging through a multicore fiber with coherence-gated digital phase conjugation,” J. Biomed. Opt. 21, 045002 (2016).
[Crossref]

J. Opt. Soc. Am. (1)

Nanophotonics (1)

G. Van der Sande, D. Brunner, and M. C. Soriano, “Advances in photonic reservoir computing,” Nanophotonics 6, 561–576 (2017).
[Crossref]

Nat. Commun. (1)

T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun. 3, 1027 (2012).
[Crossref]

Nat. Methods (1)

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. M. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005).
[Crossref]

Nat. Photonics (2)

D. J. Richardson, J. M. Fini, and L. E. Nelson, “Space-division multiplexing in optical fibres,” Nat. Photonics 7, 354–362 (2013).
[Crossref]

Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11, 441–446 (2017).
[Crossref]

New J. Phys. (1)

S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13, 123021 (2011).
[Crossref]

Opt. Express (6)

Opt. Lett. (4)

Optica (3)

Phys. Rev. Lett. (1)

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref]

Sci. Rep. (1)

A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri, and P. R. Prucnal, “Neuromorphic photonic networks using silicon photonic weight banks,” Sci. Rep. 7, 7430 (2017).
[Crossref]

Other (3)

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations (ICLR) (2015).

O. Ronneberger, P. Fischer, and T. Brox, “U-net: convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, eds. (Springer, 2015), Vol. 9351, pp. 234–241.

A. Saade, F. Caltagirone, I. Carron, L. Daudet, A. Drémeau, S. Gigan, and F. Krzakala, “Random projections through multiple optical scattering: approximating Kernels at the speed of light,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016), pp. 6215–6219.

Supplementary Material (2)

NameDescription
» Supplement 1       Supplementary measurements
» Visualization 1       Speckle drift with time due to thermal or mechanical fluctuations on the optical table. We project a constant input at the proximal facet of 1 km long multimode fiber and record the speckle pattern generated at the distal end.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. Experimental setup for pattern transmission through the MMF. The laser beam is expanded and collimated and is directed onto the SLM. The light is modulated by the SLM and its plane is imaged by means of a 4f system onto the proximal facet of a GRIN fiber. The distal facet is imaged by a second 4f system on a CCD camera (CCD2). The modulated light after the SLM is also captured by an imaging system in 2f configuration on the CCD1. HWP, half-wave plate; M, mirror; SLM, spatial light modulator; P, linear polarizer; L, lens; BS, beam splitter; OBJ, microscope objective lens; OF, optical fiber; CCD, camera.
Fig. 2.
Fig. 2. Images of the digits 0 and 4: (a) and (b) input pattern on the SLM, (c) and (d) amplitude-modulated output from the SLM, (e) and (f) phase-modulated output from the SLM, (g) and (h) speckle patterns of each digit, respectively, for the amplitude inputs, and (i) the difference between speckles (g) and (h).
Fig. 3.
Fig. 3. Details of the implemented (a) VGG type image classifier and (b) U-net type image reconstruction convolutional neural networks.
Fig. 4.
Fig. 4. Examples and accuracies of the reconstructed SLM input images from the recorded distal speckle intensity patterns for amplitude-modulated proximal inputs.
Fig. 5.
Fig. 5. Classification accuracy as a function of fiber length. (Solid line: the inputs to the VGG-CNN are the recorded speckles at the distal fiber end; dotted line: the inputs to the VGG-CNN are the reconstructed images obtained by the U-net-CNN; circles: amplitude images projected on the proximal fiber facet; squares: phase images projected on the proximal fiber facet).
Fig. 6.
Fig. 6. Images of speckle patterns recorded on the camera at time point (a) 0 s and (b) 2 s, and (c) the difference between the images in parts (a) and (b).
Fig. 7.
Fig. 7. Normalized confusion matrices for the classification of the reconstructed SLM input images for the (a) 10 m and (b) 1 km GRIN fibers with an amplitude-modulated proximal input.
Fig. 8.
Fig. 8. Training and validation classification accuracies as a function of epoch for the (a) 10 m fiber distal speckle intensity pattern, (b) 10 m fiber SLM reconstructed input, (c) 1 km fiber distal speckle intensity pattern, and (d) 1 km fiber SLM reconstructed input.

Tables (1)

Tables Icon

Table 1. Classification Accuracy for the Four Different Fiber Lengths Using Amplitude or Phase Input Patternsa

Metrics