Abstract

We demonstrate the use of deep learning for fast spectral deconstruction of speckle patterns. The artificial neural network can be effectively trained using numerically constructed multispectral datasets taken from a measured spectral transmission matrix. Optimized neural networks trained on these datasets achieve reliable reconstruction of both discrete and continuous spectra from a monochromatic camera image. Deep learning is compared to analytical inversion methods as well as to a compressive sensing algorithm and shows favourable characteristics both in the oversampling and in the sparse undersampling (compressive) regimes. The deep learning approach offers significant advantages in robustness to drift or noise and in reconstruction speed. In a proof-of-principle demonstrator we achieve real time recovery of hyperspectral information using a multi-core, multi-mode fiber array as a random scattering medium.

Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Full Article  |  PDF Article
OSA Recommended Articles
Snapshot fiber spectral imaging using speckle correlations and compressive sensing

Rebecca French, Sylvain Gigan, and Otto l. Muskens
Opt. Express 26(24) 32302-32316 (2018)

Learning from simulation: An end-to-end deep-learning approach for computational ghost imaging

Fei Wang, Hao Wang, Haichao Wang, Guowei Li, and Guohai Situ
Opt. Express 27(18) 25560-25572 (2019)

On the use of deep learning for computational imaging

George Barbastathis, Aydogan Ozcan, and Guohai Situ
Optica 6(8) 921-943 (2019)

References

  • View by:
  • |
  • |
  • |

  1. S. Rotter and S. Gigan, “Light fields in complex media: Mesoscopic scattering meets wave control,” Rev. Mod. Phys. 89, 015005 (2017).
    [Crossref]
  2. O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012).
    [Crossref]
  3. J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
    [Crossref] [PubMed]
  4. H. Defienne, M. Barbieri, B. Chalopin, B. Chatel, I. A. Walmsley, B. J. Smith, and S. Gigan, “Nonclassical light manipulation in a multiple-scattering medium,” Opt. Lett. 39, 6090–6093 (2014).
    [Crossref] [PubMed]
  5. S. R. Huisman, T. J. Huisman, T. A. W. Wolterink, A. P. Mosk, and P. W. H. Pinkse, “Programmable multiport optical circuits in opaque scattering materials,” Opt. Express 23, 3102–3116 (2015).
    [Crossref] [PubMed]
  6. T. Strudley, R. Bruck, B. Mills, and O. L. Muskens, “An ultrafast reconfigurable nanophotonic switch using wavefront shaping of light in a nonlinear nanomaterial,” Light. Sci. & Appl. 3, e207 (2014).
    [Crossref]
  7. J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
    [Crossref] [PubMed]
  8. S. A. Goorden, M. Horstmann, A. P. Mosk, B. Škorić, and P. W. H. Pinkse, “Quantum-secure authentication of a physical unclonable key,” Optica 1, 421–424 (2014).
    [Crossref]
  9. A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
    [Crossref]
  10. H. Cao, “Perspective on speckle spectrometers,” J. Opt. 19, 060402 (2017).
    [Crossref]
  11. B. Redding and H. Cao, “Using a multimode fiber as a high-resolution, low-loss spectrometer,” Opt. Lett. 37, 3384–3386 (2012).
    [Crossref]
  12. M. Mazilu, T. Vettenburg, A. D. Falco, and K. Dholakia, “Random super-prism wavelength meter,” Opt. Lett. 39, 96–99 (2014).
    [Crossref]
  13. M. Chakrabarti, M. L. Jakobsen, and S. G. Hanson, “Speckle-based spectrometer,” Opt. Lett. 40, 3264–3267 (2015).
    [Crossref] [PubMed]
  14. S. F. Liew, B. Redding, M. A. Choma, H. D. Tagare, and H. Cao, “Broadband multimode fiber spectrometer,” Opt. Lett. 41, 2029–2032 (2016).
    [Crossref] [PubMed]
  15. G. C. Valley, G. A. Sefler, and T. J. Shaw, “Multimode waveguide speckle patterns for compressive sensing,” Opt. Lett. 41, 2529–2532 (2016).
    [Crossref] [PubMed]
  16. B. Redding, M. Alam, M. Seifert, and H. Cao, “High-resolution and broadband all-fiber spectrometers,” Optica 1, 175–180 (2014).
    [Crossref]
  17. N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
    [Crossref] [PubMed]
  18. N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013).
    [Crossref]
  19. J. G. Dwight and T. S. Tkaczyk, “Lenslet array tunable snapshot imaging spectrometer (LATIS) for hyperspectral fluorescence microscopy,” Biomed. Opt. Express 8, 1950–1964 (2017).
    [Crossref] [PubMed]
  20. S. K. Sahoo, D. Tang, and C. Dang, “Single-shot multispectral imaging with a monochromatic camera,” Optica 4, 1209–1213 (2017).
    [Crossref]
  21. P. Wang and R. Menon, “Computational multispectral video imaging [invited],” JOSA A 35, 189–199 (2018).
    [Crossref]
  22. R. French, S. Gigan, and O. L. Muskens, “Speckle-based hyperspectral imaging combining multiple scattering and compressive sensing in nanowire mats,” Opt. Lett. 42, 1820–1823 (2017).
    [Crossref] [PubMed]
  23. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
    [Crossref] [PubMed]
  24. M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9, 529–535 (2015).
    [Crossref]
  25. E. E. Morales-Delgado, D. Psaltis, and C. Moser, “Two-photon imaging through a multimode fiber,” Opt. Express 23, 32158–32170 (2015).
    [Crossref] [PubMed]
  26. A. Porat, E. R. Andresen, H. Rigneault, D. Oron, S. Gigan, and O. Katz, “Widefield lensless imaging through a fiber bundle via speckle correlations,” Opt. Express 24, 16835–16855 (2016).
    [Crossref] [PubMed]
  27. R. French, S. Gigan, and O. L. Muskens, “Snapshot fiber spectral imaging using speckle correlations and compressive sensing,” Opt. Express 26, 32302–32316 (2018).
    [Crossref]
  28. T. Villmann, M. Kästner, A. Backhaus, and U. Seiffert, “Processing hyperspectral data in machine learning,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Proceedings, vol. 21 (2013), pp. 1–10.
  29. W. Zhao and S. Du, “Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach,” IEEE Transactions on Geosci. Remote. Sens. 54, 4544–4554 (2016).
    [Crossref]
  30. Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hyperspectral image classification via manifold ranking,” IEEE Transactions on Neural Networks Learn. Syst. 27, 1279–1289 (2016).
    [Crossref]
  31. Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
    [Crossref]
  32. E. Aptoula, M. C. Ozdemir, and B. Yanikoglu, “Deep learning with attribute profiles for hyperspectral image classification,” IEEE Geosci. Remote. Sens. Lett. 13, 1970–1974 (2016).
    [Crossref]
  33. W. Li, G. Wu, and Q. Du, “Transferred deep learning for hyperspectral target detection,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), (IEEE, 2017), pp. 5177–5180.
    [Crossref]
  34. K. Yao, R. Unni, and Y. Zheng, “Intelligent nanophotonics: merging photonics and artificial intelligence at the nanoscale,” https://arxiv.org/abs/1810.11709 (2018).
  35. M. A. Nielsen, Neural networks and deep learning (Determination Press, 2015).
  36. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT Press, 2016).
  37. P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).
  38. L. Yunzhe, X. Yujia, and T. Lei, “Deep speckle correlation: A deep learning approach toward scalable imaging through scattering media,” Optica 5, 1181–11819 (2018).
    [Crossref]
  39. G. Satat, M. Tancik, O. Gupta, B. Heshmat, and R. Raskar, “Object classification through scattering media with deep learning on time resolved measurement,” Opt. Express 25, 17466–17479 (2017).
    [Crossref] [PubMed]
  40. E. Valent and Y. Silberberg, “Scatterer recognition via analysis of speckle patterns,” Optica 5, 204–207 (2018).
    [Crossref]
  41. R. Horisaki, R. Takagi, and J. Tanida, “Learning-based imaging through scattering media,” Opt. Express 24, 13738–13743 (2016).
    [Crossref] [PubMed]
  42. B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, “Multimode optical fiber transmission with a deep learning network,” Light. Sci. & Appl. 7, 69 (2018).
    [Crossref]
  43. N. Borhani, E. Kakkava, C. Moser, and D. Psaltis, “Learning to see through multimode fibers,” Optica 5, 960–966 (2018).
    [Crossref]
  44. O. Moran, P. Caramazza, D. Faccio, and R. Murray-Smith, “Deep, complex, invertible networks for inversion of transmission effects in multimode optical fibres,” in Proceedings of the 32Nd International Conference on Neural Information Processing Systems, (Curran Associates Inc., USA, 2018), NIPS’18, pp. 3284–3295.
  45. A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
    [Crossref]
  46. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine lLearning on heterogeneous systems,” https://www.tensorflow.org/ (2015).
  47. K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

2019 (1)

P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).

2018 (8)

L. Yunzhe, X. Yujia, and T. Lei, “Deep speckle correlation: A deep learning approach toward scalable imaging through scattering media,” Optica 5, 1181–11819 (2018).
[Crossref]

Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
[Crossref]

E. Valent and Y. Silberberg, “Scatterer recognition via analysis of speckle patterns,” Optica 5, 204–207 (2018).
[Crossref]

B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, “Multimode optical fiber transmission with a deep learning network,” Light. Sci. & Appl. 7, 69 (2018).
[Crossref]

N. Borhani, E. Kakkava, C. Moser, and D. Psaltis, “Learning to see through multimode fibers,” Optica 5, 960–966 (2018).
[Crossref]

A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
[Crossref]

P. Wang and R. Menon, “Computational multispectral video imaging [invited],” JOSA A 35, 189–199 (2018).
[Crossref]

R. French, S. Gigan, and O. L. Muskens, “Snapshot fiber spectral imaging using speckle correlations and compressive sensing,” Opt. Express 26, 32302–32316 (2018).
[Crossref]

2017 (6)

2016 (8)

S. F. Liew, B. Redding, M. A. Choma, H. D. Tagare, and H. Cao, “Broadband multimode fiber spectrometer,” Opt. Lett. 41, 2029–2032 (2016).
[Crossref] [PubMed]

G. C. Valley, G. A. Sefler, and T. J. Shaw, “Multimode waveguide speckle patterns for compressive sensing,” Opt. Lett. 41, 2529–2532 (2016).
[Crossref] [PubMed]

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

E. Aptoula, M. C. Ozdemir, and B. Yanikoglu, “Deep learning with attribute profiles for hyperspectral image classification,” IEEE Geosci. Remote. Sens. Lett. 13, 1970–1974 (2016).
[Crossref]

R. Horisaki, R. Takagi, and J. Tanida, “Learning-based imaging through scattering media,” Opt. Express 24, 13738–13743 (2016).
[Crossref] [PubMed]

W. Zhao and S. Du, “Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach,” IEEE Transactions on Geosci. Remote. Sens. 54, 4544–4554 (2016).
[Crossref]

Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hyperspectral image classification via manifold ranking,” IEEE Transactions on Neural Networks Learn. Syst. 27, 1279–1289 (2016).
[Crossref]

A. Porat, E. R. Andresen, H. Rigneault, D. Oron, S. Gigan, and O. Katz, “Widefield lensless imaging through a fiber bundle via speckle correlations,” Opt. Express 24, 16835–16855 (2016).
[Crossref] [PubMed]

2015 (5)

2014 (6)

2013 (1)

N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013).
[Crossref]

2012 (4)

B. Redding and H. Cao, “Using a multimode fiber as a high-resolution, low-loss spectrometer,” Opt. Lett. 37, 3384–3386 (2012).
[Crossref]

O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012).
[Crossref]

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Agrawal, A.

A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
[Crossref]

Alam, M.

Andresen, E. R.

Aptoula, E.

E. Aptoula, M. C. Ozdemir, and B. Yanikoglu, “Deep learning with attribute profiles for hyperspectral image classification,” IEEE Geosci. Remote. Sens. Lett. 13, 1970–1974 (2016).
[Crossref]

Backhaus, A.

T. Villmann, M. Kästner, A. Backhaus, and U. Seiffert, “Processing hyperspectral data in machine learning,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Proceedings, vol. 21 (2013), pp. 1–10.

Barbieri, M.

Bengio, Y.

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT Press, 2016).

Bertolotti, J.

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Blum, C.

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Borhani, N.

Boyd, S.

A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
[Crossref]

Bruck, R.

T. Strudley, R. Bruck, B. Mills, and O. L. Muskens, “An ultrafast reconfigurable nanophotonic switch using wavefront shaping of light in a nonlinear nanomaterial,” Light. Sci. & Appl. 3, e207 (2014).
[Crossref]

Cao, H.

Caramazza, P.

O. Moran, P. Caramazza, D. Faccio, and R. Murray-Smith, “Deep, complex, invertible networks for inversion of transmission effects in multimode optical fibres,” in Proceedings of the 32Nd International Conference on Neural Information Processing Systems, (Curran Associates Inc., USA, 2018), NIPS’18, pp. 3284–3295.

Carron, I.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Chakrabarti, M.

Chalopin, B.

Chapman, M.

Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
[Crossref]

Chardon, G.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Chatel, B.

Chen, E. H.

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Cho, J.-Y.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Cho, Y.-H.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Choi, W.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Choi, Y.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Choma, M. A.

Chung, E.

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

Cižmár, T.

M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9, 529–535 (2015).
[Crossref]

Courville, A.

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT Press, 2016).

Dang, C.

Dasari, R. R.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Daudet, L.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Defienne, H.

Dholakia, K.

Diamond, S.

A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
[Crossref]

Du, Q.

W. Li, G. Wu, and Q. Du, “Transferred deep learning for hyperspectral target detection,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), (IEEE, 2017), pp. 5177–5180.
[Crossref]

Du, S.

W. Zhao and S. Du, “Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach,” IEEE Transactions on Geosci. Remote. Sens. 54, 4544–4554 (2016).
[Crossref]

Dwight, J. G.

Englund, D.

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Faccio, D.

O. Moran, P. Caramazza, D. Faccio, and R. Murray-Smith, “Deep, complex, invertible networks for inversion of transmission effects in multimode optical fibres,” in Proceedings of the 32Nd International Conference on Neural Information Processing Systems, (Curran Associates Inc., USA, 2018), NIPS’18, pp. 3284–3295.

Falco, A. D.

Fang-Yen, C.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Fowers, J.

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

French, R.

Gigan, S.

Goodfellow, I.

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT Press, 2016).

Goorden, S. A.

Gupta, O.

Hagen, N. A.

N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013).
[Crossref]

Hanson, S. G.

Heshmat, B.

Horisaki, R.

Horstmann, M.

Huisman, S. R.

Huisman, T. J.

Jakobsen, M. L.

Kakkava, E.

Kästner, M.

T. Villmann, M. Kästner, A. Backhaus, and U. Seiffert, “Processing hyperspectral data in machine learning,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Proceedings, vol. 21 (2013), pp. 1–10.

Katz, O.

A. Porat, E. R. Andresen, H. Rigneault, D. Oron, S. Gigan, and O. Katz, “Widefield lensless imaging through a fiber bundle via speckle correlations,” Opt. Express 24, 16835–16855 (2016).
[Crossref] [PubMed]

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012).
[Crossref]

Kim, J.-Y.

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

Kim, M.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Konstantinou, G.

B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, “Multimode optical fiber transmission with a deep learning network,” Light. Sci. & Appl. 7, 69 (2018).
[Crossref]

Kudenov, M. W.

N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013).
[Crossref]

Lagendijk, A.

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Larrieu, G.

P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).

Lecestre, A.

P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).

Lee, H.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Lee, K.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Lee, K. J.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Lei, T.

Lerosey, G.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Li, J.

Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
[Crossref]

Li, W.

W. Li, G. Wu, and Q. Du, “Transferred deep learning for hyperspectral target detection,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), (IEEE, 2017), pp. 5177–5180.
[Crossref]

Liew, S. F.

Lin, J.

Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hyperspectral image classification via manifold ranking,” IEEE Transactions on Neural Networks Learn. Syst. 27, 1279–1289 (2016).
[Crossref]

Liutkus, A.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Loterie, D.

B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, “Multimode optical fiber transmission with a deep learning network,” Light. Sci. & Appl. 7, 69 (2018).
[Crossref]

Luo, Z.

Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
[Crossref]

Mallet, N.

P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).

Martina, D.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Mazilu, M.

Meng, F.

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Menon, R.

P. Wang and R. Menon, “Computational multispectral video imaging [invited],” JOSA A 35, 189–199 (2018).
[Crossref]

Mills, B.

T. Strudley, R. Bruck, B. Mills, and O. L. Muskens, “An ultrafast reconfigurable nanophotonic switch using wavefront shaping of light in a nonlinear nanomaterial,” Light. Sci. & Appl. 3, e207 (2014).
[Crossref]

Morales-Delgado, E. E.

Moran, O.

O. Moran, P. Caramazza, D. Faccio, and R. Murray-Smith, “Deep, complex, invertible networks for inversion of transmission effects in multimode optical fibres,” in Proceedings of the 32Nd International Conference on Neural Information Processing Systems, (Curran Associates Inc., USA, 2018), NIPS’18, pp. 3284–3295.

Moser, C.

Mosk, A. P.

Murray-Smith, R.

O. Moran, P. Caramazza, D. Faccio, and R. Murray-Smith, “Deep, complex, invertible networks for inversion of transmission effects in multimode optical fibres,” in Proceedings of the 32Nd International Conference on Neural Information Processing Systems, (Curran Associates Inc., USA, 2018), NIPS’18, pp. 3284–3295.

Muskens, O. L.

Nielsen, M. A.

M. A. Nielsen, Neural networks and deep learning (Determination Press, 2015).

Oron, D.

Ovtcharov, K.

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

Ozdemir, M. C.

E. Aptoula, M. C. Ozdemir, and B. Yanikoglu, “Deep learning with attribute profiles for hyperspectral image classification,” IEEE Geosci. Remote. Sens. Lett. 13, 1970–1974 (2016).
[Crossref]

Park, C.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Park, J.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Park, Y.

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Pinkse, P. W. H.

Plöschner, M.

M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9, 529–535 (2015).
[Crossref]

Popoff, S.

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Porat, A.

Psaltis, D.

Rahmani, B.

B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, “Multimode optical fiber transmission with a deep learning network,” Light. Sci. & Appl. 7, 69 (2018).
[Crossref]

Raskar, R.

Redding, B.

Rigneault, H.

Rotter, S.

S. Rotter and S. Gigan, “Light fields in complex media: Mesoscopic scattering meets wave control,” Rev. Mod. Phys. 89, 015005 (2017).
[Crossref]

Ruwase, O.

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

Sahoo, S. K.

Satat, G.

Schröder, T.

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Sefler, G. A.

Seifert, M.

Seiffert, U.

T. Villmann, M. Kästner, A. Backhaus, and U. Seiffert, “Processing hyperspectral data in machine learning,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Proceedings, vol. 21 (2013), pp. 1–10.

Shaw, T. J.

Shiue, R.-J.

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Silberberg, Y.

E. Valent and Y. Silberberg, “Scatterer recognition via analysis of speckle patterns,” Optica 5, 204–207 (2018).
[Crossref]

O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012).
[Crossref]

Škoric, B.

Small, E.

O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012).
[Crossref]

Smith, B. J.

Strauss, K.

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

Strudley, T.

T. Strudley, R. Bruck, B. Mills, and O. L. Muskens, “An ultrafast reconfigurable nanophotonic switch using wavefront shaping of light in a nonlinear nanomaterial,” Light. Sci. & Appl. 3, e207 (2014).
[Crossref]

Tagare, H. D.

Takagi, R.

Tancik, M.

Tang, D.

Tanida, J.

Tkaczyk, T. S.

Tyc, T.

M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9, 529–535 (2015).
[Crossref]

Valent, E.

Valley, G. C.

van Putten, E. G.

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Verschueren, R.

A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
[Crossref]

Vettenburg, T.

Villmann, T.

T. Villmann, M. Kästner, A. Backhaus, and U. Seiffert, “Processing hyperspectral data in machine learning,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Proceedings, vol. 21 (2013), pp. 1–10.

Vos, W. L.

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Walmsley, I. A.

Wan, N. H.

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Wang, P.

P. Wang and R. Menon, “Computational multispectral video imaging [invited],” JOSA A 35, 189–199 (2018).
[Crossref]

Wang, Q.

Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hyperspectral image classification via manifold ranking,” IEEE Transactions on Neural Networks Learn. Syst. 27, 1279–1289 (2016).
[Crossref]

Wiecha, P. R.

P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).

Wolterink, T. A. W.

Wu, G.

W. Li, G. Wu, and Q. Du, “Transferred deep learning for hyperspectral target detection,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), (IEEE, 2017), pp. 5177–5180.
[Crossref]

Yang, T. D.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Yanikoglu, B.

E. Aptoula, M. C. Ozdemir, and B. Yanikoglu, “Deep learning with attribute profiles for hyperspectral image classification,” IEEE Geosci. Remote. Sens. Lett. 13, 1970–1974 (2016).
[Crossref]

Yoon, C.

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Yuan, Y.

Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hyperspectral image classification via manifold ranking,” IEEE Transactions on Neural Networks Learn. Syst. 27, 1279–1289 (2016).
[Crossref]

Yujia, X.

Yunzhe, L.

Zhao, W.

W. Zhao and S. Du, “Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach,” IEEE Transactions on Geosci. Remote. Sens. 54, 4544–4554 (2016).
[Crossref]

Zhong, Z.

Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
[Crossref]

ACS Nano (1)

J. Park, J.-Y. Cho, C. Park, K. Lee, H. Lee, Y.-H. Cho, and Y. Park, “Scattering optical elements: Stand-alone optical elements exploiting multiple light scattering,” ACS Nano 10, 6871–6876 (2016).
[Crossref] [PubMed]

Biomed. Opt. Express (1)

IEEE Geosci. Remote. Sens. Lett. (1)

E. Aptoula, M. C. Ozdemir, and B. Yanikoglu, “Deep learning with attribute profiles for hyperspectral image classification,” IEEE Geosci. Remote. Sens. Lett. 13, 1970–1974 (2016).
[Crossref]

IEEE Transactions on Geosci. Remote. Sens. (2)

W. Zhao and S. Du, “Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach,” IEEE Transactions on Geosci. Remote. Sens. 54, 4544–4554 (2016).
[Crossref]

Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” IEEE Transactions on Geosci. Remote. Sens. 56, 847–858 (2018).
[Crossref]

IEEE Transactions on Neural Networks Learn. Syst. (1)

Q. Wang, J. Lin, and Y. Yuan, “Salient band selection for hyperspectral image classification via manifold ranking,” IEEE Transactions on Neural Networks Learn. Syst. 27, 1279–1289 (2016).
[Crossref]

J. Control. Decis. (1)

A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd, “A rewriting system for convex optimization problems,” J. Control. Decis. 5, 42–60 (2018).
[Crossref]

J. Opt. (1)

H. Cao, “Perspective on speckle spectrometers,” J. Opt. 19, 060402 (2017).
[Crossref]

JOSA A (1)

P. Wang and R. Menon, “Computational multispectral video imaging [invited],” JOSA A 35, 189–199 (2018).
[Crossref]

Light. Sci. & Appl. (2)

T. Strudley, R. Bruck, B. Mills, and O. L. Muskens, “An ultrafast reconfigurable nanophotonic switch using wavefront shaping of light in a nonlinear nanomaterial,” Light. Sci. & Appl. 3, e207 (2014).
[Crossref]

B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, “Multimode optical fiber transmission with a deep learning network,” Light. Sci. & Appl. 7, 69 (2018).
[Crossref]

Nat. Commun. (1)

N. H. Wan, F. Meng, T. Schröder, R.-J. Shiue, E. H. Chen, and D. Englund, “High-resolution optical spectroscopy using multimode interference in a compact tapered fibre,” Nat. Commun. 6, 7762 (2015).
[Crossref] [PubMed]

Nat. Nanotechnol. (1)

P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol. 14, 237 (2019).

Nat. Photonics (2)

M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9, 529–535 (2015).
[Crossref]

O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6, 549–553 (2012).
[Crossref]

Nature (1)

J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491, 232–234 (2012).
[Crossref] [PubMed]

Opt. Eng. (1)

N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013).
[Crossref]

Opt. Express (6)

Opt. Lett. (7)

Optica (6)

Phys. Rev. Lett. (1)

Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109, 203901 (2012).
[Crossref] [PubMed]

Rev. Mod. Phys. (1)

S. Rotter and S. Gigan, “Light fields in complex media: Mesoscopic scattering meets wave control,” Rev. Mod. Phys. 89, 015005 (2017).
[Crossref]

Sci. Reports (1)

A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, and I. Carron, “Imaging with nature: Compressive imaging using a multiply scattering medium,” Sci. Reports 4, 5552 (2014).
[Crossref]

Other (8)

W. Li, G. Wu, and Q. Du, “Transferred deep learning for hyperspectral target detection,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS), (IEEE, 2017), pp. 5177–5180.
[Crossref]

K. Yao, R. Unni, and Y. Zheng, “Intelligent nanophotonics: merging photonics and artificial intelligence at the nanoscale,” https://arxiv.org/abs/1810.11709 (2018).

M. A. Nielsen, Neural networks and deep learning (Determination Press, 2015).

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT Press, 2016).

T. Villmann, M. Kästner, A. Backhaus, and U. Seiffert, “Processing hyperspectral data in machine learning,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning Proceedings, vol. 21 (2013), pp. 1–10.

O. Moran, P. Caramazza, D. Faccio, and R. Murray-Smith, “Deep, complex, invertible networks for inversion of transmission effects in multimode optical fibres,” in Proceedings of the 32Nd International Conference on Neural Information Processing Systems, (Curran Associates Inc., USA, 2018), NIPS’18, pp. 3284–3295.

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine lLearning on heterogeneous systems,” https://www.tensorflow.org/ (2015).

K. Ovtcharov, O. Ruwase, J.-Y. Kim, J. Fowers, K. Strauss, and E. Chung, “Accelerating deep convolutional neural networks using specialized hardware,” Microsoft Res. (2015).

Supplementary Material (5)

NameDescription
» Visualization 1       Visualization 1: ‘La Linea’ short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm for several AOTF wavelength sweep sequences. La Linea © CAVA/QUIPOS. La Linea usage rights granted to the University of
» Visualization 2       Visualization 2: ‘La Linea’ short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm for several AOTF wavelength sweep sequences. La Linea © CAVA/QUIPOS. La Linea usage rights granted to the University of
» Visualization 3       Visualization 3: ‘La Linea’ short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm for several AOTF wavelength sweep sequences. La Linea © CAVA/QUIPOS. La Linea usage rights granted to the University of
» Visualization 4       Visualization 4: ‘La Linea’ short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm for several AOTF wavelength sweep sequences. La Linea © CAVA/QUIPOS. La Linea usage rights granted to the University of
» Visualization 5       Visualization 5: ‘Eclipse’ short animation demonstrating the spectral deconstruction using the MCMMF and deep learning algorithm.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1 (a) Scheme of the experimental setup including broadband supercontinuum laser source, acousto-optical tunable filter (AOTF), Spatial Light Modulator (SLM) used for image generation and the multi-core, multimode fiber (MCMMF). (b) Original projected image and detected camera image of the exit interface of the fiber bundle at a single selected wavelength with typical speckle patterns of a selected fiber core for different input wavelengths (λ1λn). La Linea, with permission, copyright CAVA/QUIPOS.
Fig. 2
Fig. 2 Neural network structures used in this study for pixel areas of (i) 5×5 pixels (Y/X = 0.58) and (ii) 20×20 pixels (Y/X = 9.30).
Fig. 3
Fig. 3 (a) Numerical illustration and (b) calculation of reconstruction quality using DL for different sampling rates Y/X, for Nλ=1 and Nλ=10 non-zero wavelengths. One wavelength carries the encoded image (smiley), all other non-zero channels encode the image of a capital “X”, which becomes slightly visible at low sampling rates due to cross-talk (see Appendix). (c) Numerical illustration of image reconstruction using DL for dense spectra (Nλ=42) showing 14 RGB images that are encoded in 42 wavelength channels, the 43rd, blank channel serves for cross-talk control. Reconstructions are shown for undersampling Y/X=0.84 and oversampling Y/X=9.30 regimes. (d) Cross-correlation with ground truth as a function of number of non-zero wavelengths in the spectrum for different sampling rates. Results in (b,d) are averaged over the whole fiber stack and for 100 spectra per fiber core. Light areas indicate the standard deviation of the data. Dashed line at Y/X=1 corresponds to Nyquist-Shannon sampling limit. Dashed line at a cross-correlation of 0.5 indicates the threshold below which the reconstruction is considered to have failed. Underlying full spectral data for (a) and (c) are presented in Fig. 9 and Fig. 10 of the Appendix. All shown cliparts are from www.openclipart.org and public domain.
Fig. 4
Fig. 4 (a) Examples of speckle images and reconstructed spectral information for sparse (three top rows) and dense spectra (two bottom rows, generated by a random-walk like algorithm). (b) oversampling regime with Y/X=9.3, right column: undersampling regime with Y/X=0.58. The black box inside the speckle pattern shows the ROI used in the undersampling case. (c–d) Histograms comparing average cross correlations from 1000 randomly generated sparse (< 50% sparsity) and dense (all wavelengths non-zero) spectra, obtained with deep learning (DL), Tikhonov regularization inversion (TR) and compressive sensing (CS) in the oversampling (c) and undersampling (d) regime.
Fig. 5
Fig. 5 (a) Map showing ratio of reconstruction quality (cross-correlations) for deep learning trained on noisy data (DL+N) over compressive sensing (CS) for 10% added noise. Blue indicates DL+N better than CS, red indicates CS better than DL+N. Contour lines indicate the cross-correlation of DL+N speckle reconstruction. (b) Calculated cross-correlations without noise and in presence of 25% noise. DL+N outperforms CS over a large part of parameter space where the cross-correlation < 0.9. (c) Robustness of the reconstruction against shifts of the speckle patterns by one pixel in a random direction. (d) Calculated cross-correlations without shift and with shift of 1 pixel. DL can be trained on data including shift (DL+S), which renders the method very robust in such scenario, largely outperforming TR and CS.
Fig. 6
Fig. 6 (a) Real-time speckle-based hyperspectral video reconstruction via DL. A video is projected on the fiber bundle using an SLM in amplitude-modulation configuration. During playback the wavelength of the projecting light is changed. Top: Original frames of the input video (see also Visualizations). Bottom: Spectral reconstruction of three wavelength channels of the full multi-core fiber (approx. 2700 fiber cores). (b) Bar graph showing the timings of the different execution steps. Full visualizations are included in the Supporting Materials of this study. La Linea, with permission, copyright CAVA/QUIPOS.
Fig. 7
Fig. 7 Least squares loss on validation data (1000 spectra) as function of training epoch. The training set contains 29000 spectra.
Fig. 8
Fig. 8 Training multi-channel 2D convolutional – 1D convolutional upsampling networks on multiple fibers per network. (a) architecture of a multi-fiber network: speckle images from N different fibers are fed into the network and translated into N spectra at the output. (b) timing and reconstruction quality for a fixed network architecture for increasing number N of reconstructed fibers (corresponding to the number of input and output channels). Trained on 10,000×N speckles. On 32GB RAM, training data for up to 20 fibers could be kept in memory. The trained networks show no significant difference in reconstruction quality, while the timing per fiber reduces almost linearly. The Memory requirement of each pretrained network is approximately constant, hence required RAM linearly decreases with the number of reconstructed fibers per network.
Fig. 9
Fig. 9 Reconstruction of all spectral channels with 10 channels containing images at a sampling rate of (a) Y/X=0.21, (b) Y/X=0.58, (c) and Y/X=1.14. Channels carrying information are 2; 3; 4; 9; 17; 18; 19; 24; 28; 33.
Fig. 10
Fig. 10 Encoding and reconstruction of 14 RGB images in the speckle patterns of the multi-core fiber. Raw reconstructions of the spectral channels, containing the red, green and blue parts of the color images, at a sampling rates of (a) Y/X=0.84 and (b) Y/X=9.3.
Fig. 11
Fig. 11 Visual comparison of reconstruction quality of the different methods (deep learning [DL], Tikhonov regularization [TR] and compressive sensing [CS]) on the RGB dataset (see also Fig. 3). (a) sampling rate of Y/X=0.84 and (b) Y/X=9.3.

Metrics