Abstract

We propose photonic reservoir computing as a new approach to optical signal processing in the context of large scale pattern recognition problems. Photonic reservoir computing is a photonic implementation of the recently proposed reservoir computing concept, where the dynamics of a network of nonlinear elements are exploited to perform general signal processing tasks. In our proposed photonic implementation, we employ a network of coupled Semiconductor Optical Amplifiers (SOA) as the basic building blocks for the reservoir. Although they differ in many key respects from traditional software-based hyperbolic tangent reservoirs, we show using simulations that such a photonic reservoir can outperform traditional reservoirs on a benchmark classification task. Moreover, a photonic implementation offers the promise of massively parallel information processing with low power and high speed.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. Jaeger and H. Haas, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,” Science 304, 78–80 (2004).
    [Crossref] [PubMed]
  2. W. Maass, T. Natschlager, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computing 14, 2531–2560 (2002).
    [Crossref]
  3. W. Maass, T. Natschläger, and H. Markram, “A model for real-time computation in generic neural microcircuits,” in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229–236.
  4. M. D. Skowronski and J. G. Harris, “Minimum mean squared error time series classification using an echo state network prediction model,” in Proceedings of IEEE International symposium on circuits and systems (Institute of Electrical and Electronics Engineers, Island of Kos, Greece, 2006).
  5. D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
    [Crossref]
  6. H. Jaeger “Reservoir riddles: Suggestions for echo state network research (extended abstract).” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1460–1462.
    [Crossref]
  7. P. Joshi and W. Maass, “Movement generation and control with generic neural micrrocircuits,” in Proceedings of BIO-ADIT (2004), pp. 16–31.
  8. J. J. Steil, “Online stability of backpropagation-decorrelation recurrent learning,” Neurocomputing,  69, 642–650 (2006).
    [Crossref]
  9. H. Y. S. Li, Y. Qiao, and D. Psaltis, “Optical Network for Real-Time Face Recognition,” Appl. Opt. 32, 5026–5035 (1993).
    [Crossref] [PubMed]
  10. B. Javidi, J. Li, and Q. Tang, “Optical Implementation of Neural Networks for Face Recognition by the Use of Nonlinear Joint Transform Correlators,” Appl. Opt. 34, 3950–3962 (1995).
    [Crossref] [PubMed]
  11. M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
    [Crossref]
  12. C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995).
  13. V. N. Vapnik, “An overview of statistical learning theory,” IEEE Trans. Neural Networks 10, 988–999 (1999).
    [Crossref]
  14. R. Legenstein and W. Maass, “What makes a dynamical system computationally powerful?” in New directions in statistical signal processing : from systems to brain, S. Haykin, ed. (MIT Press, Cambridge, MA, 2007).
  15. Reservoir lab, “Reservoir Computing Toolbox”. http://www.elis.ugent.be/rct
  16. D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
    [Crossref] [PubMed]
  17. G. P. Agrawal and N. A. Olsson, “Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers,” IEEE J. Quantum Electron. 25, 2297–2306 (1989).
    [Crossref]
  18. H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, “Monolithic integrated Raman silicon laser,” Opt. Express 14, 6705–6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705.
    [Crossref] [PubMed]
  19. M. Cernansky and M. Makula, “Feed-forward echo state networks,” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1479–1482 vol.1473.
    [Crossref]
  20. A. N. Tikhonov and V. I. Arsenin, Solutions of ill-posed problems (Winston & Sons, Washington, 1977).
  21. H. Jaeger, “Adaptive nonlinear system identification with echo state networks,” in Proceedings of NIPS, (MIT Press, Cambridge, MA, 2003), pp. 593–600.

2007 (1)

D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
[Crossref] [PubMed]

2006 (2)

2005 (1)

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
[Crossref]

2004 (1)

H. Jaeger and H. Haas, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,” Science 304, 78–80 (2004).
[Crossref] [PubMed]

2002 (2)

W. Maass, T. Natschlager, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computing 14, 2531–2560 (2002).
[Crossref]

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

1999 (1)

V. N. Vapnik, “An overview of statistical learning theory,” IEEE Trans. Neural Networks 10, 988–999 (1999).
[Crossref]

1995 (1)

1993 (1)

1989 (1)

G. P. Agrawal and N. A. Olsson, “Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers,” IEEE J. Quantum Electron. 25, 2297–2306 (1989).
[Crossref]

Agrawal, G. P.

G. P. Agrawal and N. A. Olsson, “Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers,” IEEE J. Quantum Electron. 25, 2297–2306 (1989).
[Crossref]

Arsenin, V. I.

A. N. Tikhonov and V. I. Arsenin, Solutions of ill-posed problems (Winston & Sons, Washington, 1977).

Bishop, C. M.

C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995).

Campenhout, J. Van

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
[Crossref]

Cernansky, M.

M. Cernansky and M. Makula, “Feed-forward echo state networks,” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1479–1482 vol.1473.
[Crossref]

D’Haene, M.

D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
[Crossref] [PubMed]

Dorren, H. J. S.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Edward, E.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Frietman, E.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Haas, H.

H. Jaeger and H. Haas, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,” Science 304, 78–80 (2004).
[Crossref] [PubMed]

Harris, J. G.

M. D. Skowronski and J. G. Harris, “Minimum mean squared error time series classification using an echo state network prediction model,” in Proceedings of IEEE International symposium on circuits and systems (Institute of Electrical and Electronics Engineers, Island of Kos, Greece, 2006).

Hill, M.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Jaeger, H.

H. Jaeger and H. Haas, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,” Science 304, 78–80 (2004).
[Crossref] [PubMed]

H. Jaeger “Reservoir riddles: Suggestions for echo state network research (extended abstract).” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1460–1462.
[Crossref]

H. Jaeger, “Adaptive nonlinear system identification with echo state networks,” in Proceedings of NIPS, (MIT Press, Cambridge, MA, 2003), pp. 593–600.

Javidi, B.

Jones, R.

Joshi, P.

P. Joshi and W. Maass, “Movement generation and control with generic neural micrrocircuits,” in Proceedings of BIO-ADIT (2004), pp. 16–31.

Khoe, G.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Kuo, Y. H.

Legenstein, R.

R. Legenstein and W. Maass, “What makes a dynamical system computationally powerful?” in New directions in statistical signal processing : from systems to brain, S. Haykin, ed. (MIT Press, Cambridge, MA, 2007).

Li, H. Y. S.

Li, J.

Liu, A. S.

Maass, W.

W. Maass, T. Natschlager, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computing 14, 2531–2560 (2002).
[Crossref]

W. Maass, T. Natschläger, and H. Markram, “A model for real-time computation in generic neural microcircuits,” in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229–236.

P. Joshi and W. Maass, “Movement generation and control with generic neural micrrocircuits,” in Proceedings of BIO-ADIT (2004), pp. 16–31.

R. Legenstein and W. Maass, “What makes a dynamical system computationally powerful?” in New directions in statistical signal processing : from systems to brain, S. Haykin, ed. (MIT Press, Cambridge, MA, 2007).

Makula, M.

M. Cernansky and M. Makula, “Feed-forward echo state networks,” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1479–1482 vol.1473.
[Crossref]

Markram, H.

W. Maass, T. Natschlager, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computing 14, 2531–2560 (2002).
[Crossref]

W. Maass, T. Natschläger, and H. Markram, “A model for real-time computation in generic neural microcircuits,” in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229–236.

Natschlager, T.

W. Maass, T. Natschlager, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computing 14, 2531–2560 (2002).
[Crossref]

Natschläger, T.

W. Maass, T. Natschläger, and H. Markram, “A model for real-time computation in generic neural microcircuits,” in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229–236.

Olsson, N. A.

G. P. Agrawal and N. A. Olsson, “Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers,” IEEE J. Quantum Electron. 25, 2297–2306 (1989).
[Crossref]

Paniccia, M.

Psaltis, D.

Qiao, Y.

Rong, H. S.

Schrauwen, B.

D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
[Crossref] [PubMed]

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
[Crossref]

Skowronski, M. D.

M. D. Skowronski and J. G. Harris, “Minimum mean squared error time series classification using an echo state network prediction model,” in Proceedings of IEEE International symposium on circuits and systems (Institute of Electrical and Electronics Engineers, Island of Kos, Greece, 2006).

Steil, J. J.

J. J. Steil, “Online stability of backpropagation-decorrelation recurrent learning,” Neurocomputing,  69, 642–650 (2006).
[Crossref]

Stroobandt, D.

D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
[Crossref] [PubMed]

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
[Crossref]

Tang, Q.

Tikhonov, A. N.

A. N. Tikhonov and V. I. Arsenin, Solutions of ill-posed problems (Winston & Sons, Washington, 1977).

Vapnik, V. N.

V. N. Vapnik, “An overview of statistical learning theory,” IEEE Trans. Neural Networks 10, 988–999 (1999).
[Crossref]

Verstraeten, D.

D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
[Crossref] [PubMed]

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
[Crossref]

Waardt, H. de

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Xu, S. B.

Appl. Opt. (2)

IEEE J. Quantum Electron. (1)

G. P. Agrawal and N. A. Olsson, “Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers,” IEEE J. Quantum Electron. 25, 2297–2306 (1989).
[Crossref]

IEEE Trans. Neural Networks (2)

V. N. Vapnik, “An overview of statistical learning theory,” IEEE Trans. Neural Networks 10, 988–999 (1999).
[Crossref]

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, “All Fiber-Optic Neural Network Using Coupled SOA Based Ring Lasers,” IEEE Trans. Neural Networks,  13, 1504–1513 (2002).
[Crossref]

Information Processing Lett. (1)

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, “Isolated word recognition with the Liquid State Machine: a case study,” Information Processing Lett. 95, 521–528 (2005).
[Crossref]

Neural Computing (1)

W. Maass, T. Natschlager, and H. Markram, “Real-time computing without stable states: A new framework for neural computation based on perturbations,” Neural Computing 14, 2531–2560 (2002).
[Crossref]

Neural Networks (1)

D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” Neural Networks 20, 391–403 (2007).
[Crossref] [PubMed]

Neurocomputing (1)

J. J. Steil, “Online stability of backpropagation-decorrelation recurrent learning,” Neurocomputing,  69, 642–650 (2006).
[Crossref]

Opt. Express (1)

Science (1)

H. Jaeger and H. Haas, “Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication,” Science 304, 78–80 (2004).
[Crossref] [PubMed]

Other (10)

R. Legenstein and W. Maass, “What makes a dynamical system computationally powerful?” in New directions in statistical signal processing : from systems to brain, S. Haykin, ed. (MIT Press, Cambridge, MA, 2007).

Reservoir lab, “Reservoir Computing Toolbox”. http://www.elis.ugent.be/rct

M. Cernansky and M. Makula, “Feed-forward echo state networks,” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1479–1482 vol.1473.
[Crossref]

A. N. Tikhonov and V. I. Arsenin, Solutions of ill-posed problems (Winston & Sons, Washington, 1977).

H. Jaeger, “Adaptive nonlinear system identification with echo state networks,” in Proceedings of NIPS, (MIT Press, Cambridge, MA, 2003), pp. 593–600.

W. Maass, T. Natschläger, and H. Markram, “A model for real-time computation in generic neural microcircuits,” in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229–236.

M. D. Skowronski and J. G. Harris, “Minimum mean squared error time series classification using an echo state network prediction model,” in Proceedings of IEEE International symposium on circuits and systems (Institute of Electrical and Electronics Engineers, Island of Kos, Greece, 2006).

H. Jaeger “Reservoir riddles: Suggestions for echo state network research (extended abstract).” in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1460–1462.
[Crossref]

P. Joshi and W. Maass, “Movement generation and control with generic neural micrrocircuits,” in Proceedings of BIO-ADIT (2004), pp. 16–31.

C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1.

Reservoir Computing

Fig. 2.
Fig. 2.

(left) tanh transfer characteristic used in analog neural network — (right) SOA: steady state power transfer curve

Fig. 3.
Fig. 3.

Two topologies: (left) a feed-forward network - (right) a waterfall network with feedback connections (long dash) at the edges

Fig. 4.
Fig. 4.

Pattern recognition task: a) Input signal with different transitions between the rectangular and triangular waveform b) desired output c) state (i.e. optical power level) of some of the reservoir nodes d) The approximation (blue) of the desired output (black) by the readout function, e) final output of the system (red)

Fig. 5.
Fig. 5.

Results with a signal frequency of 0.5GHz, simulation time: 100ns, amplitude power: 5mW, delay: 6.25ps, 25nodes: (left) - photonic reservoirs with and without feedback, (right) - classical reservoirs versus photonic reservoirs with feedback

Fig. 6.
Fig. 6.

This figure shows the influence of the reservoir size on the performance of the classical and photonic reservoirs with feedback.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

P out ( τ ) = P in exp [ h ( τ ) ]
ϕ out ( τ ) = ϕ in 1 2 α h ( τ ) ,
h ( τ ) = 0 L g z τ dz
dh d τ = g 0 L h τ c P in ( τ ) P sat τ c [ exp ( h ) - 1 ] ,
Aw B 2 + λ w 2
ρ ( C lin ( 0 ) ) = max 1 i n λ i

Metrics