Abstract

We propose photonic reservoir computing as a new approach to optical signal processing in the context of large scale pattern recognition problems. Photonic reservoir computing is a photonic implementation of the recently proposed reservoir computing concept, where the dynamics of a network of nonlinear elements are exploited to perform general signal processing tasks. In our proposed photonic implementation, we employ a network of coupled Semiconductor Optical Amplifiers (SOA) as the basic building blocks for the reservoir. Although they differ in many key respects from traditional software-based hyperbolic tangent reservoirs, we show using simulations that such a photonic reservoir can outperform traditional reservoirs on a benchmark classification task. Moreover, a photonic implementation offers the promise of massively parallel information processing with low power and high speed.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. Jaeger and H. Haas, "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication," Science 304, 78-80 (2004).
    [CrossRef] [PubMed]
  2. W. Maass, T. Natschlager and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computing 14, 2531-2560 (2002).
    [CrossRef]
  3. W. Maass, T. Natschl¨ager, H. Markram, "A model for real-time computation in generic neural microcircuits," in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229-236.
  4. M. D. Skowronski, J. G. Harris, "Minimum mean squared error time series classification using an echo state network prediction model," in Proceedings of IEEE International symposium on circuits and systems (Institute of Electrical and Electronics Engineers, Island of Kos, Greece, 2006).
  5. D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
    [CrossRef]
  6. H. Jaeger, "Reservoir riddles: Suggestions for echo state network research (extended abstract)." in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1460-1462.
    [CrossRef]
  7. P. Joshi, W. Maass, "Movement generation and control with generic neural micrrocircuits," in Proceedings of BIO-ADIT (2004), pp. 16-31.
  8. J. J. Steil, "Online stability of backpropagation-decorrelation recurrent learning," Neurocomputing 69, 642-650 (2006).
    [CrossRef]
  9. H. Y. S. Li, Y. Qiao, and D. Psaltis, "Optical Network for Real-Time Face Recognition," Appl. Opt. 32, 5026-5035 (1993).
    [CrossRef] [PubMed]
  10. B. Javidi, J. Li, and Q. Tang, "Optical Implementation of Neural Networks for Face Recognition by the use of Nonlinear Joint Transform Correlators," Appl. Opt. 34, 3950-3962 (1995).
    [CrossRef] [PubMed]
  11. M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
    [CrossRef]
  12. C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995).
  13. V. N. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Networks 10, 988-999 (1999).
    [CrossRef]
  14. R. Legenstein and W. Maass, "What makes a dynamical system computationally powerful?" in New directions in statistical signal processing: from systems to brain, S. Haykin, ed. (MIT Press, Cambridge, MA, 2007).
  15. Reservoir lab, "Reservoir Computing Toolbox". http://www.elis.ugent.be/rct
  16. D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
    [CrossRef] [PubMed]
  17. G. P. Agrawal and N. A. Olsson, "Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers," IEEE J. Quantum Electron. 25, 2297-2306 (1989).
    [CrossRef]
  18. H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
    [CrossRef] [PubMed]
  19. M. Cernansky and M. Makula, "Feed-forward echo state networks," in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), vol. 1473, pp. 1479-1482.
    [CrossRef]
  20. A. N. Tikhonov and V. I. Arsenin, Solutions of ill-posed problems (Winston & Sons, Washington, 1977).
  21. H. Jaeger, "Adaptive nonlinear system identification with echo state networks," in Proceedings of NIPS, (MIT Press, Cambridge, MA, 2003), pp. 593-600.

2007

D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
[CrossRef] [PubMed]

2006

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

J. J. Steil, "Online stability of backpropagation-decorrelation recurrent learning," Neurocomputing 69, 642-650 (2006).
[CrossRef]

2005

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
[CrossRef]

2004

H. Jaeger and H. Haas, "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication," Science 304, 78-80 (2004).
[CrossRef] [PubMed]

2002

W. Maass, T. Natschlager and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computing 14, 2531-2560 (2002).
[CrossRef]

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

1999

V. N. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Networks 10, 988-999 (1999).
[CrossRef]

1995

B. Javidi, J. Li, and Q. Tang, "Optical Implementation of Neural Networks for Face Recognition by the use of Nonlinear Joint Transform Correlators," Appl. Opt. 34, 3950-3962 (1995).
[CrossRef] [PubMed]

1993

H. Y. S. Li, Y. Qiao, and D. Psaltis, "Optical Network for Real-Time Face Recognition," Appl. Opt. 32, 5026-5035 (1993).
[CrossRef] [PubMed]

1989

G. P. Agrawal and N. A. Olsson, "Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers," IEEE J. Quantum Electron. 25, 2297-2306 (1989).
[CrossRef]

Agrawal, G. P.

G. P. Agrawal and N. A. Olsson, "Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers," IEEE J. Quantum Electron. 25, 2297-2306 (1989).
[CrossRef]

D???Haene, M.

D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
[CrossRef] [PubMed]

de Waardt, H.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Dorren, H. J. S.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Edward, E.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Frietman, E.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Haas, H.

H. Jaeger and H. Haas, "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication," Science 304, 78-80 (2004).
[CrossRef] [PubMed]

Hill, M.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Jaeger, H.

H. Jaeger and H. Haas, "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication," Science 304, 78-80 (2004).
[CrossRef] [PubMed]

Javidi, B.

B. Javidi, J. Li, and Q. Tang, "Optical Implementation of Neural Networks for Face Recognition by the use of Nonlinear Joint Transform Correlators," Appl. Opt. 34, 3950-3962 (1995).
[CrossRef] [PubMed]

Jones, R.

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Khoe, G.

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Kuo, Y. H.

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Li, H. Y. S.

H. Y. S. Li, Y. Qiao, and D. Psaltis, "Optical Network for Real-Time Face Recognition," Appl. Opt. 32, 5026-5035 (1993).
[CrossRef] [PubMed]

Li, J.

B. Javidi, J. Li, and Q. Tang, "Optical Implementation of Neural Networks for Face Recognition by the use of Nonlinear Joint Transform Correlators," Appl. Opt. 34, 3950-3962 (1995).
[CrossRef] [PubMed]

Liu, A. S.

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Maass, W.

W. Maass, T. Natschlager and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computing 14, 2531-2560 (2002).
[CrossRef]

Markram, H.

W. Maass, T. Natschlager and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computing 14, 2531-2560 (2002).
[CrossRef]

Natschlager, T.

W. Maass, T. Natschlager and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computing 14, 2531-2560 (2002).
[CrossRef]

Olsson, N. A.

G. P. Agrawal and N. A. Olsson, "Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers," IEEE J. Quantum Electron. 25, 2297-2306 (1989).
[CrossRef]

Paniccia, M.

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Psaltis, D.

H. Y. S. Li, Y. Qiao, and D. Psaltis, "Optical Network for Real-Time Face Recognition," Appl. Opt. 32, 5026-5035 (1993).
[CrossRef] [PubMed]

Qiao, Y.

H. Y. S. Li, Y. Qiao, and D. Psaltis, "Optical Network for Real-Time Face Recognition," Appl. Opt. 32, 5026-5035 (1993).
[CrossRef] [PubMed]

Rong, H. S.

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Schrauwen, B.

D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
[CrossRef] [PubMed]

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
[CrossRef]

Steil, J. J.

J. J. Steil, "Online stability of backpropagation-decorrelation recurrent learning," Neurocomputing 69, 642-650 (2006).
[CrossRef]

Stroobandt, D.

D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
[CrossRef] [PubMed]

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
[CrossRef]

Tang, Q.

B. Javidi, J. Li, and Q. Tang, "Optical Implementation of Neural Networks for Face Recognition by the use of Nonlinear Joint Transform Correlators," Appl. Opt. 34, 3950-3962 (1995).
[CrossRef] [PubMed]

Van Campenhout, J.

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
[CrossRef]

Vapnik, V. N.

V. N. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Networks 10, 988-999 (1999).
[CrossRef]

Verstraeten, D.

D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
[CrossRef] [PubMed]

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
[CrossRef]

Xu, S. B.

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Appl. Opt.

H. Y. S. Li, Y. Qiao, and D. Psaltis, "Optical Network for Real-Time Face Recognition," Appl. Opt. 32, 5026-5035 (1993).
[CrossRef] [PubMed]

B. Javidi, J. Li, and Q. Tang, "Optical Implementation of Neural Networks for Face Recognition by the use of Nonlinear Joint Transform Correlators," Appl. Opt. 34, 3950-3962 (1995).
[CrossRef] [PubMed]

IEEE J. Quantum Electron.

G. P. Agrawal and N. A. Olsson, "Self-Phase Modulation and Spectral Broadening of Optical Pulses in Semiconductor-Laser Amplifiers," IEEE J. Quantum Electron. 25, 2297-2306 (1989).
[CrossRef]

IEEE Trans. Neural Networks

V. N. Vapnik, "An overview of statistical learning theory," IEEE Trans. Neural Networks 10, 988-999 (1999).
[CrossRef]

M. Hill, E. Edward, E. Frietman, H. de Waardt, H. J. S. Dorren, and G. Khoe, "All Fiber-Optic Neural Network using coupled SOA based Ring Lasers," IEEE Trans. Neural Networks  13, 1504-1513 (2002).
[CrossRef]

Information Processing Lett.

D. Verstraeten, B. Schrauwen, D. Stroobandt, and J. Van Campenhout, "Isolated word recognition with the Liquid State Machine: a case study," Information Processing Lett. 95, 521-528 (2005).
[CrossRef]

Neural Computing

W. Maass, T. Natschlager and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computing 14, 2531-2560 (2002).
[CrossRef]

Neural Networks

D. Verstraeten, B. Schrauwen, M. D�??Haene, and D. Stroobandt, "An experimental unification of reservoir computing methods," Neural Networks 20, 391-403 (2007).
[CrossRef] [PubMed]

Neurocomputing

J. J. Steil, "Online stability of backpropagation-decorrelation recurrent learning," Neurocomputing 69, 642-650 (2006).
[CrossRef]

Opt. Express

H. S. Rong, Y. H. Kuo, S. B. Xu, A. S. Liu, R. Jones, and M. Paniccia, "Monolithic integrated Raman silicon laser," Opt. Express 14, 6705-6712 (2006), http://www.opticsinfobase.org/abstract.cfm?URI=oe-14-15-6705
[CrossRef] [PubMed]

Science

H. Jaeger and H. Haas, "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication," Science 304, 78-80 (2004).
[CrossRef] [PubMed]

Other

R. Legenstein and W. Maass, "What makes a dynamical system computationally powerful?" in New directions in statistical signal processing: from systems to brain, S. Haykin, ed. (MIT Press, Cambridge, MA, 2007).

Reservoir lab, "Reservoir Computing Toolbox". http://www.elis.ugent.be/rct

M. Cernansky and M. Makula, "Feed-forward echo state networks," in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), vol. 1473, pp. 1479-1482.
[CrossRef]

A. N. Tikhonov and V. I. Arsenin, Solutions of ill-posed problems (Winston & Sons, Washington, 1977).

H. Jaeger, "Adaptive nonlinear system identification with echo state networks," in Proceedings of NIPS, (MIT Press, Cambridge, MA, 2003), pp. 593-600.

W. Maass, T. Natschl¨ager, H. Markram, "A model for real-time computation in generic neural microcircuits," in Proceedings of NIPS, (MIT Press, Vancouver, British Columbia, 2003), pp. 229-236.

M. D. Skowronski, J. G. Harris, "Minimum mean squared error time series classification using an echo state network prediction model," in Proceedings of IEEE International symposium on circuits and systems (Institute of Electrical and Electronics Engineers, Island of Kos, Greece, 2006).

H. Jaeger, "Reservoir riddles: Suggestions for echo state network research (extended abstract)." in Proceedings of IEEE International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, Montreal, 2005), pp. 1460-1462.
[CrossRef]

P. Joshi, W. Maass, "Movement generation and control with generic neural micrrocircuits," in Proceedings of BIO-ADIT (2004), pp. 16-31.

C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1.

Reservoir Computing

Fig. 2.
Fig. 2.

(left) tanh transfer characteristic used in analog neural network — (right) SOA: steady state power transfer curve

Fig. 3.
Fig. 3.

Two topologies: (left) a feed-forward network - (right) a waterfall network with feedback connections (long dash) at the edges

Fig. 4.
Fig. 4.

Pattern recognition task: a) Input signal with different transitions between the rectangular and triangular waveform b) desired output c) state (i.e. optical power level) of some of the reservoir nodes d) The approximation (blue) of the desired output (black) by the readout function, e) final output of the system (red)

Fig. 5.
Fig. 5.

Results with a signal frequency of 0.5GHz, simulation time: 100ns, amplitude power: 5mW, delay: 6.25ps, 25nodes: (left) - photonic reservoirs with and without feedback, (right) - classical reservoirs versus photonic reservoirs with feedback

Fig. 6.
Fig. 6.

This figure shows the influence of the reservoir size on the performance of the classical and photonic reservoirs with feedback.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

P out ( τ ) = P in exp [ h ( τ ) ]
ϕ out ( τ ) = ϕ in 1 2 α h ( τ ) ,
h ( τ ) = 0 L g z τ dz
dh d τ = g 0 L h τ c P in ( τ ) P sat τ c [ exp ( h ) - 1 ] ,
Aw B 2 + λ w 2
ρ ( C lin ( 0 ) ) = max 1 i n λ i

Metrics