Abstract

An anti-Hebbian local learning algorithm for two-layer optical neural networks is introduced. With this learning rule, the weight update for a certain connection depends only on the input and output of that connection and a global, scalar error signal. Therefore the backpropagation of error signals through the network, as required by the commonly used back error propagation algorithm, is avoided. It still guarantees, however, that the synaptic weights are updated in the error descent direction. With the apparent advantage of simpler optical implementation this learning rule is also shown by simulations to be computationally effective.

© 1992 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. E. Rumelhart, G. E. Hinton, R. J. Williams, in Parallel Distributed Processing, D. E. Rumelhart, J. L. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.
  2. K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
    [CrossRef] [PubMed]
  3. D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
    [CrossRef]
  4. T. Grossman, R. Meir, E. Domany, “Learning by choice of internal representations,” Complex Syst. 2, 555–575 (1988).
  5. J. J. Hopfield, “Learning algorithms and probability distributions in feed-forward and feed-back networks,” Proc. Natl. Acad. Sci. USA 84, 8429–8433 (1987).
    [CrossRef] [PubMed]
  6. S. A. Solla, E. Levin, M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst. 2, 625–640 (1988).

1988 (3)

T. Grossman, R. Meir, E. Domany, “Learning by choice of internal representations,” Complex Syst. 2, 555–575 (1988).

S. A. Solla, E. Levin, M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst. 2, 625–640 (1988).

D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
[CrossRef]

1987 (2)

K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
[CrossRef] [PubMed]

J. J. Hopfield, “Learning algorithms and probability distributions in feed-forward and feed-back networks,” Proc. Natl. Acad. Sci. USA 84, 8429–8433 (1987).
[CrossRef] [PubMed]

Brady, D.

Domany, E.

T. Grossman, R. Meir, E. Domany, “Learning by choice of internal representations,” Complex Syst. 2, 555–575 (1988).

Fleisher, M.

S. A. Solla, E. Levin, M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst. 2, 625–640 (1988).

Grossman, T.

T. Grossman, R. Meir, E. Domany, “Learning by choice of internal representations,” Complex Syst. 2, 555–575 (1988).

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, in Parallel Distributed Processing, D. E. Rumelhart, J. L. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Hopfield, J. J.

J. J. Hopfield, “Learning algorithms and probability distributions in feed-forward and feed-back networks,” Proc. Natl. Acad. Sci. USA 84, 8429–8433 (1987).
[CrossRef] [PubMed]

Levin, E.

S. A. Solla, E. Levin, M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst. 2, 625–640 (1988).

Meir, R.

T. Grossman, R. Meir, E. Domany, “Learning by choice of internal representations,” Complex Syst. 2, 555–575 (1988).

Psaltis, D.

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, in Parallel Distributed Processing, D. E. Rumelhart, J. L. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Solla, S. A.

S. A. Solla, E. Levin, M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst. 2, 625–640 (1988).

Wagner, K.

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, in Parallel Distributed Processing, D. E. Rumelhart, J. L. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Appl. Opt. (2)

Complex Syst. (2)

T. Grossman, R. Meir, E. Domany, “Learning by choice of internal representations,” Complex Syst. 2, 555–575 (1988).

S. A. Solla, E. Levin, M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst. 2, 625–640 (1988).

Proc. Natl. Acad. Sci. USA (1)

J. J. Hopfield, “Learning algorithms and probability distributions in feed-forward and feed-back networks,” Proc. Natl. Acad. Sci. USA 84, 8429–8433 (1987).
[CrossRef] [PubMed]

Other (1)

D. E. Rumelhart, G. E. Hinton, R. J. Williams, in Parallel Distributed Processing, D. E. Rumelhart, J. L. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (2)

Fig. 1
Fig. 1

Schematic diagram of a feed-forward two-layer neural network.

Fig. 2
Fig. 2

Optical architecture that implements the ALL algorithm.

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

s j ( n ) = i = 1 N n - 1 w j i ( n ) o i ( n - 1 ) ,
o j ( n ) = f [ s j ( n ) ] ,
E = k = 1 N 2 [ ( 1 + t k ) ln 1 + t k 1 + o k ( 2 ) + ( 1 - t k ) ln 1 - t k 1 - o k ( 2 ) ] .
Δ w k j ( 2 ) - E w k j ( 2 ) = 2 δ k o j ( 1 ) ,
Δ w j i ( 1 ) - E w j i ( 1 ) = 2 { 1 - [ o j ( 1 ) ] 2 } o i ( 0 ) k = 1 N 2 δ k w k j ( 2 ) ,
Δ w j i ( 1 ) o j ( 1 ) o i ( 0 )
Δ w j i ( 1 ) γ o j ( 1 ) o i ( 0 ) { 1 - [ o j ( 1 ) ] 2 } .
Δ E = j = 1 N 1 E w j i ( 1 ) Δ w j i ( 1 ) - γ [ o i ( 0 ) ] 2 k = 1 N 2 δ k j = 1 N 1 w k j ( 2 ) o j ( 1 ) = - γ [ o i ( 0 ) ] 2 k = 1 N 2 δ S k sbx ( 2 ) = - γ 2 [ o i ( 0 ) ] 2 0 ,

Metrics