Abstract

A perfectly convergent unipolar neural associative-memory system based on nonlinear dynamical terminal attractors is presented. With adaptive setting of the threshold values for the dynamic iteration for the unipolar binary neuron states with terminal attractors, perfect convergence is achieved. This achievement and correct retrieval are demonstrated by computer simulation. The simulations are completed (1) by exhaustive tests with all of the possible combinations of stored and test vectors in small-scale networks and (2) by Monte Carlo simulations with randomly generated stored and test vectors in large-scale networks with an M/N ratio of 4 (M is the number of stored vectors; N is the number of neurons ≤ 256). An experiment with exclusive-or logic operations with liquid-crystal-television spatial light modulators is used to show the feasibility of an optoelectronic implementation of the model. The behavior of terminal attractors in basins of energy space is illustrated by examples.

© 1994 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. (USA) 79, 2254–2258 (1982).
    [CrossRef]
  2. R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
    [CrossRef]
  3. B. L. Montgomery, B. V. K. Vijaya Kumar, “Evaluation of the use of Hopfield neural network model as a nearest neighbor algorithm,” Appl. Opt. 25, 3759–3766 (1986).
    [CrossRef] [PubMed]
  4. M. Zak, “Terminal attractors for addressable memory in neural networks,” Phys. Lett. A 133, 18–22 (1988).
    [CrossRef]
  5. M. Zak, “Terminal attractors in neural networks,” Neural Networks 2, 259–274 (1989).
    [CrossRef]
  6. H. K. Liu, J. Barhen, N. H. Farhat, “Optical implementation of terminal attractor based associative memory,” Appl. Opt. 31, 4631–4644 (1992).
    [CrossRef] [PubMed]
  7. S. Y. Kung, H. K. Liu, “An optical inner-product array processor for associative retrieval,” in Nonlinear Optics and Applications, P. A. Yeh, ed., Proc. Soc. Photo-Opt. Instrum. Eng.613, 214–219 (1986).
  8. H. K. Liu, T. H. Chao, “Liquid crystal television spatial light modulators,” Appl. Opt. 28, 4772–4780 (1989).
    [CrossRef] [PubMed]
  9. T. Lu, S. Wu, X. Wu, F. T. S. Yu, “Optical implementation of programmable neural networks,” in Optical Pattern Recognition, H. K. Liu, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1053, 30–39 (1989).
  10. C.-H. Wu, H. K. Liu, “A unipolar terminal-attractor based neural associative memory with adaptive threshold and perfect convergence,” in Proceedings of International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, New York, 1992), Vol. 1, pp. 47–52.

1992 (1)

1989 (2)

1988 (1)

M. Zak, “Terminal attractors for addressable memory in neural networks,” Phys. Lett. A 133, 18–22 (1988).
[CrossRef]

1987 (1)

R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
[CrossRef]

1986 (1)

1982 (1)

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. (USA) 79, 2254–2258 (1982).
[CrossRef]

Barhen, J.

Chao, T. H.

Farhat, N. H.

Hopfield, J. J.

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. (USA) 79, 2254–2258 (1982).
[CrossRef]

Kung, S. Y.

S. Y. Kung, H. K. Liu, “An optical inner-product array processor for associative retrieval,” in Nonlinear Optics and Applications, P. A. Yeh, ed., Proc. Soc. Photo-Opt. Instrum. Eng.613, 214–219 (1986).

Liu, H. K.

H. K. Liu, J. Barhen, N. H. Farhat, “Optical implementation of terminal attractor based associative memory,” Appl. Opt. 31, 4631–4644 (1992).
[CrossRef] [PubMed]

H. K. Liu, T. H. Chao, “Liquid crystal television spatial light modulators,” Appl. Opt. 28, 4772–4780 (1989).
[CrossRef] [PubMed]

C.-H. Wu, H. K. Liu, “A unipolar terminal-attractor based neural associative memory with adaptive threshold and perfect convergence,” in Proceedings of International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, New York, 1992), Vol. 1, pp. 47–52.

S. Y. Kung, H. K. Liu, “An optical inner-product array processor for associative retrieval,” in Nonlinear Optics and Applications, P. A. Yeh, ed., Proc. Soc. Photo-Opt. Instrum. Eng.613, 214–219 (1986).

Lu, T.

T. Lu, S. Wu, X. Wu, F. T. S. Yu, “Optical implementation of programmable neural networks,” in Optical Pattern Recognition, H. K. Liu, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1053, 30–39 (1989).

McElieve, R. J.

R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
[CrossRef]

Montgomery, B. L.

Posner, E. C.

R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
[CrossRef]

Rodemich, E. R.

R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
[CrossRef]

Venkatesch, S. S.

R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
[CrossRef]

Vijaya Kumar, B. V. K.

Wu, C.-H.

C.-H. Wu, H. K. Liu, “A unipolar terminal-attractor based neural associative memory with adaptive threshold and perfect convergence,” in Proceedings of International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, New York, 1992), Vol. 1, pp. 47–52.

Wu, S.

T. Lu, S. Wu, X. Wu, F. T. S. Yu, “Optical implementation of programmable neural networks,” in Optical Pattern Recognition, H. K. Liu, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1053, 30–39 (1989).

Wu, X.

T. Lu, S. Wu, X. Wu, F. T. S. Yu, “Optical implementation of programmable neural networks,” in Optical Pattern Recognition, H. K. Liu, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1053, 30–39 (1989).

Yu, F. T. S.

T. Lu, S. Wu, X. Wu, F. T. S. Yu, “Optical implementation of programmable neural networks,” in Optical Pattern Recognition, H. K. Liu, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1053, 30–39 (1989).

Zak, M.

M. Zak, “Terminal attractors in neural networks,” Neural Networks 2, 259–274 (1989).
[CrossRef]

M. Zak, “Terminal attractors for addressable memory in neural networks,” Phys. Lett. A 133, 18–22 (1988).
[CrossRef]

Appl. Opt. (3)

IEEE Trans. Inf. Theory (1)

R. J. McElieve, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, “The capacity of the Hopfield associative memory,” IEEE Trans. Inf. Theory IT-33, 461–482 (1987).
[CrossRef]

Neural Networks (1)

M. Zak, “Terminal attractors in neural networks,” Neural Networks 2, 259–274 (1989).
[CrossRef]

Phys. Lett. A (1)

M. Zak, “Terminal attractors for addressable memory in neural networks,” Phys. Lett. A 133, 18–22 (1988).
[CrossRef]

Proc. Natl. Acad. Sci. (USA) (1)

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. (USA) 79, 2254–2258 (1982).
[CrossRef]

Other (3)

T. Lu, S. Wu, X. Wu, F. T. S. Yu, “Optical implementation of programmable neural networks,” in Optical Pattern Recognition, H. K. Liu, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1053, 30–39 (1989).

C.-H. Wu, H. K. Liu, “A unipolar terminal-attractor based neural associative memory with adaptive threshold and perfect convergence,” in Proceedings of International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineers, New York, 1992), Vol. 1, pp. 47–52.

S. Y. Kung, H. K. Liu, “An optical inner-product array processor for associative retrieval,” in Nonlinear Optics and Applications, P. A. Yeh, ed., Proc. Soc. Photo-Opt. Instrum. Eng.613, 214–219 (1986).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1
Fig. 1

Dynamic logistic function with a sharp transition at x = θ (threshold).

Fig. 2
Fig. 2

Adaptive threshold, θ, is set to achieve the maximum noise immunity.

Fig. 3
Fig. 3

Experimental setup for the optical xor operations.

Fig. 4
Fig. 4

Oscilloscope display of the intensities of the two spots detected by the CCD camera.

Tables (5)

Tables Icon

Table 1 Exclusive-or(xor) Relationship

Tables Icon

Table 2 Hamming Distance between g[x i (t)] and υ i m

Tables Icon

Table 3 Computer Simulation Results of Unipolar Inner-Product and Cross-Talk-Reduced Terminal-Attractor-Based Associative-Memory (UIT) and (CRIT) Models

Tables Icon

Table 4 Monte Carlo Simulation Results of the Cross-Talk-Reduced Terminal-Attractor-Based Associative-Memory Model in Large-Scale Networksa

Tables Icon

Table 5 Comparison of the Neural Associative-Retrieval Outputs of the Unipolar Inner-Product and the Cross-Talk-Reduced Terminal-Attractor-Based Associative-Memory (UIT) and (CRIT) Models for Three Stored Patterns (1001, 0111, 0000)

Equations (33)

Equations on this page are rendered with MathJax. Learn more.

x i ( t + 1 ) = m = 1 M υ i m α m ( t ) + I i ,
I i = m = 1 M α m ( t ) { g [ x i ( t ) ] - υ i m } 1 / 3 δ m ( t ) , δ m ( t ) = exp ( - β i = 1 N { g [ x i ( t ) ] - υ i m } 2 ) , α m ( t ) = j = 1 N υ j m g [ x j ( t ) ] ,
g [ x j ( t ) ] = 1 / ( 1 + exp { - a [ x j ( t ) - θ ( t ) ] } ) .
x i ( t + 1 ) = m = 1 M α m ( t ) { [ 1 + δ m ( t ) ] υ i m - g [ x i ( t ) ] δ m ( t ) } ,
g [ x i ( t ) ] - υ i m = 1 , 0 , or - 1 , { g [ x i ( t ) ] - υ i m } 1 / 3 = g [ x i ( t ) ] - υ i m .
x i ( t + 1 ) = m = 1 M α m ( t ) { [ 1 + δ m ( t ) ] υ i m + g [ x i ( t ) ] δ m ( t ) } .
θ ( t ) = α m ( t ) [ ½ + δ m ( t ) ] ,
α m ( t ) = 0             for all m m .
( case 1 ) If υ i m = 1 and g [ x i ( t ) ] 1 , then x i ( t + 1 ) α m ( t ) [ 1 + 2 δ m ( t ) ] . ( case 2 ) If υ i m = 0 and g [ x i ( t ) ] 0 , then x i ( t + 1 ) 0. ( case 3 ) If υ i m = 1 and g [ x i ( t ) ] 0 , then x i ( t + 1 ) α m ( t ) [ 1 + δ m ( t ) ] . ( case 4 ) If υ i m = 0 and g [ x i ( t ) ] 1 , then x i ( t + 1 ) α m ( t ) δ m ( t ) .
θ ( t ) = m = 1 M α m ( t ) [ ( 1 / 2 ) + δ m ( t ) ] .
x i ( t + 1 ) = m = 1 M α m ( t ) δ m ( t ) { [ 1 + δ m ( t ) ] υ i m + g [ x i ( t ) ] δ m ( t ) } .
θ ( t ) = m = 1 M α m ( t ) δ m ( t ) [ ( 1 / 2 ) + δ m ( t ) ] .
{ g [ x i ( t ) ] - υ i m } 2 g [ x i ( t ) ] - υ i m ;
i = 1 N { g [ x i ( t ) ] - υ i m } 2 = i = 1 N g [ x i ( t ) ] - υ i m = i = 1 N { 1 + g [ x i ( t ) ] XOR υ i m } = N - α m ( t ) ,
α m ( t ) = i = 1 N g [ x i ( t ) ] XOR υ i m .
δ m ( t ) = exp { - β [ N - α m ( t ) ] } ,
[ 1 0 0 1 ] ,             [ 0 1 1 1 ] ,             [ 0 0 0 0 ] ,
[ 1 1 0 0 ] ,
[ 2.3 4.1 5.7 8.0 ] ,
[ 0 0 1 1 ]
α 1 = 2 ,             α 2 = 3 ,             α 3 = 2.
δ 1 = e - 2 ,             δ 2 = e - 1 ,             δ 3 = e - 2 ,
[ 0.3 1.5 2.0 2.3 ] ,
[ 0 1 1 1 ] ,
[ 0.0 0.5 0.5 0.6 ]
[ 0.0 0.2 0.2 0.2 ] ,
[ 1 0 1 1 ] ,
[ 1 1 1 1 ] , [ 1 1 0 1 ] ,
[ 1 0 1 1 ] , [ 1 0 0 1 ] .
[ 7.7 6.4 5.5 7.7 ] ,
[ 17 17 13 17 ] ,
[ 2.3 1.8 2.0 2.3 ] .
[ 14 14 12 14 ] ,

Metrics