Abstract

An algorithm for optimizing a bipolar interconnection weight matrix with the Hopfield network is proposed. The effectiveness of this algorithm is demonstrated by computer simulation and optical implementation. In the optical implementation of the neural network the interconnection weights are biased to yield a nonnegative weight matrix. Moreover, a threshold subchannel is added so that the system can realize, in real time, the bipolar weighted summation in a single channel. Preliminary experimental results obtained from the applications in associative memories and multitarget classification with rotation invariance are shown.

© 1999 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. N. Farhat, D. Psaltis, A. Prata, E. Peak, “Optical implementation of the Hopfield model,” Appl. Opt. 24, 1469–1474 (1985).
    [CrossRef] [PubMed]
  2. Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
    [CrossRef]
  3. K. Noguch, “Large-scale two-dimensional optical Hopfield associative memory using an incoherent optical free-space interconnection,” Opt. Lett. 16, 1110–1112 (1991).
    [CrossRef]
  4. T. Lu, S. Wu, X. Xu, F. T. S. Yu, “Two-dimensional programmable optical neural network,” Appl. Opt. 28, 4908–4913 (1989).
    [CrossRef] [PubMed]
  5. M. Lu, Y. Zhan, G. Mu, “Bipolar optical neural network with adaptive threshold,” Optik 91, 178–182 (1992).
  6. S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).
  7. T. Lu, X. Xu, S. Wu, F. T. S. Yu, “Neural network model using interpattern association,” Appl. Opt. 29, 284–288 (1990).
    [CrossRef] [PubMed]
  8. X. Yang, T. Lu, F. T. S. Yu, D. Gregory, “Redundant interconnection interpattern association neural network,” Appl. Opt. 30, 5182–5187 (1991).
    [CrossRef] [PubMed]
  9. W. Huang, Y. Zhang, “A Monte Carlo learning algorithm for clipped neural network model,” Optoelectron. 7, 133–137 (1992).
  10. Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).
  11. J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA 79, 2254–2558 (1982).
    [CrossRef]
  12. C. Peterson, “Parallel distributed approaches to combinatorial optimization: benchmark studies on travelling salesman problem,” Neural Comput. 2, 261–269 (1990).
    [CrossRef]
  13. D. W. Tank, J. J. Hopfield, “Simple neural optimization network: an A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst. CAS-35, 533–541 (1986).
    [CrossRef]
  14. T. Lu, K. Choi, S. Wu, X. Xu, F. T. S. Yu, “Optical disk based neural network,” Appl. Opt. 28, 4722–4724 (1989).
    [CrossRef] [PubMed]
  15. C. Uang, G. Lu, F. T. S. Yu, “Unipolar interpattern association (IPA) neural networks,” Opt. Lett. 19, 52–54 (1994).
    [CrossRef] [PubMed]
  16. S. Chang, J. Shen, Y. Zhang, “High-capacity neural networks model with unipolar and binary interconnection,” Acta Photonica Sin. (China) 25, 866–870 (1996).
  17. S. Chang, J. Shen, Y. Zhang, “A neural networks model with trinary interconnection weights based on orthogonal algorithm,” Acta Optica Sin. (China) 16, 1126–1132 (1996).
  18. C. Uang, S. Yin, G. Lu, F. T. S. Yu, “Shift- and rotation-invariant interpattern heteroassociation (IHA) model,” Opt. Commun. 104, 285–292 (1994).
    [CrossRef]

1998 (1)

S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).

1997 (1)

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

1996 (3)

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

S. Chang, J. Shen, Y. Zhang, “High-capacity neural networks model with unipolar and binary interconnection,” Acta Photonica Sin. (China) 25, 866–870 (1996).

S. Chang, J. Shen, Y. Zhang, “A neural networks model with trinary interconnection weights based on orthogonal algorithm,” Acta Optica Sin. (China) 16, 1126–1132 (1996).

1994 (2)

C. Uang, S. Yin, G. Lu, F. T. S. Yu, “Shift- and rotation-invariant interpattern heteroassociation (IHA) model,” Opt. Commun. 104, 285–292 (1994).
[CrossRef]

C. Uang, G. Lu, F. T. S. Yu, “Unipolar interpattern association (IPA) neural networks,” Opt. Lett. 19, 52–54 (1994).
[CrossRef] [PubMed]

1992 (2)

M. Lu, Y. Zhan, G. Mu, “Bipolar optical neural network with adaptive threshold,” Optik 91, 178–182 (1992).

W. Huang, Y. Zhang, “A Monte Carlo learning algorithm for clipped neural network model,” Optoelectron. 7, 133–137 (1992).

1991 (2)

1990 (2)

T. Lu, X. Xu, S. Wu, F. T. S. Yu, “Neural network model using interpattern association,” Appl. Opt. 29, 284–288 (1990).
[CrossRef] [PubMed]

C. Peterson, “Parallel distributed approaches to combinatorial optimization: benchmark studies on travelling salesman problem,” Neural Comput. 2, 261–269 (1990).
[CrossRef]

1989 (2)

1986 (1)

D. W. Tank, J. J. Hopfield, “Simple neural optimization network: an A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst. CAS-35, 533–541 (1986).
[CrossRef]

1985 (1)

1982 (1)

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA 79, 2254–2558 (1982).
[CrossRef]

Chang, S.

S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).

S. Chang, J. Shen, Y. Zhang, “A neural networks model with trinary interconnection weights based on orthogonal algorithm,” Acta Optica Sin. (China) 16, 1126–1132 (1996).

S. Chang, J. Shen, Y. Zhang, “High-capacity neural networks model with unipolar and binary interconnection,” Acta Photonica Sin. (China) 25, 866–870 (1996).

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

Choi, K.

Farhat, N.

Feng, Z.

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

Gao, S.

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

Gregory, D.

Hopfield, J. J.

D. W. Tank, J. J. Hopfield, “Simple neural optimization network: an A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst. CAS-35, 533–541 (1986).
[CrossRef]

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA 79, 2254–2558 (1982).
[CrossRef]

Huang, W.

W. Huang, Y. Zhang, “A Monte Carlo learning algorithm for clipped neural network model,” Optoelectron. 7, 133–137 (1992).

Jin, C.

S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).

Li, Y. H.

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

Lu, G.

C. Uang, S. Yin, G. Lu, F. T. S. Yu, “Shift- and rotation-invariant interpattern heteroassociation (IHA) model,” Opt. Commun. 104, 285–292 (1994).
[CrossRef]

C. Uang, G. Lu, F. T. S. Yu, “Unipolar interpattern association (IPA) neural networks,” Opt. Lett. 19, 52–54 (1994).
[CrossRef] [PubMed]

Lu, M.

M. Lu, Y. Zhan, G. Mu, “Bipolar optical neural network with adaptive threshold,” Optik 91, 178–182 (1992).

Lu, T.

Mu, G.

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

M. Lu, Y. Zhan, G. Mu, “Bipolar optical neural network with adaptive threshold,” Optik 91, 178–182 (1992).

Noguch, K.

Peak, E.

Peterson, C.

C. Peterson, “Parallel distributed approaches to combinatorial optimization: benchmark studies on travelling salesman problem,” Neural Comput. 2, 261–269 (1990).
[CrossRef]

Prata, A.

Psaltis, D.

Shen, J.

S. Chang, J. Shen, Y. Zhang, “High-capacity neural networks model with unipolar and binary interconnection,” Acta Photonica Sin. (China) 25, 866–870 (1996).

S. Chang, J. Shen, Y. Zhang, “A neural networks model with trinary interconnection weights based on orthogonal algorithm,” Acta Optica Sin. (China) 16, 1126–1132 (1996).

Song, Z.

S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).

Sun, Y.

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

Tank, D. W.

D. W. Tank, J. J. Hopfield, “Simple neural optimization network: an A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst. CAS-35, 533–541 (1986).
[CrossRef]

Uang, C.

C. Uang, G. Lu, F. T. S. Yu, “Unipolar interpattern association (IPA) neural networks,” Opt. Lett. 19, 52–54 (1994).
[CrossRef] [PubMed]

C. Uang, S. Yin, G. Lu, F. T. S. Yu, “Shift- and rotation-invariant interpattern heteroassociation (IHA) model,” Opt. Commun. 104, 285–292 (1994).
[CrossRef]

Wang, Z. Q.

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

Wu, S.

Xu, X.

Yang, X.

Yin, S.

C. Uang, S. Yin, G. Lu, F. T. S. Yu, “Shift- and rotation-invariant interpattern heteroassociation (IHA) model,” Opt. Commun. 104, 285–292 (1994).
[CrossRef]

Yu, F. T. S.

Zhan, Y.

M. Lu, Y. Zhan, G. Mu, “Bipolar optical neural network with adaptive threshold,” Optik 91, 178–182 (1992).

Zhang, Y.

S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).

S. Chang, J. Shen, Y. Zhang, “A neural networks model with trinary interconnection weights based on orthogonal algorithm,” Acta Optica Sin. (China) 16, 1126–1132 (1996).

S. Chang, J. Shen, Y. Zhang, “High-capacity neural networks model with unipolar and binary interconnection,” Acta Photonica Sin. (China) 25, 866–870 (1996).

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

W. Huang, Y. Zhang, “A Monte Carlo learning algorithm for clipped neural network model,” Optoelectron. 7, 133–137 (1992).

Zhang, Y. M.

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

Zhang, Y. X.

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

Acta Optica Sin. (China) (1)

S. Chang, J. Shen, Y. Zhang, “A neural networks model with trinary interconnection weights based on orthogonal algorithm,” Acta Optica Sin. (China) 16, 1126–1132 (1996).

Acta Photonica Sin. (China) (1)

S. Chang, J. Shen, Y. Zhang, “High-capacity neural networks model with unipolar and binary interconnection,” Acta Photonica Sin. (China) 25, 866–870 (1996).

Appl. Opt. (5)

IEEE Trans. Circuits Syst. (1)

D. W. Tank, J. J. Hopfield, “Simple neural optimization network: an A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst. CAS-35, 533–541 (1986).
[CrossRef]

Neural Comput. (1)

C. Peterson, “Parallel distributed approaches to combinatorial optimization: benchmark studies on travelling salesman problem,” Neural Comput. 2, 261–269 (1990).
[CrossRef]

Opt. Commun. (1)

C. Uang, S. Yin, G. Lu, F. T. S. Yu, “Shift- and rotation-invariant interpattern heteroassociation (IHA) model,” Opt. Commun. 104, 285–292 (1994).
[CrossRef]

Opt. Eng. (1)

Y. Zhang, S. Chang, Z. Feng, S. Gao, G. Mu, “Multimode operation in an optical/digital neuroprocesser with 1024 neurons,” Opt. Eng. 35, 2132–2135 (1996).
[CrossRef]

Opt. Lett. (2)

Optik (2)

M. Lu, Y. Zhan, G. Mu, “Bipolar optical neural network with adaptive threshold,” Optik 91, 178–182 (1992).

S. Chang, Z. Song, C. Jin, Y. Zhang, “Adaptive clipped model and its optical implementation,” Optik 107, 135–140 (1998).

Optik. (1)

Y. H. Li, Y. M. Zhang, Z. Q. Wang, Y. Sun, Y. X. Zhang, “Real-time multi-target classification of the cascaded neural network and its opto-electric hybrid implementation,” Optik. 106, 91–95 (1997).

Optoelectron. (1)

W. Huang, Y. Zhang, “A Monte Carlo learning algorithm for clipped neural network model,” Optoelectron. 7, 133–137 (1992).

Proc. Natl. Acad. Sci. USA (1)

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA 79, 2254–2558 (1982).
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1

Schematic diagram of the cascaded model of the neural network. WTA, winner-take-all model.

Fig. 2
Fig. 2

Four different types of aircraft: bomber (upper left), fighter (upper right), airliner (lower left), rocket (lower right).

Fig. 3
Fig. 3

Dependence of averaged error rates of recovered vectors on M/N with the OA and the MOA.

Fig. 4
Fig. 4

Storage capacities versus Hamming distance N′ between the addressing vector and the stored vector with N = 100. MCA, Monte Carlo algorithm.

Fig. 5
Fig. 5

Schematic diagram of the optical implementation of the neural network. A/D, analog to digital; LCSA, liquid-crystal switch array; PC, personal computer.

Fig. 6
Fig. 6

Photograph of the CLA.

Fig. 7
Fig. 7

Arrangement of IWM and CLA: TSC, thresholding subchannel; SIWM, sub-IWM; AE, area encoding.

Fig. 8
Fig. 8

Results for optical associative memories: (a) four sets of Chinese phrases used as the stored patterns, (b) incomplete input patterns, (c) recovered results.

Tables (1)

Tables Icon

Table 1 Reference Codes for the Four Types of Aircraft

Equations (22)

Equations on this page are rendered with MathJax. Learn more.

Wij=m=1M VimVjm,  i, j=1, 2, , N.
neti0m=j=1N Ti0jSjm.
Si0mneti0m0.
Ei0=12m=1MSi0mneti0m-D2=12m=1MSi0mj=1N Ti0jSjm-D2,
Ei0=12m=1Mj1=1Nj2=1N Ti0j1Sj1mTi0j2Sj2m-2DSi0mj1=1N Ti0j1Sj1m+D2=12j1=1Nj2=1N Ti0j1Ti0j2m=1MSj1mSj2m-j1=1N Dm=1M Sj1mSi0mTi0j1+12 D2.
E=-12i=1Nj=1N WijVimVjm+i=1N IiVim,
Wij0=-m=1M SimSjm,
Ii0=-D m=1M SimSi0m.
Tkit+1=f j=1N Wij0Tkjt-Ii0,
fx =sgnx=1x>00x=0-1x<0.
S*i=Si-l=1i-1S*l, SiS*l S*l,
Wijop=m=1M Si*mSj*m.
neti0*m=j=1N Wi0jopSjm.
Ei0=12m=1MSi0mj=1N Ti0jSjm-DSi0mj0=1N Wi0j0opSj0m2=12j1=1Nj2=1N Ti0j1Ti0j2m=1MSj1m Sj2m-j1=1N D×m=1Mj0=1N Wi0j0opSj0mSj1mTi0j1+const.
Wij=-m=1M SimSjm,
Ii=-D m=1Mj0=1N Wi0j0opSj0mSim.
Ei0=12k=1Km=1MCi0kj=1N Wi0jXjmk-Di02=12j1=1Nj2=1N Wi0j1Wi0j2k=1Km=1M Xj1mkXj2mk-Di0j1=1Nk=1Km=1M Ci0kXj1mkWi0j1+const,
Wij0=-k=1Km=1MXimkXjmk,
Ii0=-Di0k=1Km=1MXimkCi0k.
Sijt+1=f l=1Ik=1I WijlkSlkt=f l=1Ik=1IWijlk+LSlkt-L l=1Ik=1I Slkt,
fx=10x0x<0  or  fx=1x0-1x<0.
Sijt+1=fl=1Ik=1I12LWijlk+LSlkt-12 Slkt=L=1fl=1Ik=1IWijlk+12 Slkt-12l=1Ik=1I Slkt.

Metrics