Abstract

The bidirectional associative memory (BAM) is a powerful neural network paradigm that is well suited to optical implementation. The BAM is heteroassociative (of which autoassociative operation is a special case) and is guaranteed to converge to a stable final state regardless of the connection weight matrix used. The BAM is placed in a conceptual framework that facilitates comparison with other neural network models. Variations on the BAM such as the bidirectional optimal memory (BOM), the competitive BAM (CBAM), and the adaptive BAM (ABAM) indicate some of the interesting directions this simple structure may evolve, leading in a natural progression toward the power of a model such as the Carpenter-Grossberg ART. The simplicity of the BAM invites uncomplicated optical implementations. BAM designs based on optical matrix—vector multipliers (MVMs) and on volume holographic connections are presented. Spatial light modulator (SLM) device designs currently under development to support the MVM BAMs are given.

© 1987 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. Kosko, “Bidirectional Associative Memories,” IEEE Trans. Syst. Man Cybern. SMC (1987), in press.
  2. J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
    [CrossRef] [PubMed]
  3. M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
    [CrossRef]
  4. G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
    [CrossRef]
  5. B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.
  6. W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
    [CrossRef]
  7. T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).
  8. B. Kosko, “Adaptive Bidirectional Associative Memories,” Appl. Opt. 26, 4947 (1987).
    [CrossRef] [PubMed]
  9. B. Kosko, “Competitive Adaptive Bidirectional Associative Memories,” in Proceedings, International Conference on Neural Networks, San Diego (June 1987).
  10. J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
    [CrossRef] [PubMed]
  11. G. R. Knight, “Holographic Associative Memory and Processor,” Appl. Opt. 14, 1088 (1975).
    [CrossRef] [PubMed]
  12. M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
    [CrossRef]
  13. V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
    [CrossRef]
  14. C. C. Guest, T. K. Gaylord, “Truth-Table Look-Up Optical Processing Utilizing Binary and Residue Arithmetic,” Appl. Opt. 19, 1201 (1980).
    [CrossRef] [PubMed]
  15. N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469 (1985).
    [CrossRef] [PubMed]
  16. A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.
  17. N. H. Farhat, “Architectures for Optoelectronic Analogs of Self-Organizing Neural Networks,” Opt. Lett. 12, 448 (1987).
    [CrossRef] [PubMed]
  18. Y. Owechko, G. J. Dunning, E. Marom, B. H. Soffer, “Holographic Associative Memory with Nonlinearities in the Correlation Domain,” Appl. Opt. 26, 1900 (1987).
    [CrossRef] [PubMed]
  19. T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
    [CrossRef]
  20. E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
    [CrossRef]
  21. D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
    [CrossRef]
  22. R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).
  23. R. A. Athale, BDM Corp.; personal correspondence (May1986).
  24. S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
    [CrossRef]
  25. R. Hecht-Nielsen, “Nearest Matched Filter Classification of Spatiotemporal Patterns,” Appl. Opt. 26,1892 (1987).
    [CrossRef] [PubMed]
  26. P. J. van Heerden, “Theory of Optical Information Storage in Solids,” Appl. Opt. 2,393 (1963).
    [CrossRef]
  27. T. K. Gaylord, “Digital Data Storage,” in Handbook of Optical Holography, H. J. Caulfield, Ed. (Academic, New York, 1979), pp. 379–414.

1987 (9)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

B. Kosko, “Adaptive Bidirectional Associative Memories,” Appl. Opt. 26, 4947 (1987).
[CrossRef] [PubMed]

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

N. H. Farhat, “Architectures for Optoelectronic Analogs of Self-Organizing Neural Networks,” Opt. Lett. 12, 448 (1987).
[CrossRef] [PubMed]

Y. Owechko, G. J. Dunning, E. Marom, B. H. Soffer, “Holographic Associative Memory with Nonlinearities in the Correlation Domain,” Appl. Opt. 26, 1900 (1987).
[CrossRef] [PubMed]

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

R. Hecht-Nielsen, “Nearest Matched Filter Classification of Spatiotemporal Patterns,” Appl. Opt. 26,1892 (1987).
[CrossRef] [PubMed]

1986 (2)

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

1985 (1)

1984 (1)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
[CrossRef] [PubMed]

1983 (1)

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

1982 (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
[CrossRef] [PubMed]

1980 (1)

1978 (1)

V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
[CrossRef]

1975 (1)

1970 (1)

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

1963 (1)

1943 (1)

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Anderson, D. Z.

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

Athale, R. A.

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

R. A. Athale, BDM Corp.; personal correspondence (May1986).

Carpenter, G. A.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

Cohen, M. A.

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

Drabik, T. J.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Dunning, G. J.

Erie, M. C.

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

Esener, S. C.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Farhat, N. H.

Fisher, A. D.

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

Friedlander, C. B.

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

Gaylord, T. K.

C. C. Guest, T. K. Gaylord, “Truth-Table Look-Up Optical Processing Utilizing Binary and Residue Arithmetic,” Appl. Opt. 19, 1201 (1980).
[CrossRef] [PubMed]

T. K. Gaylord, “Digital Data Storage,” in Handbook of Optical Holography, H. J. Caulfield, Ed. (Academic, New York, 1979), pp. 379–414.

Grossberg, S.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

Guest, C.

B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.

Guest, C. C.

Hecht-Nielsen, R.

Hopfield, J. J.

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
[CrossRef] [PubMed]

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
[CrossRef] [PubMed]

Jannson, T.

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Karagaleff, C.

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Knight, G. R.

Kohonen, T.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).

Kosko, B.

B. Kosko, “Adaptive Bidirectional Associative Memories,” Appl. Opt. 26, 4947 (1987).
[CrossRef] [PubMed]

B. Kosko, “Competitive Adaptive Bidirectional Associative Memories,” in Proceedings, International Conference on Neural Networks, San Diego (June 1987).

B. Kosko, “Bidirectional Associative Memories,” IEEE Trans. Syst. Man Cybern. SMC (1987), in press.

B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.

Kushner, B. G.

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

Lee, J. N.

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

Lee, S. H.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Marom, E.

McCulloch, W. S.

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Morozov, V. N.

V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
[CrossRef]

Nemoto, T.

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

Nishida, N.

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

Owechko, Y.

Paek, E.

Paek, E. G.

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

Pitts, W.

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Prata, A.

Psaltis, D.

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469 (1985).
[CrossRef] [PubMed]

Sakaguchi, M.

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

Soffer, B. H.

Stoll, H.

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Title, M. A.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

van Heerden, P. J.

Wang, J. H.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Appl. Opt. (7)

Bull. Math. Biophys. (1)

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Comput. Vision Graphics Image Process. (1)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

IEEE Trans. Comput. (1)

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

IEEE Trans. Syst. Man Cybern. (1)

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

J. Opt. Soc. Am. A (1)

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Opt. Eng. (3)

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Opt. Lett. (1)

Proc. Natl. Acad. Sci. U.S.A. (2)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
[CrossRef] [PubMed]

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
[CrossRef] [PubMed]

Proc. Soc. Photo-Opt. Instrum. Eng. (1)

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

Sov. J. Quantum Electron. (1)

V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
[CrossRef]

Technical Digest of OSA Topical Meeting on Optical Computing (1)

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

Other (6)

T. K. Gaylord, “Digital Data Storage,” in Handbook of Optical Holography, H. J. Caulfield, Ed. (Academic, New York, 1979), pp. 379–414.

R. A. Athale, BDM Corp.; personal correspondence (May1986).

B. Kosko, “Bidirectional Associative Memories,” IEEE Trans. Syst. Man Cybern. SMC (1987), in press.

B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).

B. Kosko, “Competitive Adaptive Bidirectional Associative Memories,” in Proceedings, International Conference on Neural Networks, San Diego (June 1987).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

The BAM is composed of two fields of processing elements. Weighted connections exist between the fields, but not within them.

Fig. 2
Fig. 2

Structure of the connection matrix for several neural network models is compared: (a) the Hopfield model, (b) two independent systems, (c) two connected systems, (d) hierarchical system with connections between adjacent levels (D,E) and nonadjacent levels (F), (e) the BAM, (f) the CBAM.

Fig. 3
Fig. 3

This BAM implementation based on an optical matrix–vector multiplier uses linear arrays of paired detector and sources for the neural element fields. Light passes through the connection matrix in both directions.

Fig. 4
Fig. 4

This optical matrix-vector multiplier BAM system is folded with a mirror to allow both BAM fields to lie in the same plane.

Fig. 5
Fig. 5

This spatial light modulator device consists of alternating stripes of silicon photodetectors and electrooptic modulators. The signal from each detector is amplified and thresholded by silicon circuitry that then drives the associated modulator.

Fig. 6
Fig. 6

Two line modulator arrays can be used with a connection matrix transparency to implement a sandwich form of optical matrix–vector multiplier BAM.

Fig. 7
Fig. 7

Single signal line carrying a voltage proportional to the number of active elements can be used to produce the same effect as inhibitory connections between all elements in a field.

Fig. 8
Fig. 8

This element of a spatial light modulator device contains all the components needed to implement an adaptive BAM connection matrix.

Fig. 9
Fig. 9

Interfield connections in this BAM implementation are implemented with a volume reflection hologram.

Fig. 10
Fig. 10

Volume transmission hologram is used in this BAM design.

Metrics