Abstract

The bidirectional associative memory (BAM) is a powerful neural network paradigm that is well suited to optical implementation. The BAM is heteroassociative (of which autoassociative operation is a special case) and is guaranteed to converge to a stable final state regardless of the connection weight matrix used. The BAM is placed in a conceptual framework that facilitates comparison with other neural network models. Variations on the BAM such as the bidirectional optimal memory (BOM), the competitive BAM (CBAM), and the adaptive BAM (ABAM) indicate some of the interesting directions this simple structure may evolve, leading in a natural progression toward the power of a model such as the Carpenter-Grossberg ART. The simplicity of the BAM invites uncomplicated optical implementations. BAM designs based on optical matrix—vector multipliers (MVMs) and on volume holographic connections are presented. Spatial light modulator (SLM) device designs currently under development to support the MVM BAMs are given.

© 1987 Optical Society of America

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. Kosko, “Bidirectional Associative Memories,” IEEE Trans. Syst. Man Cybern. SMC (1987), in press.
  2. J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
    [CrossRef] [PubMed]
  3. M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
    [CrossRef]
  4. G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
    [CrossRef]
  5. B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.
  6. W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
    [CrossRef]
  7. T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).
  8. B. Kosko, “Adaptive Bidirectional Associative Memories,” Appl. Opt. 26, 4947 (1987).
    [CrossRef] [PubMed]
  9. B. Kosko, “Competitive Adaptive Bidirectional Associative Memories,” in Proceedings, International Conference on Neural Networks, San Diego (June 1987).
  10. J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
    [CrossRef] [PubMed]
  11. G. R. Knight, “Holographic Associative Memory and Processor,” Appl. Opt. 14, 1088 (1975).
    [CrossRef] [PubMed]
  12. M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
    [CrossRef]
  13. V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
    [CrossRef]
  14. C. C. Guest, T. K. Gaylord, “Truth-Table Look-Up Optical Processing Utilizing Binary and Residue Arithmetic,” Appl. Opt. 19, 1201 (1980).
    [CrossRef] [PubMed]
  15. N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469 (1985).
    [CrossRef] [PubMed]
  16. A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.
  17. N. H. Farhat, “Architectures for Optoelectronic Analogs of Self-Organizing Neural Networks,” Opt. Lett. 12, 448 (1987).
    [CrossRef] [PubMed]
  18. Y. Owechko, G. J. Dunning, E. Marom, B. H. Soffer, “Holographic Associative Memory with Nonlinearities in the Correlation Domain,” Appl. Opt. 26, 1900 (1987).
    [CrossRef] [PubMed]
  19. T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
    [CrossRef]
  20. E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
    [CrossRef]
  21. D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
    [CrossRef]
  22. R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).
  23. R. A. Athale, BDM Corp.; personal correspondence (May1986).
  24. S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
    [CrossRef]
  25. R. Hecht-Nielsen, “Nearest Matched Filter Classification of Spatiotemporal Patterns,” Appl. Opt. 26,1892 (1987).
    [CrossRef] [PubMed]
  26. P. J. van Heerden, “Theory of Optical Information Storage in Solids,” Appl. Opt. 2,393 (1963).
    [CrossRef]
  27. T. K. Gaylord, “Digital Data Storage,” in Handbook of Optical Holography, H. J. Caulfield, Ed. (Academic, New York, 1979), pp. 379–414.

1987 (9)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

N. H. Farhat, “Architectures for Optoelectronic Analogs of Self-Organizing Neural Networks,” Opt. Lett. 12, 448 (1987).
[CrossRef] [PubMed]

R. Hecht-Nielsen, “Nearest Matched Filter Classification of Spatiotemporal Patterns,” Appl. Opt. 26,1892 (1987).
[CrossRef] [PubMed]

Y. Owechko, G. J. Dunning, E. Marom, B. H. Soffer, “Holographic Associative Memory with Nonlinearities in the Correlation Domain,” Appl. Opt. 26, 1900 (1987).
[CrossRef] [PubMed]

B. Kosko, “Adaptive Bidirectional Associative Memories,” Appl. Opt. 26, 4947 (1987).
[CrossRef] [PubMed]

1986 (2)

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

1985 (1)

1984 (1)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
[CrossRef] [PubMed]

1983 (1)

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

1982 (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
[CrossRef] [PubMed]

1980 (1)

1978 (1)

V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
[CrossRef]

1975 (1)

1970 (1)

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

1963 (1)

1943 (1)

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Anderson, D. Z.

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

Athale, R. A.

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

R. A. Athale, BDM Corp.; personal correspondence (May1986).

Carpenter, G. A.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

Cohen, M. A.

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

Drabik, T. J.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Dunning, G. J.

Erie, M. C.

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

Esener, S. C.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Farhat, N. H.

Fisher, A. D.

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

Friedlander, C. B.

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

Gaylord, T. K.

C. C. Guest, T. K. Gaylord, “Truth-Table Look-Up Optical Processing Utilizing Binary and Residue Arithmetic,” Appl. Opt. 19, 1201 (1980).
[CrossRef] [PubMed]

T. K. Gaylord, “Digital Data Storage,” in Handbook of Optical Holography, H. J. Caulfield, Ed. (Academic, New York, 1979), pp. 379–414.

Grossberg, S.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

Guest, C.

B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.

Guest, C. C.

Hecht-Nielsen, R.

Hopfield, J. J.

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
[CrossRef] [PubMed]

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
[CrossRef] [PubMed]

Jannson, T.

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Karagaleff, C.

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Knight, G. R.

Kohonen, T.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).

Kosko, B.

B. Kosko, “Adaptive Bidirectional Associative Memories,” Appl. Opt. 26, 4947 (1987).
[CrossRef] [PubMed]

B. Kosko, “Bidirectional Associative Memories,” IEEE Trans. Syst. Man Cybern. SMC (1987), in press.

B. Kosko, “Competitive Adaptive Bidirectional Associative Memories,” in Proceedings, International Conference on Neural Networks, San Diego (June 1987).

B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.

Kushner, B. G.

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

Lee, J. N.

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

Lee, S. H.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Marom, E.

McCulloch, W. S.

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Morozov, V. N.

V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
[CrossRef]

Nemoto, T.

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

Nishida, N.

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

Owechko, Y.

Paek, E.

Paek, E. G.

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

Pitts, W.

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Prata, A.

Psaltis, D.

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469 (1985).
[CrossRef] [PubMed]

Sakaguchi, M.

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

Soffer, B. H.

Stoll, H.

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Title, M. A.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

van Heerden, P. J.

Wang, J. H.

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Appl. Opt. (7)

Bull. Math. Biophys. (1)

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Imminent in Nervous Activity,” Bull. Math. Biophys. 5, 115 (1943).
[CrossRef]

Comput. Vision Graphics Image Process. (1)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54 (1987).
[CrossRef]

IEEE Trans. Comput. (1)

M. Sakaguchi, N. Nishida, T. Nemoto, “A New Associative Memory System Utilizing Holography,” IEEE Trans. Comput. C-19, 1174 (1970).
[CrossRef]

IEEE Trans. Syst. Man Cybern. (1)

M. A. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

J. Opt. Soc. Am. A (1)

T. Jannson, C. Karagaleff, H. Stoll, “Photorefractive LiNbO3 as a Storage Medium for High-Density Optical Neural Networks,” J. Opt. Soc. Am. A 3(13), P 64 (1986).
[CrossRef]

Opt. Eng. (3)

E. G. Paek, D. Psaltis, “Optical Associative Memory Using Fourier Transform Holograms,” Opt. Eng. 26, 428 (1987).
[CrossRef]

D. Z. Anderson, M. C. Erie, “Resonator Memories and Optical Novelty Filters,” Opt. Eng. 25, 434 (1987).
[CrossRef]

S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, S. H. Lee, “One-Dimensional Silicon/PLZT Spatial Light Modulators,” Opt. Eng. 26,406 (1987).
[CrossRef]

Opt. Lett. (1)

Proc. Natl. Acad. Sci. U.S.A. (2)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).
[CrossRef] [PubMed]

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
[CrossRef] [PubMed]

Proc. Soc. Photo-Opt. Instrum. Eng. (1)

R. A. Athale, C. B. Friedlander, B. G. Kushner, “Attentive Associative Architectures and Their Implications to Optical Computing,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179 (1986).

Sov. J. Quantum Electron. (1)

V. N. Morozov, “Associative Parallel Search Memory,” Sov. J. Quantum Electron. 8, 1 (1978).
[CrossRef]

Technical Digest of OSA Topical Meeting on Optical Computing (1)

A. D. Fisher, J. N. Lee, “Optical Associative Processing Elements with Versatile Adaptive Learning Capabilities,” Technical Digest of OSA Topical Meeting on Optical Computing, vol. 11, pp. 137–140, March1987.

Other (6)

T. K. Gaylord, “Digital Data Storage,” in Handbook of Optical Holography, H. J. Caulfield, Ed. (Academic, New York, 1979), pp. 379–414.

R. A. Athale, BDM Corp.; personal correspondence (May1986).

B. Kosko, “Competitive Adaptive Bidirectional Associative Memories,” in Proceedings, International Conference on Neural Networks, San Diego (June 1987).

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).

B. Kosko, “Bidirectional Associative Memories,” IEEE Trans. Syst. Man Cybern. SMC (1987), in press.

B. Kosko, C. Guest, “Optical Bidirectional Associative Memories”, Proc. Soc. Photo-Opt. Instrum. Eng.758, (1987), in press.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Metrics