Abstract

A new approach to learning in a multilayer optical neural network based on holographically interconnected nonlinear devices is presented. The proposed network can learn the interconnections that form a distributed representation of a desired pattern transformation operation. The interconnections are formed in an adaptive and self-aligning fashion as volume holographic gratings in photorefractive crystals. Parallel arrays of globally space-integrated inner products diffracted by the interconnecting hologram illuminate arrays of nonlinear Fabry-Perot etalons for fast thresholding of the transformed patterns. A phase conjugated reference wave interferes with a backward propagating error signal to form holographic interference patterns which are time integrated in the volume of a photorefractive crystal to modify slowly and learn the appropriate self-aligning interconnections. This multilayer system performs an approximate implementation of the backpropagation learning procedure in a massively parallel high-speed nonlinear optical network.

© 1987 Optical Society of America

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, MA, 1986), Chap. 8.
  2. D. B. Parker, “Learning Logic,” Invention Report S81-64, File 1, Office of Technology Licensing, Stanford U. (Oct.1982).
  3. J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons,” Proc. Natl. Acad. Sci. USA 81, 3088 (1984).
    [CrossRef] [PubMed]
  4. S. Grossberg, Studies of Mind and Brain (Reidel, Boston, 1982).
  5. T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, Berlin, 1984).
  6. Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am. 256, 88 (1987).
    [CrossRef]
  7. D. Psaltis, N. H. Farhat, “Optical Information Processing Based on an Associative Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett 10, 98 (1985).
    [CrossRef] [PubMed]
  8. D. Z. Anderson, “Coherent Optical Eigenstate Memory,” Opt. Lett. 11, 56 (1986).
    [CrossRef] [PubMed]
  9. B. H. Soffer, G. J. Dunning, Y. Owechko, E. Marom, “Associative Holographic Memory with Feedback Using Phase-Conjugate Mirrors,” Opt. Lett. 11, 118 (1986).
    [CrossRef] [PubMed]
  10. A. Yariv, S. Kwong, “Associative Memories Based on Message-Bearing Optical Modes in Phase Conjugate Resonators,” Opt. Lett. 11, 186 (1986).
    [CrossRef] [PubMed]
  11. T. Jannson et al., “The Interconnectability of Neuro-Optic Processors,” Proc. Soc. Photo-Opt. Instrum. Eng. 698, 157 (1986).
  12. A. D. Fisher et al., “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 525, 196 (1986).
  13. M. Cohen, “Design of a New Medium for Volume Holographic Information Processing,” Appl. Opt. 25, 2288 (1986).
    [CrossRef] [PubMed]
  14. N. Farhat, “Architectures for Opto-Electronic Analogs of Self-Organizing Neural Networks,” in Technical Digest of Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 125.
  15. K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 16 (1987).
  16. K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” in Technical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 133.
  17. D. Psaltis, C. Park, “Nonlinear Discriminant Functions and Associative Memories,” in Proceedings, Conference on Neural Networks for Computing, Snowbird, UT, APS Conf. Proc.151 (1986).
  18. T. J. Sejnowski, C. R. Rosenberg, “NETtalk: a Parallel-Network that Learns to Read Aloud,” John Hopkins U., JHU/ EECS-86/01 (1986).
  19. B. Widrow, M. E. Hoff, “Adaptive Switching Circuits,” IRE Wescon Conv. Rec. 4, 96 (1960).
  20. H. M. Gibbs, Optical Bistability: Controlling Light with Light (Academic, New York, 1985).
  21. H. M. Gibbs et al., “Optical Modulation by Optical Tuning of a Cavity,” Appl. Phys. Lett. 34, 511 (1979).
    [CrossRef]
  22. A. W. Lohmann, “Polarization and Optical Logic,” Appl. Opt. 25,1594 (1986).
    [CrossRef] [PubMed]
  23. D. Psaltis et al., “Optical Neural Nets Implemented with Volume Holograms,” in Technical Digest Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987).
  24. J. P. Huignard et al., “Coherent Selective Erasure of Superimposed Volume Holograms in LiNbO3,” Appl. Phys. Lett. 26, 256 (1975).
    [CrossRef]
  25. D. L. Stabler et al., “Multiplier Storage and Erasure of Fixed Holograms in Fe-Doped LiNbO3,” Appl. Phys. Lett. 26, 182 (1975).
    [CrossRef]
  26. J. D. Cresser, P. Meystre, “The Role of Phases in the Trasient Dynamics of Nonlinear Interferometers,” in Optical Bistability, C. M. Bowden Ed. (Plenum, New York, 1980).
  27. A. Merrakchi, R. V. Johnson, A. Tanguay, “Polarization Properties of Photorefractive Diffraction in Electrooptic and Optically Active Sillenite Crystals (Bragg Regime),” J. Opt. Soc. Am. B. 3, 321 (1986).
    [CrossRef]
  28. A. E. Chiou, P. Yeh, “Parallel Image Subtraction Using a Phase Conjugate Michelson Interferometer,” Opt. Lett. 11, 306 (1986).
    [CrossRef] [PubMed]
  29. J. L. Jewell et al., “3pJ 82MHz Optical Logic Gates in a Room Temperature GaAs-AlGaAs Multiple Quantum Well Etalon,” Appl. Phys. Lett. 46, 918 (1985).
    [CrossRef]

1987 (2)

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am. 256, 88 (1987).
[CrossRef]

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 16 (1987).

1986 (10)

T. J. Sejnowski, C. R. Rosenberg, “NETtalk: a Parallel-Network that Learns to Read Aloud,” John Hopkins U., JHU/ EECS-86/01 (1986).

A. Merrakchi, R. V. Johnson, A. Tanguay, “Polarization Properties of Photorefractive Diffraction in Electrooptic and Optically Active Sillenite Crystals (Bragg Regime),” J. Opt. Soc. Am. B. 3, 321 (1986).
[CrossRef]

T. Jannson et al., “The Interconnectability of Neuro-Optic Processors,” Proc. Soc. Photo-Opt. Instrum. Eng. 698, 157 (1986).

A. D. Fisher et al., “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 525, 196 (1986).

D. Z. Anderson, “Coherent Optical Eigenstate Memory,” Opt. Lett. 11, 56 (1986).
[CrossRef] [PubMed]

B. H. Soffer, G. J. Dunning, Y. Owechko, E. Marom, “Associative Holographic Memory with Feedback Using Phase-Conjugate Mirrors,” Opt. Lett. 11, 118 (1986).
[CrossRef] [PubMed]

A. Yariv, S. Kwong, “Associative Memories Based on Message-Bearing Optical Modes in Phase Conjugate Resonators,” Opt. Lett. 11, 186 (1986).
[CrossRef] [PubMed]

A. E. Chiou, P. Yeh, “Parallel Image Subtraction Using a Phase Conjugate Michelson Interferometer,” Opt. Lett. 11, 306 (1986).
[CrossRef] [PubMed]

A. W. Lohmann, “Polarization and Optical Logic,” Appl. Opt. 25,1594 (1986).
[CrossRef] [PubMed]

M. Cohen, “Design of a New Medium for Volume Holographic Information Processing,” Appl. Opt. 25, 2288 (1986).
[CrossRef] [PubMed]

1985 (2)

J. L. Jewell et al., “3pJ 82MHz Optical Logic Gates in a Room Temperature GaAs-AlGaAs Multiple Quantum Well Etalon,” Appl. Phys. Lett. 46, 918 (1985).
[CrossRef]

D. Psaltis, N. H. Farhat, “Optical Information Processing Based on an Associative Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett 10, 98 (1985).
[CrossRef] [PubMed]

1984 (1)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons,” Proc. Natl. Acad. Sci. USA 81, 3088 (1984).
[CrossRef] [PubMed]

1979 (1)

H. M. Gibbs et al., “Optical Modulation by Optical Tuning of a Cavity,” Appl. Phys. Lett. 34, 511 (1979).
[CrossRef]

1975 (2)

J. P. Huignard et al., “Coherent Selective Erasure of Superimposed Volume Holograms in LiNbO3,” Appl. Phys. Lett. 26, 256 (1975).
[CrossRef]

D. L. Stabler et al., “Multiplier Storage and Erasure of Fixed Holograms in Fe-Doped LiNbO3,” Appl. Phys. Lett. 26, 182 (1975).
[CrossRef]

1960 (1)

B. Widrow, M. E. Hoff, “Adaptive Switching Circuits,” IRE Wescon Conv. Rec. 4, 96 (1960).

Abu-Mostafa, Y. S.

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am. 256, 88 (1987).
[CrossRef]

Anderson, D. Z.

Chiou, A. E.

Cohen, M.

Cresser, J. D.

J. D. Cresser, P. Meystre, “The Role of Phases in the Trasient Dynamics of Nonlinear Interferometers,” in Optical Bistability, C. M. Bowden Ed. (Plenum, New York, 1980).

Dunning, G. J.

Farhat, N.

N. Farhat, “Architectures for Opto-Electronic Analogs of Self-Organizing Neural Networks,” in Technical Digest of Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 125.

Farhat, N. H.

D. Psaltis, N. H. Farhat, “Optical Information Processing Based on an Associative Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett 10, 98 (1985).
[CrossRef] [PubMed]

Fisher, A. D.

A. D. Fisher et al., “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 525, 196 (1986).

Gibbs, H. M.

H. M. Gibbs et al., “Optical Modulation by Optical Tuning of a Cavity,” Appl. Phys. Lett. 34, 511 (1979).
[CrossRef]

H. M. Gibbs, Optical Bistability: Controlling Light with Light (Academic, New York, 1985).

Grossberg, S.

S. Grossberg, Studies of Mind and Brain (Reidel, Boston, 1982).

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, MA, 1986), Chap. 8.

Hoff, M. E.

B. Widrow, M. E. Hoff, “Adaptive Switching Circuits,” IRE Wescon Conv. Rec. 4, 96 (1960).

Hopfield, J. J.

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons,” Proc. Natl. Acad. Sci. USA 81, 3088 (1984).
[CrossRef] [PubMed]

Huignard, J. P.

J. P. Huignard et al., “Coherent Selective Erasure of Superimposed Volume Holograms in LiNbO3,” Appl. Phys. Lett. 26, 256 (1975).
[CrossRef]

Jannson, T.

T. Jannson et al., “The Interconnectability of Neuro-Optic Processors,” Proc. Soc. Photo-Opt. Instrum. Eng. 698, 157 (1986).

Jewell, J. L.

J. L. Jewell et al., “3pJ 82MHz Optical Logic Gates in a Room Temperature GaAs-AlGaAs Multiple Quantum Well Etalon,” Appl. Phys. Lett. 46, 918 (1985).
[CrossRef]

Johnson, R. V.

A. Merrakchi, R. V. Johnson, A. Tanguay, “Polarization Properties of Photorefractive Diffraction in Electrooptic and Optically Active Sillenite Crystals (Bragg Regime),” J. Opt. Soc. Am. B. 3, 321 (1986).
[CrossRef]

Kohonen, T.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, Berlin, 1984).

Kwong, S.

Lohmann, A. W.

Marom, E.

Merrakchi, A.

A. Merrakchi, R. V. Johnson, A. Tanguay, “Polarization Properties of Photorefractive Diffraction in Electrooptic and Optically Active Sillenite Crystals (Bragg Regime),” J. Opt. Soc. Am. B. 3, 321 (1986).
[CrossRef]

Meystre, P.

J. D. Cresser, P. Meystre, “The Role of Phases in the Trasient Dynamics of Nonlinear Interferometers,” in Optical Bistability, C. M. Bowden Ed. (Plenum, New York, 1980).

Owechko, Y.

Park, C.

D. Psaltis, C. Park, “Nonlinear Discriminant Functions and Associative Memories,” in Proceedings, Conference on Neural Networks for Computing, Snowbird, UT, APS Conf. Proc.151 (1986).

Parker, D. B.

D. B. Parker, “Learning Logic,” Invention Report S81-64, File 1, Office of Technology Licensing, Stanford U. (Oct.1982).

Psaltis, D.

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am. 256, 88 (1987).
[CrossRef]

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 16 (1987).

D. Psaltis, N. H. Farhat, “Optical Information Processing Based on an Associative Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett 10, 98 (1985).
[CrossRef] [PubMed]

D. Psaltis, C. Park, “Nonlinear Discriminant Functions and Associative Memories,” in Proceedings, Conference on Neural Networks for Computing, Snowbird, UT, APS Conf. Proc.151 (1986).

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” in Technical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 133.

D. Psaltis et al., “Optical Neural Nets Implemented with Volume Holograms,” in Technical Digest Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987).

Rosenberg, C. R.

T. J. Sejnowski, C. R. Rosenberg, “NETtalk: a Parallel-Network that Learns to Read Aloud,” John Hopkins U., JHU/ EECS-86/01 (1986).

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, MA, 1986), Chap. 8.

Sejnowski, T. J.

T. J. Sejnowski, C. R. Rosenberg, “NETtalk: a Parallel-Network that Learns to Read Aloud,” John Hopkins U., JHU/ EECS-86/01 (1986).

Soffer, B. H.

Stabler, D. L.

D. L. Stabler et al., “Multiplier Storage and Erasure of Fixed Holograms in Fe-Doped LiNbO3,” Appl. Phys. Lett. 26, 182 (1975).
[CrossRef]

Tanguay, A.

A. Merrakchi, R. V. Johnson, A. Tanguay, “Polarization Properties of Photorefractive Diffraction in Electrooptic and Optically Active Sillenite Crystals (Bragg Regime),” J. Opt. Soc. Am. B. 3, 321 (1986).
[CrossRef]

Wagner, K.

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 16 (1987).

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” in Technical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 133.

Widrow, B.

B. Widrow, M. E. Hoff, “Adaptive Switching Circuits,” IRE Wescon Conv. Rec. 4, 96 (1960).

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, MA, 1986), Chap. 8.

Yariv, A.

Yeh, P.

Appl. Opt. (2)

Appl. Phys. Lett. (4)

J. L. Jewell et al., “3pJ 82MHz Optical Logic Gates in a Room Temperature GaAs-AlGaAs Multiple Quantum Well Etalon,” Appl. Phys. Lett. 46, 918 (1985).
[CrossRef]

H. M. Gibbs et al., “Optical Modulation by Optical Tuning of a Cavity,” Appl. Phys. Lett. 34, 511 (1979).
[CrossRef]

J. P. Huignard et al., “Coherent Selective Erasure of Superimposed Volume Holograms in LiNbO3,” Appl. Phys. Lett. 26, 256 (1975).
[CrossRef]

D. L. Stabler et al., “Multiplier Storage and Erasure of Fixed Holograms in Fe-Doped LiNbO3,” Appl. Phys. Lett. 26, 182 (1975).
[CrossRef]

IRE Wescon Conv. Rec. 4 (1)

B. Widrow, M. E. Hoff, “Adaptive Switching Circuits,” IRE Wescon Conv. Rec. 4, 96 (1960).

J. Opt. Soc. Am. B. (1)

A. Merrakchi, R. V. Johnson, A. Tanguay, “Polarization Properties of Photorefractive Diffraction in Electrooptic and Optically Active Sillenite Crystals (Bragg Regime),” J. Opt. Soc. Am. B. 3, 321 (1986).
[CrossRef]

John Hopkins U., JHU/ EECS-86/01 (1)

T. J. Sejnowski, C. R. Rosenberg, “NETtalk: a Parallel-Network that Learns to Read Aloud,” John Hopkins U., JHU/ EECS-86/01 (1986).

Opt. Lett (1)

D. Psaltis, N. H. Farhat, “Optical Information Processing Based on an Associative Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett 10, 98 (1985).
[CrossRef] [PubMed]

Opt. Lett. (4)

Proc. Natl. Acad. Sci. USA (1)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons,” Proc. Natl. Acad. Sci. USA 81, 3088 (1984).
[CrossRef] [PubMed]

Proc. Soc. Photo-Opt. Instrum. Eng. (3)

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 16 (1987).

T. Jannson et al., “The Interconnectability of Neuro-Optic Processors,” Proc. Soc. Photo-Opt. Instrum. Eng. 698, 157 (1986).

A. D. Fisher et al., “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 525, 196 (1986).

Sci. Am. (1)

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am. 256, 88 (1987).
[CrossRef]

Other (10)

H. M. Gibbs, Optical Bistability: Controlling Light with Light (Academic, New York, 1985).

J. D. Cresser, P. Meystre, “The Role of Phases in the Trasient Dynamics of Nonlinear Interferometers,” in Optical Bistability, C. M. Bowden Ed. (Plenum, New York, 1980).

D. Psaltis et al., “Optical Neural Nets Implemented with Volume Holograms,” in Technical Digest Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987).

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” in Technical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 133.

D. Psaltis, C. Park, “Nonlinear Discriminant Functions and Associative Memories,” in Proceedings, Conference on Neural Networks for Computing, Snowbird, UT, APS Conf. Proc.151 (1986).

N. Farhat, “Architectures for Opto-Electronic Analogs of Self-Organizing Neural Networks,” in Technical Digest of Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1987), p. 125.

S. Grossberg, Studies of Mind and Brain (Reidel, Boston, 1982).

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, Berlin, 1984).

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, MA, 1986), Chap. 8.

D. B. Parker, “Learning Logic,” Invention Report S81-64, File 1, Office of Technology Licensing, Stanford U. (Oct.1982).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Metrics