Abstract

We report the optical implementation of a neural network based on a nearest matched filter algorithm and extensive lateral inhibition. Extremely rapid learning is demonstrated in pattern recognition and autonomous control applications, without introducing processing artifacts such as spurious states and ambiguous solutions. The optical implementation is achieved with a reconfigurable, bipolar mask-type crossbar switch based on an inexpensive liquid crystal spatial light modulator.

© 1989 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
    [CrossRef]
  2. R. Hecht-Nielsen, “Neurocomputing: Picking the Human Brain,” IEEE Spectrum, 36–41 (March, 1988).
    [CrossRef]
  3. D. Gabor, “Associative Holographic Memories,” IBM J. Res. Dev. 13, 156–159 (1969).
    [CrossRef]
  4. Special Issue on Neural Networks, Appl. Opt. 26. No. 23 (1987).
  5. Special Issue on Neural Networks, Appl. Opt. 28. No. 2 (1989).
  6. Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am., 88–95 (Sept.1987).
    [CrossRef]
  7. Y. Li, L. Wang, P. Heos, G. Zhang, X. C. Liang, R. R. Alfano, “Ultrafast Noncollinear Second-Harmonic-Generation-Based 4 × 4 Optical Switching Array,” Opt. Lett. 14, 347–349 (1989).
    [CrossRef] [PubMed]
  8. M. Ishikawa et al., “Optical Association: A Simple Model for Optical Associative Memory,” Appl. Opt. 28, 291–301 (1989).
    [CrossRef] [PubMed]
  9. N. Farhat, Z-Y Shae, “Bimodal Stochastic Optical Learning Machine,” Proc. Int. Conf. on Neural Networks, II, IEEE Cat. No. 88CH2632-8 (1988) p. 365.
    [CrossRef]
  10. A. D. Fisher, C. L. Giles, J. N. Lee, “Associative Processor Architectures for Optical Computing,” J. Opt. Soc. Am. A 1, 1337 (1984).
  11. A. D. Fisher, “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 196–204 (1986).
  12. D. Psaltis, D. Brady, K. Wagner, “Adaptive Optical Networks Using Photorefractive Crystals,” Appl. Opt. 27, 1752–1759 (1988).
    [CrossRef]
  13. J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
    [CrossRef] [PubMed]
  14. D. Psaltis, N. Farhat, “Optical Information Processing Based on an Associative-Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett. 10, 98–100 (1985).
    [CrossRef] [PubMed]
  15. N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469–1475 (1985).
    [CrossRef] [PubMed]
  16. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland Eds. (MIT Press, Cambridge, 1986) pp. 318–362.
  17. P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched-Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing (AIP Conf. Proc. 151, Snowbird, UT, 1986) p. 194.
  18. T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).
  19. N. H. Farhat, “Robust Signal Recovery and Recognition with Optical Analogs of Neural Nets and Spurious Memory Discrimination,” Proc. Soc. Photo-Opt. Instrum. Eng. 700, 283–288 (1986).
  20. N. H. Farhat, “Optoelectronic Analogs of Self-Programming Neural Nets: Architecture and Methodologies for Implementing Fast Stochastic Learning by Simulated Annealing,” Appl. Opt. 26, 5093–5103 (1987).
    [CrossRef] [PubMed]
  21. G. R. Gindi, A. F. Gmitro, K. Parthasarathy, “Hopfield Model Associative Memory with Nonzero-Diagonal Terms in Memory Matrix,” Appl. Opt. 27, 129–134 (1988).
    [CrossRef] [PubMed]
  22. R. P. Lippman, “An Introduction to Computing with Neural Nets,” IEEE ASSP Mag., 4–10 (Apr.1987).
    [CrossRef]
  23. G. E. Hinton, T. J. Sejnowski, “Learning and Relearning in Boltzmann Machines,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, 1986) pp. 282–317.
  24. D. Michie, R. A. Chambers, “BOXES: An Experiment in Adaptive Control,” in Machine Intelligence 2, E. Dale, D. Nichie, Eds. (Oliver and Boyd, Edinburgh, 1968).
  25. A. G. Barto, R. S. Sutton, C. W. Anderson, “Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,” IEEE Trans. SMC-13 (1983) 834–846.
  26. P. J. de Groot, R. J. Noll, “A Reconfigurable, Bipolar Analog Optical Crossbar Switch,” Appl. Opt. 28, 1582–1587 (1989).
    [CrossRef] [PubMed]

1989 (4)

1988 (4)

G. R. Gindi, A. F. Gmitro, K. Parthasarathy, “Hopfield Model Associative Memory with Nonzero-Diagonal Terms in Memory Matrix,” Appl. Opt. 27, 129–134 (1988).
[CrossRef] [PubMed]

D. Psaltis, D. Brady, K. Wagner, “Adaptive Optical Networks Using Photorefractive Crystals,” Appl. Opt. 27, 1752–1759 (1988).
[CrossRef]

N. Farhat, Z-Y Shae, “Bimodal Stochastic Optical Learning Machine,” Proc. Int. Conf. on Neural Networks, II, IEEE Cat. No. 88CH2632-8 (1988) p. 365.
[CrossRef]

R. Hecht-Nielsen, “Neurocomputing: Picking the Human Brain,” IEEE Spectrum, 36–41 (March, 1988).
[CrossRef]

1987 (4)

Special Issue on Neural Networks, Appl. Opt. 26. No. 23 (1987).

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am., 88–95 (Sept.1987).
[CrossRef]

R. P. Lippman, “An Introduction to Computing with Neural Nets,” IEEE ASSP Mag., 4–10 (Apr.1987).
[CrossRef]

N. H. Farhat, “Optoelectronic Analogs of Self-Programming Neural Nets: Architecture and Methodologies for Implementing Fast Stochastic Learning by Simulated Annealing,” Appl. Opt. 26, 5093–5103 (1987).
[CrossRef] [PubMed]

1986 (2)

A. D. Fisher, “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 196–204 (1986).

N. H. Farhat, “Robust Signal Recovery and Recognition with Optical Analogs of Neural Nets and Spurious Memory Discrimination,” Proc. Soc. Photo-Opt. Instrum. Eng. 700, 283–288 (1986).

1985 (2)

1984 (1)

A. D. Fisher, C. L. Giles, J. N. Lee, “Associative Processor Architectures for Optical Computing,” J. Opt. Soc. Am. A 1, 1337 (1984).

1983 (1)

A. G. Barto, R. S. Sutton, C. W. Anderson, “Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,” IEEE Trans. SMC-13 (1983) 834–846.

1982 (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
[CrossRef] [PubMed]

1969 (1)

D. Gabor, “Associative Holographic Memories,” IBM J. Res. Dev. 13, 156–159 (1969).
[CrossRef]

1943 (1)

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Abu-Mostafa, Y. S.

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am., 88–95 (Sept.1987).
[CrossRef]

Alfano, R. R.

Anderson, C. W.

A. G. Barto, R. S. Sutton, C. W. Anderson, “Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,” IEEE Trans. SMC-13 (1983) 834–846.

Barto, A. G.

A. G. Barto, R. S. Sutton, C. W. Anderson, “Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,” IEEE Trans. SMC-13 (1983) 834–846.

Brady, D.

Chambers, R. A.

D. Michie, R. A. Chambers, “BOXES: An Experiment in Adaptive Control,” in Machine Intelligence 2, E. Dale, D. Nichie, Eds. (Oliver and Boyd, Edinburgh, 1968).

de Groot, P. J.

Farhat, N.

N. Farhat, Z-Y Shae, “Bimodal Stochastic Optical Learning Machine,” Proc. Int. Conf. on Neural Networks, II, IEEE Cat. No. 88CH2632-8 (1988) p. 365.
[CrossRef]

D. Psaltis, N. Farhat, “Optical Information Processing Based on an Associative-Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett. 10, 98–100 (1985).
[CrossRef] [PubMed]

Farhat, N. H.

Fisher, A. D.

A. D. Fisher, “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 196–204 (1986).

A. D. Fisher, C. L. Giles, J. N. Lee, “Associative Processor Architectures for Optical Computing,” J. Opt. Soc. Am. A 1, 1337 (1984).

Gabor, D.

D. Gabor, “Associative Holographic Memories,” IBM J. Res. Dev. 13, 156–159 (1969).
[CrossRef]

Giles, C. L.

A. D. Fisher, C. L. Giles, J. N. Lee, “Associative Processor Architectures for Optical Computing,” J. Opt. Soc. Am. A 1, 1337 (1984).

Gindi, G. R.

Gmitro, A. F.

Grant, P. M.

P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched-Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing (AIP Conf. Proc. 151, Snowbird, UT, 1986) p. 194.

Hecht-Nielsen, R.

R. Hecht-Nielsen, “Neurocomputing: Picking the Human Brain,” IEEE Spectrum, 36–41 (March, 1988).
[CrossRef]

Heos, P.

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland Eds. (MIT Press, Cambridge, 1986) pp. 318–362.

G. E. Hinton, T. J. Sejnowski, “Learning and Relearning in Boltzmann Machines,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, 1986) pp. 282–317.

Hopfield, J. J.

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
[CrossRef] [PubMed]

Ishikawa, M.

Kohonen, T.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).

Lee, J. N.

A. D. Fisher, C. L. Giles, J. N. Lee, “Associative Processor Architectures for Optical Computing,” J. Opt. Soc. Am. A 1, 1337 (1984).

Li, Y.

Liang, X. C.

Lippman, R. P.

R. P. Lippman, “An Introduction to Computing with Neural Nets,” IEEE ASSP Mag., 4–10 (Apr.1987).
[CrossRef]

McCulloch, W. S.

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Michie, D.

D. Michie, R. A. Chambers, “BOXES: An Experiment in Adaptive Control,” in Machine Intelligence 2, E. Dale, D. Nichie, Eds. (Oliver and Boyd, Edinburgh, 1968).

Noll, R. J.

Paek, E.

Parthasarathy, K.

Pitts, W.

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Prata, A.

Psaltis, D.

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland Eds. (MIT Press, Cambridge, 1986) pp. 318–362.

Sage, J. P.

P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched-Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing (AIP Conf. Proc. 151, Snowbird, UT, 1986) p. 194.

Sejnowski, T. J.

G. E. Hinton, T. J. Sejnowski, “Learning and Relearning in Boltzmann Machines,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, 1986) pp. 282–317.

Shae, Z-Y

N. Farhat, Z-Y Shae, “Bimodal Stochastic Optical Learning Machine,” Proc. Int. Conf. on Neural Networks, II, IEEE Cat. No. 88CH2632-8 (1988) p. 365.
[CrossRef]

Sutton, R. S.

A. G. Barto, R. S. Sutton, C. W. Anderson, “Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,” IEEE Trans. SMC-13 (1983) 834–846.

Wagner, K.

Wang, L.

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland Eds. (MIT Press, Cambridge, 1986) pp. 318–362.

Zhang, G.

Appl. Opt. (6)

Bull. Math. Biophys. (1)

W. S. McCulloch, W. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

IBM J. Res. Dev. (1)

D. Gabor, “Associative Holographic Memories,” IBM J. Res. Dev. 13, 156–159 (1969).
[CrossRef]

IEEE ASSP Mag. (1)

R. P. Lippman, “An Introduction to Computing with Neural Nets,” IEEE ASSP Mag., 4–10 (Apr.1987).
[CrossRef]

IEEE Spectrum (1)

R. Hecht-Nielsen, “Neurocomputing: Picking the Human Brain,” IEEE Spectrum, 36–41 (March, 1988).
[CrossRef]

IEEE Trans. (1)

A. G. Barto, R. S. Sutton, C. W. Anderson, “Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,” IEEE Trans. SMC-13 (1983) 834–846.

J. Opt. Soc. Am. A (1)

A. D. Fisher, C. L. Giles, J. N. Lee, “Associative Processor Architectures for Optical Computing,” J. Opt. Soc. Am. A 1, 1337 (1984).

Opt. Lett. (2)

Proc. Int. Conf. on Neural Networks (1)

N. Farhat, Z-Y Shae, “Bimodal Stochastic Optical Learning Machine,” Proc. Int. Conf. on Neural Networks, II, IEEE Cat. No. 88CH2632-8 (1988) p. 365.
[CrossRef]

Proc. Natl. Acad. Sci. U.S.A. (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
[CrossRef] [PubMed]

Proc. Soc. Photo-Opt. Instrum. Eng. (2)

A. D. Fisher, “Implementation of Adaptive Associative Optical Computing Elements,” Proc. Soc. Photo-Opt. Instrum. Eng. 625, 196–204 (1986).

N. H. Farhat, “Robust Signal Recovery and Recognition with Optical Analogs of Neural Nets and Spurious Memory Discrimination,” Proc. Soc. Photo-Opt. Instrum. Eng. 700, 283–288 (1986).

Sci. Am. (1)

Y. S. Abu-Mostafa, D. Psaltis, “Optical Neural Computers,” Sci. Am., 88–95 (Sept.1987).
[CrossRef]

Special Issue on Neural Networks, Appl. Opt. (2)

Special Issue on Neural Networks, Appl. Opt. 26. No. 23 (1987).

Special Issue on Neural Networks, Appl. Opt. 28. No. 2 (1989).

Other (5)

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning Internal Representations by Error Propagation” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland Eds. (MIT Press, Cambridge, 1986) pp. 318–362.

P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched-Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing (AIP Conf. Proc. 151, Snowbird, UT, 1986) p. 194.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, New York, 1984).

G. E. Hinton, T. J. Sejnowski, “Learning and Relearning in Boltzmann Machines,” in Parallel Distributed Processing, Vol. 1, D. E. Rumelhart, J. L. McClelland, Eds. (MIT Press, Cambridge, 1986) pp. 282–317.

D. Michie, R. A. Chambers, “BOXES: An Experiment in Adaptive Control,” in Machine Intelligence 2, E. Dale, D. Nichie, Eds. (Oliver and Boyd, Edinburgh, 1968).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1

Simplified block diagram illustrates the global operations required for an associative memory based on matched-filter operations. A neural network was designed for performing the operations prescribed by this kind of associative memory.

Fig. 2
Fig. 2

Feed-forward network topology of the associative memory involves a large number of parallel interconnects.

Fig. 3
Fig. 3

Error history of the optical/electronic associative memory shows a gradual decrease over time in the difference between the memory's response and the desired response. Local minima, such as the one that occurred here at presentation number 5, are avoided by introducing random fluctuations in the memory whenever the error is nonzero.

Fig. 4
Fig. 4

Unsupervised neural network learning scenario tested on the optical/electronic computer involved the simplified model of a space vehicle. The network learned to orient and stabilize the vehicle without any prior knowledge of the meaning of the sensor input patterns or the function of the actuator jets.

Fig. 5
Fig. 5

Optical configuration for an optical crossbar switch permits the use of low-contrast (1.3 to 1) reconfigurable mask. The crossbar performs a mapping from a bipolar input vector to an output vector, with fan-in accomplished by analog addition.

Fig. 6
Fig. 6

Optical crossbar breadboard was interfaced to a microcomputer to complete the model hybrid optical/electronic computer. The input LeD array is in the foreground, facing the collimating lens, the holographic lenslet array, the LCTV and the output detectors mounted on the same optical rail. LED driver and detector preamplifier circuits are on the surface of the optical bench.

Equations (28)

Equations on this page are rendered with MathJax. Learn more.

Y = f ( i = 1 N U i X i ) ,
C μ = i = 1 N U i μ X i .
Y j = θ ( μ = 1 M U j μ C μ ) ,
θ ( b ) = { 1 for b < 0 + 1 for b > 0 } .
Y j = θ ( μ = 1 M U j μ i = 1 N U i μ X i ) ,
Y j = θ ( i = 1 N μ = 1 M U j μ U i μ X i ) .
T j i = μ = 1 M U j μ U i μ ,
Y j = θ ( i = 1 N T j i X i ) .
U j μ = 1 N i = 1 N T j i U i μ ,
Y j = μ = 1 M V j μ S μ .
Z = μ = 1 M S μ ( t )
S μ ( t + 1 ) = Φ ( χ S μ ( t ) Z ξ ) ,
Φ ( b ) = { 0 for b < 0 b for b > 0 }
χ = M M 1 , ξ = 1 M 1 .
( χ S max ( t ) Z ξ ) > 0 ,
( χ S min ( t ) Z ξ ) < 0 .
U i μ = U i μ + η X i S μ 1 < U j μ < 1
V j μ = V j μ + η Y j S μ 1 < V j μ < 1
E = μ = 1 M E μ
E μ = N k = 1 N V ̂ k μ Y k μ .
Q = α | θ θ f | + β | ω | ,
θ θ i = t 0 t f ω d t
ω ω i = t 0 t f adt α = Γ / I ,
θ 1 = ½ a Δ t 2 + ω Δ t + θ 0
ω 1 = a Δ t + ω 0 .
X 1 if ω > 0 X 3 if ( θ θ f ) < 0 X 2 if ( θ θ f ) > 0 X 4 if ω < 0
a = ( Y 1 Y 2 ) 2 ,
Y j = i W j i X i ,

Metrics