Abstract

To fully use the advantages of optics in optical neural networks, an incoherent optical neuron (ION) model is proposed. The main purpose of this model is to provide for the requisite subtraction of signals without the phase sensitivity of a fully coherent system and without the cumbrance of photon–electron conversion and electronic subtraction. The ION model can subtract inhibitory from excitatory neuron inputs by using two device responses. Functionally it accommodates positive and negative weights, excitatory and inhibitory inputs, non-negative neuron outputs, and can be used in a variety of neural network models. This technique can implement conventional inner-product neuron units and Grossberg’s mass action law neuron units. Some implementation considerations, such as the effect of nonlinearities on device response, noise, and fan-in/fan-out capability, are discussed and simulated by computer. An experimental demonstration of optical excitation and inhibition on a 2-D array of neuron units using a single Hughes liquid crystal light valve is also reported.

© 1990 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. Z. Anderson, D. M. Lininger, “Dynamic Optical Interconnects: Volume Holograms as Optical Two-Port Operators,” Appl. Opt. 26, 5031–5038 (1987).
    [CrossRef] [PubMed]
  2. J. F. Ebersole, “Optical Image Subtraction,” Opt. Eng. 14, 436–447 (1975).
  3. E. Marom, J. Grinberg, “Subtraction of Images with Incoherent Illumination in Real Time,” Appl. Opt. 16, 3086–3087 (1977).
    [CrossRef] [PubMed]
  4. D. Psaltis, X.-G. Gu, D. Brady, “Fractical Sampling Grids for Holographic Interconnections,” Proc. Soc. Photo-Opt. Instrum. Eng. 963, 468–474 (1988).
  5. D. Psaltis, D. Brady, K. Wagner, “Adaptive Optical Networks Using Photorefractive Crystals,” Appl. Opt. 27, 1752–1759 (1988).
    [CrossRef]
  6. N. H. Farhat, D. Psaltis, A. Prata, E. Pack, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469–1475 (1985).
    [CrossRef] [PubMed]
  7. I. Shariv, A. A. Friesem, “All-Optical Neural Network with Inhibitory Neurons,” Opt. Lett. 14, 485–487 (1989).
    [CrossRef] [PubMed]
  8. R. D. TeKolste, C. C. Guest, “Optical Competitive Neural Network with Optical Feedback,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 625–629.
  9. W. S. McCulloch, W. H. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
    [CrossRef]
  10. S. Grossberg, “Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks,” Studies in Appl. Math. 52, 213–257 (1973).
  11. K. Fukushima, “Cognitron: a Self-Organizing Multilayered Neural Network,” Biol. Cybernet. 20, 121–136 (1975).
    [CrossRef]
  12. S-I. Amari, “Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements,” IEEE Trans. Comput. C-21, 1197–1206 (1972).
    [CrossRef]
  13. J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Ability,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
    [CrossRef] [PubMed]
  14. K. Fukushima, “Neocognitron: a Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position,” Biol. Cybernet. 36, 193–202 (1980).
    [CrossRef]
  15. S. Miyake, K. Fukushima, “A Neural Network Model for the Mechanism of Feature Extraction,” Biol. Cybernet. 50, 377–384 (1984).
    [CrossRef]
  16. K. Fukushima, “A Hierarchical Neural Network Model for Associative Memory,” Biol. Cybernet. 50, 105–113 (1984).
    [CrossRef]
  17. K. Fukushima, “A Neural Network Model for Selective Attention in Visual Pattern Recognition,” Biol. Cybernet. 55, 5–15 (1986).
    [CrossRef]
  18. K. Fukushima, “Neural Network Model for Selective Attention in Visual Pattern Recognition and Associative Recall,” Appl. Opt. 26, 4985–4992 (1987).
    [CrossRef] [PubMed]
  19. S. Grossberg, “On the Development of Feature Detectors in the Visual Cortex with Applications to Learning and Reaction-Diffusion Systems,” Biol. Cybernet. 21, 145–159 (1976).
    [CrossRef]
  20. S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: I. Parallel Development and Coding of Neural Feature Detectors,” Biol. Cybernet. 23, 121–134 (1976).
    [CrossRef]
  21. S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: II. Feedback Expectation, Olfaction, Illusions,” Biol. Cybernet. 23, 187–202 (1976).
  22. G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54–115 (1987).
    [CrossRef]
  23. G. A. Carpenter, S. Grossberg, “ART 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns,” Appl. Opt. 26, 4919–4930 (1987).
    [CrossRef] [PubMed]
  24. C. von der Malsburg, “Self-Organization of Orientation Sensitive Cells in the Striate Cortex,” Kybernetik 14, 85–100 (1973).
    [CrossRef] [PubMed]
  25. A. F. Gmitro, G. R. Gindi, “Optical Neurocomputer for Implementation of the Marr-Poggio Stereo Algorithm,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 599–606.
  26. S. A. Ellias, S. Grossberg, “Pattern Formation, Contrast Control and Oscillations in Short Term Memory of Shunting On-Center Off-Surround Networks,” Biol. Cybernet. 20, 69–98 (1975).
    [CrossRef]
  27. M. Wang, A. Freeman, Neural Function (Little, Brown, Boston, 1987).
  28. C. F. Stevens, “The Neuron,” Sci. Am. 241, 54–65 (1979).
    [CrossRef] [PubMed]
  29. G. M. Shepherd, “Microcircuits in the Nervous System,” Sci. Am. 238, 92–103 (1978).
    [CrossRef]
  30. A. C. Walker, “Application of Bistable Optical Logic Gate Arrays to All-Optical Digital Parallel Processing,” Appl. Opt. 25, 1578–1585 (1986).
    [CrossRef] [PubMed]
  31. D. A. B. Miller et al., “The Quantum Well Self-Electrooptic Effect Device: Optoelectronic Bistability and Oscillation, and Self-Linearized Modulation,” IEEE J. Quantum Electron. QE-21, 1462–1476 (1985).
    [CrossRef]
  32. A. L. Lentine et al., “Symmetric Self-Electro-Optic Effect Device: Optical Set-Reset Latch,” Appl. Phys. Lett. 52, 1419–1421 (1988).
    [CrossRef]
  33. W. P. Bleha et al., “Application of the Liquid Crystal Light Valve to Real-Time Optical Data Processing,” Opt. Eng. 17, 371–384 (1978).
  34. B. K. Jenkins, A. A. Sawchuk, T. C. Strand, R. Forchheimer, B. H. Soffer, “Sequential Optical Logic Implementation,” Appl. Opt. 23, 3455–3464 (1984).
    [CrossRef] [PubMed]
  35. C. W. Stirk, S. M. Rovnyak, R. A. Athale, “Effects of System Noise on an Optical Implementation of an Additive Lateral Inhibitory Network,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 615–624.
  36. B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” Opt. Lett. 13, 892–894 (1988).
    [CrossRef] [PubMed]
  37. S. Grossberg, “Biological Competition: Decision Rules, Pattern Formation, and Oscillations,” Proc. Natl. Acad. Sci. U.S.A. 77, 2338–2342 (1980).
    [CrossRef] [PubMed]
  38. S-I. Amari, “Competitive and Cooperative Aspects in Dynamics of Neural Excitation and Self-Organization,” in Proceedings, Competition and Cooperation in Neural Nets, S. Amari, M. A. Arbib, Eds. (Springer-Verlag, New York, 1982), pp. 1–28.
    [CrossRef]
  39. U. Efron, E. Marom, B. H. Soffer, “Array Division by Optical Computing,” in Topical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1985), paper TuF2.
  40. B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” J. Opt. Soc. Am. A 4(13), P127 (1987).
  41. C. H. Wang, B. K. Jenkins, “Implementation Considerations of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. II, (1988), pp. 403–410.
    [CrossRef]
  42. C. H. Wang, B. K. Jenkins, “Implementation of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE Third Annual Parallel Processing Symposium, Fullerton, CA (Mar. 1989).
  43. C. H. Wang, B. K. Jenkins, “Subtracting Incoherent Optical Neuron: Experimental Demonstration,” in Technical Digest, OSA Annual Meeting (Optical Society of America, Washington, DC, 1989), paper WU1.

1989 (1)

1988 (5)

C. H. Wang, B. K. Jenkins, “Implementation Considerations of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. II, (1988), pp. 403–410.
[CrossRef]

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” Opt. Lett. 13, 892–894 (1988).
[CrossRef] [PubMed]

A. L. Lentine et al., “Symmetric Self-Electro-Optic Effect Device: Optical Set-Reset Latch,” Appl. Phys. Lett. 52, 1419–1421 (1988).
[CrossRef]

D. Psaltis, X.-G. Gu, D. Brady, “Fractical Sampling Grids for Holographic Interconnections,” Proc. Soc. Photo-Opt. Instrum. Eng. 963, 468–474 (1988).

D. Psaltis, D. Brady, K. Wagner, “Adaptive Optical Networks Using Photorefractive Crystals,” Appl. Opt. 27, 1752–1759 (1988).
[CrossRef]

1987 (8)

G. A. Carpenter, S. Grossberg, “ART 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns,” Appl. Opt. 26, 4919–4930 (1987).
[CrossRef] [PubMed]

K. Fukushima, “Neural Network Model for Selective Attention in Visual Pattern Recognition and Associative Recall,” Appl. Opt. 26, 4985–4992 (1987).
[CrossRef] [PubMed]

D. Z. Anderson, D. M. Lininger, “Dynamic Optical Interconnects: Volume Holograms as Optical Two-Port Operators,” Appl. Opt. 26, 5031–5038 (1987).
[CrossRef] [PubMed]

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54–115 (1987).
[CrossRef]

R. D. TeKolste, C. C. Guest, “Optical Competitive Neural Network with Optical Feedback,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 625–629.

A. F. Gmitro, G. R. Gindi, “Optical Neurocomputer for Implementation of the Marr-Poggio Stereo Algorithm,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 599–606.

C. W. Stirk, S. M. Rovnyak, R. A. Athale, “Effects of System Noise on an Optical Implementation of an Additive Lateral Inhibitory Network,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 615–624.

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” J. Opt. Soc. Am. A 4(13), P127 (1987).

1986 (2)

K. Fukushima, “A Neural Network Model for Selective Attention in Visual Pattern Recognition,” Biol. Cybernet. 55, 5–15 (1986).
[CrossRef]

A. C. Walker, “Application of Bistable Optical Logic Gate Arrays to All-Optical Digital Parallel Processing,” Appl. Opt. 25, 1578–1585 (1986).
[CrossRef] [PubMed]

1985 (2)

N. H. Farhat, D. Psaltis, A. Prata, E. Pack, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469–1475 (1985).
[CrossRef] [PubMed]

D. A. B. Miller et al., “The Quantum Well Self-Electrooptic Effect Device: Optoelectronic Bistability and Oscillation, and Self-Linearized Modulation,” IEEE J. Quantum Electron. QE-21, 1462–1476 (1985).
[CrossRef]

1984 (3)

B. K. Jenkins, A. A. Sawchuk, T. C. Strand, R. Forchheimer, B. H. Soffer, “Sequential Optical Logic Implementation,” Appl. Opt. 23, 3455–3464 (1984).
[CrossRef] [PubMed]

S. Miyake, K. Fukushima, “A Neural Network Model for the Mechanism of Feature Extraction,” Biol. Cybernet. 50, 377–384 (1984).
[CrossRef]

K. Fukushima, “A Hierarchical Neural Network Model for Associative Memory,” Biol. Cybernet. 50, 105–113 (1984).
[CrossRef]

1982 (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Ability,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
[CrossRef] [PubMed]

1980 (2)

K. Fukushima, “Neocognitron: a Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position,” Biol. Cybernet. 36, 193–202 (1980).
[CrossRef]

S. Grossberg, “Biological Competition: Decision Rules, Pattern Formation, and Oscillations,” Proc. Natl. Acad. Sci. U.S.A. 77, 2338–2342 (1980).
[CrossRef] [PubMed]

1979 (1)

C. F. Stevens, “The Neuron,” Sci. Am. 241, 54–65 (1979).
[CrossRef] [PubMed]

1978 (2)

G. M. Shepherd, “Microcircuits in the Nervous System,” Sci. Am. 238, 92–103 (1978).
[CrossRef]

W. P. Bleha et al., “Application of the Liquid Crystal Light Valve to Real-Time Optical Data Processing,” Opt. Eng. 17, 371–384 (1978).

1977 (1)

1976 (3)

S. Grossberg, “On the Development of Feature Detectors in the Visual Cortex with Applications to Learning and Reaction-Diffusion Systems,” Biol. Cybernet. 21, 145–159 (1976).
[CrossRef]

S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: I. Parallel Development and Coding of Neural Feature Detectors,” Biol. Cybernet. 23, 121–134 (1976).
[CrossRef]

S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: II. Feedback Expectation, Olfaction, Illusions,” Biol. Cybernet. 23, 187–202 (1976).

1975 (3)

J. F. Ebersole, “Optical Image Subtraction,” Opt. Eng. 14, 436–447 (1975).

K. Fukushima, “Cognitron: a Self-Organizing Multilayered Neural Network,” Biol. Cybernet. 20, 121–136 (1975).
[CrossRef]

S. A. Ellias, S. Grossberg, “Pattern Formation, Contrast Control and Oscillations in Short Term Memory of Shunting On-Center Off-Surround Networks,” Biol. Cybernet. 20, 69–98 (1975).
[CrossRef]

1973 (2)

S. Grossberg, “Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks,” Studies in Appl. Math. 52, 213–257 (1973).

C. von der Malsburg, “Self-Organization of Orientation Sensitive Cells in the Striate Cortex,” Kybernetik 14, 85–100 (1973).
[CrossRef] [PubMed]

1972 (1)

S-I. Amari, “Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements,” IEEE Trans. Comput. C-21, 1197–1206 (1972).
[CrossRef]

1943 (1)

W. S. McCulloch, W. H. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Amari, S-I.

S-I. Amari, “Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements,” IEEE Trans. Comput. C-21, 1197–1206 (1972).
[CrossRef]

S-I. Amari, “Competitive and Cooperative Aspects in Dynamics of Neural Excitation and Self-Organization,” in Proceedings, Competition and Cooperation in Neural Nets, S. Amari, M. A. Arbib, Eds. (Springer-Verlag, New York, 1982), pp. 1–28.
[CrossRef]

Anderson, D. Z.

Athale, R. A.

C. W. Stirk, S. M. Rovnyak, R. A. Athale, “Effects of System Noise on an Optical Implementation of an Additive Lateral Inhibitory Network,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 615–624.

Bleha, W. P.

W. P. Bleha et al., “Application of the Liquid Crystal Light Valve to Real-Time Optical Data Processing,” Opt. Eng. 17, 371–384 (1978).

Brady, D.

D. Psaltis, D. Brady, K. Wagner, “Adaptive Optical Networks Using Photorefractive Crystals,” Appl. Opt. 27, 1752–1759 (1988).
[CrossRef]

D. Psaltis, X.-G. Gu, D. Brady, “Fractical Sampling Grids for Holographic Interconnections,” Proc. Soc. Photo-Opt. Instrum. Eng. 963, 468–474 (1988).

Carpenter, G. A.

G. A. Carpenter, S. Grossberg, “ART 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns,” Appl. Opt. 26, 4919–4930 (1987).
[CrossRef] [PubMed]

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54–115 (1987).
[CrossRef]

Ebersole, J. F.

J. F. Ebersole, “Optical Image Subtraction,” Opt. Eng. 14, 436–447 (1975).

Efron, U.

U. Efron, E. Marom, B. H. Soffer, “Array Division by Optical Computing,” in Topical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1985), paper TuF2.

Ellias, S. A.

S. A. Ellias, S. Grossberg, “Pattern Formation, Contrast Control and Oscillations in Short Term Memory of Shunting On-Center Off-Surround Networks,” Biol. Cybernet. 20, 69–98 (1975).
[CrossRef]

Farhat, N. H.

Forchheimer, R.

Freeman, A.

M. Wang, A. Freeman, Neural Function (Little, Brown, Boston, 1987).

Friesem, A. A.

Fukushima, K.

K. Fukushima, “Neural Network Model for Selective Attention in Visual Pattern Recognition and Associative Recall,” Appl. Opt. 26, 4985–4992 (1987).
[CrossRef] [PubMed]

K. Fukushima, “A Neural Network Model for Selective Attention in Visual Pattern Recognition,” Biol. Cybernet. 55, 5–15 (1986).
[CrossRef]

S. Miyake, K. Fukushima, “A Neural Network Model for the Mechanism of Feature Extraction,” Biol. Cybernet. 50, 377–384 (1984).
[CrossRef]

K. Fukushima, “A Hierarchical Neural Network Model for Associative Memory,” Biol. Cybernet. 50, 105–113 (1984).
[CrossRef]

K. Fukushima, “Neocognitron: a Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position,” Biol. Cybernet. 36, 193–202 (1980).
[CrossRef]

K. Fukushima, “Cognitron: a Self-Organizing Multilayered Neural Network,” Biol. Cybernet. 20, 121–136 (1975).
[CrossRef]

Gindi, G. R.

A. F. Gmitro, G. R. Gindi, “Optical Neurocomputer for Implementation of the Marr-Poggio Stereo Algorithm,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 599–606.

Gmitro, A. F.

A. F. Gmitro, G. R. Gindi, “Optical Neurocomputer for Implementation of the Marr-Poggio Stereo Algorithm,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 599–606.

Grinberg, J.

Grossberg, S.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54–115 (1987).
[CrossRef]

G. A. Carpenter, S. Grossberg, “ART 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns,” Appl. Opt. 26, 4919–4930 (1987).
[CrossRef] [PubMed]

S. Grossberg, “Biological Competition: Decision Rules, Pattern Formation, and Oscillations,” Proc. Natl. Acad. Sci. U.S.A. 77, 2338–2342 (1980).
[CrossRef] [PubMed]

S. Grossberg, “On the Development of Feature Detectors in the Visual Cortex with Applications to Learning and Reaction-Diffusion Systems,” Biol. Cybernet. 21, 145–159 (1976).
[CrossRef]

S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: I. Parallel Development and Coding of Neural Feature Detectors,” Biol. Cybernet. 23, 121–134 (1976).
[CrossRef]

S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: II. Feedback Expectation, Olfaction, Illusions,” Biol. Cybernet. 23, 187–202 (1976).

S. A. Ellias, S. Grossberg, “Pattern Formation, Contrast Control and Oscillations in Short Term Memory of Shunting On-Center Off-Surround Networks,” Biol. Cybernet. 20, 69–98 (1975).
[CrossRef]

S. Grossberg, “Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks,” Studies in Appl. Math. 52, 213–257 (1973).

Gu, X.-G.

D. Psaltis, X.-G. Gu, D. Brady, “Fractical Sampling Grids for Holographic Interconnections,” Proc. Soc. Photo-Opt. Instrum. Eng. 963, 468–474 (1988).

Guest, C. C.

R. D. TeKolste, C. C. Guest, “Optical Competitive Neural Network with Optical Feedback,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 625–629.

Hopfield, J. J.

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Ability,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
[CrossRef] [PubMed]

Jenkins, B. K.

C. H. Wang, B. K. Jenkins, “Implementation Considerations of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. II, (1988), pp. 403–410.
[CrossRef]

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” Opt. Lett. 13, 892–894 (1988).
[CrossRef] [PubMed]

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” J. Opt. Soc. Am. A 4(13), P127 (1987).

B. K. Jenkins, A. A. Sawchuk, T. C. Strand, R. Forchheimer, B. H. Soffer, “Sequential Optical Logic Implementation,” Appl. Opt. 23, 3455–3464 (1984).
[CrossRef] [PubMed]

C. H. Wang, B. K. Jenkins, “Implementation of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE Third Annual Parallel Processing Symposium, Fullerton, CA (Mar. 1989).

C. H. Wang, B. K. Jenkins, “Subtracting Incoherent Optical Neuron: Experimental Demonstration,” in Technical Digest, OSA Annual Meeting (Optical Society of America, Washington, DC, 1989), paper WU1.

Lentine, A. L.

A. L. Lentine et al., “Symmetric Self-Electro-Optic Effect Device: Optical Set-Reset Latch,” Appl. Phys. Lett. 52, 1419–1421 (1988).
[CrossRef]

Lininger, D. M.

Marom, E.

E. Marom, J. Grinberg, “Subtraction of Images with Incoherent Illumination in Real Time,” Appl. Opt. 16, 3086–3087 (1977).
[CrossRef] [PubMed]

U. Efron, E. Marom, B. H. Soffer, “Array Division by Optical Computing,” in Topical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1985), paper TuF2.

McCulloch, W. S.

W. S. McCulloch, W. H. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Miller, D. A. B.

D. A. B. Miller et al., “The Quantum Well Self-Electrooptic Effect Device: Optoelectronic Bistability and Oscillation, and Self-Linearized Modulation,” IEEE J. Quantum Electron. QE-21, 1462–1476 (1985).
[CrossRef]

Miyake, S.

S. Miyake, K. Fukushima, “A Neural Network Model for the Mechanism of Feature Extraction,” Biol. Cybernet. 50, 377–384 (1984).
[CrossRef]

Pack, E.

Pitts, W. H.

W. S. McCulloch, W. H. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Prata, A.

Psaltis, D.

Rovnyak, S. M.

C. W. Stirk, S. M. Rovnyak, R. A. Athale, “Effects of System Noise on an Optical Implementation of an Additive Lateral Inhibitory Network,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 615–624.

Sawchuk, A. A.

Shariv, I.

Shepherd, G. M.

G. M. Shepherd, “Microcircuits in the Nervous System,” Sci. Am. 238, 92–103 (1978).
[CrossRef]

Soffer, B. H.

B. K. Jenkins, A. A. Sawchuk, T. C. Strand, R. Forchheimer, B. H. Soffer, “Sequential Optical Logic Implementation,” Appl. Opt. 23, 3455–3464 (1984).
[CrossRef] [PubMed]

U. Efron, E. Marom, B. H. Soffer, “Array Division by Optical Computing,” in Topical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1985), paper TuF2.

Stevens, C. F.

C. F. Stevens, “The Neuron,” Sci. Am. 241, 54–65 (1979).
[CrossRef] [PubMed]

Stirk, C. W.

C. W. Stirk, S. M. Rovnyak, R. A. Athale, “Effects of System Noise on an Optical Implementation of an Additive Lateral Inhibitory Network,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 615–624.

Strand, T. C.

TeKolste, R. D.

R. D. TeKolste, C. C. Guest, “Optical Competitive Neural Network with Optical Feedback,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 625–629.

von der Malsburg, C.

C. von der Malsburg, “Self-Organization of Orientation Sensitive Cells in the Striate Cortex,” Kybernetik 14, 85–100 (1973).
[CrossRef] [PubMed]

Wagner, K.

Walker, A. C.

Wang, C. H.

C. H. Wang, B. K. Jenkins, “Implementation Considerations of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. II, (1988), pp. 403–410.
[CrossRef]

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” Opt. Lett. 13, 892–894 (1988).
[CrossRef] [PubMed]

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” J. Opt. Soc. Am. A 4(13), P127 (1987).

C. H. Wang, B. K. Jenkins, “Implementation of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE Third Annual Parallel Processing Symposium, Fullerton, CA (Mar. 1989).

C. H. Wang, B. K. Jenkins, “Subtracting Incoherent Optical Neuron: Experimental Demonstration,” in Technical Digest, OSA Annual Meeting (Optical Society of America, Washington, DC, 1989), paper WU1.

Wang, M.

M. Wang, A. Freeman, Neural Function (Little, Brown, Boston, 1987).

Appl. Opt. (8)

Appl. Phys. Lett. (1)

A. L. Lentine et al., “Symmetric Self-Electro-Optic Effect Device: Optical Set-Reset Latch,” Appl. Phys. Lett. 52, 1419–1421 (1988).
[CrossRef]

Biol. Cybernet. (9)

K. Fukushima, “Cognitron: a Self-Organizing Multilayered Neural Network,” Biol. Cybernet. 20, 121–136 (1975).
[CrossRef]

K. Fukushima, “Neocognitron: a Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position,” Biol. Cybernet. 36, 193–202 (1980).
[CrossRef]

S. Miyake, K. Fukushima, “A Neural Network Model for the Mechanism of Feature Extraction,” Biol. Cybernet. 50, 377–384 (1984).
[CrossRef]

K. Fukushima, “A Hierarchical Neural Network Model for Associative Memory,” Biol. Cybernet. 50, 105–113 (1984).
[CrossRef]

K. Fukushima, “A Neural Network Model for Selective Attention in Visual Pattern Recognition,” Biol. Cybernet. 55, 5–15 (1986).
[CrossRef]

S. Grossberg, “On the Development of Feature Detectors in the Visual Cortex with Applications to Learning and Reaction-Diffusion Systems,” Biol. Cybernet. 21, 145–159 (1976).
[CrossRef]

S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: I. Parallel Development and Coding of Neural Feature Detectors,” Biol. Cybernet. 23, 121–134 (1976).
[CrossRef]

S. Grossberg, “Adaptive Pattern Classification and Universal Recoding: II. Feedback Expectation, Olfaction, Illusions,” Biol. Cybernet. 23, 187–202 (1976).

S. A. Ellias, S. Grossberg, “Pattern Formation, Contrast Control and Oscillations in Short Term Memory of Shunting On-Center Off-Surround Networks,” Biol. Cybernet. 20, 69–98 (1975).
[CrossRef]

Bull. Math. Biophys. (1)

W. S. McCulloch, W. H. Pitts, “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bull. Math. Biophys. 5, 115–133 (1943).
[CrossRef]

Comput. Vision Graphics Image Process. (1)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vision Graphics Image Process. 37, 54–115 (1987).
[CrossRef]

IEEE First International Conference on Neural Networks (4)

R. D. TeKolste, C. C. Guest, “Optical Competitive Neural Network with Optical Feedback,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 625–629.

A. F. Gmitro, G. R. Gindi, “Optical Neurocomputer for Implementation of the Marr-Poggio Stereo Algorithm,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 599–606.

C. W. Stirk, S. M. Rovnyak, R. A. Athale, “Effects of System Noise on an Optical Implementation of an Additive Lateral Inhibitory Network,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. 3 (1987), pp. 615–624.

C. H. Wang, B. K. Jenkins, “Implementation Considerations of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE First International Conference on Neural Networks, Vol. II, (1988), pp. 403–410.
[CrossRef]

IEEE J. Quantum Electron. (1)

D. A. B. Miller et al., “The Quantum Well Self-Electrooptic Effect Device: Optoelectronic Bistability and Oscillation, and Self-Linearized Modulation,” IEEE J. Quantum Electron. QE-21, 1462–1476 (1985).
[CrossRef]

IEEE Trans. Comput. (1)

S-I. Amari, “Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements,” IEEE Trans. Comput. C-21, 1197–1206 (1972).
[CrossRef]

J. Opt. Soc. Am. A (1)

B. K. Jenkins, C. H. Wang, “Model for an Incoherent Optical Neuron that Subtracts,” J. Opt. Soc. Am. A 4(13), P127 (1987).

Kybernetik (1)

C. von der Malsburg, “Self-Organization of Orientation Sensitive Cells in the Striate Cortex,” Kybernetik 14, 85–100 (1973).
[CrossRef] [PubMed]

Opt. Eng. (2)

J. F. Ebersole, “Optical Image Subtraction,” Opt. Eng. 14, 436–447 (1975).

W. P. Bleha et al., “Application of the Liquid Crystal Light Valve to Real-Time Optical Data Processing,” Opt. Eng. 17, 371–384 (1978).

Opt. Lett. (2)

Proc. Natl. Acad. Sci. U.S.A. (2)

S. Grossberg, “Biological Competition: Decision Rules, Pattern Formation, and Oscillations,” Proc. Natl. Acad. Sci. U.S.A. 77, 2338–2342 (1980).
[CrossRef] [PubMed]

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Ability,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
[CrossRef] [PubMed]

Proc. Soc. Photo-Opt. Instrum. Eng. (1)

D. Psaltis, X.-G. Gu, D. Brady, “Fractical Sampling Grids for Holographic Interconnections,” Proc. Soc. Photo-Opt. Instrum. Eng. 963, 468–474 (1988).

Sci. Am. (2)

C. F. Stevens, “The Neuron,” Sci. Am. 241, 54–65 (1979).
[CrossRef] [PubMed]

G. M. Shepherd, “Microcircuits in the Nervous System,” Sci. Am. 238, 92–103 (1978).
[CrossRef]

Studies in Appl. Math. (1)

S. Grossberg, “Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks,” Studies in Appl. Math. 52, 213–257 (1973).

Other (5)

M. Wang, A. Freeman, Neural Function (Little, Brown, Boston, 1987).

S-I. Amari, “Competitive and Cooperative Aspects in Dynamics of Neural Excitation and Self-Organization,” in Proceedings, Competition and Cooperation in Neural Nets, S. Amari, M. A. Arbib, Eds. (Springer-Verlag, New York, 1982), pp. 1–28.
[CrossRef]

U. Efron, E. Marom, B. H. Soffer, “Array Division by Optical Computing,” in Topical Digest, Topical Meeting on Optical Computing (Optical Society of America, Washington, DC, 1985), paper TuF2.

C. H. Wang, B. K. Jenkins, “Implementation of a Subtracting Incoherent Optical Neuron,” in Proceedings, IEEE Third Annual Parallel Processing Symposium, Fullerton, CA (Mar. 1989).

C. H. Wang, B. K. Jenkins, “Subtracting Incoherent Optical Neuron: Experimental Demonstration,” in Technical Digest, OSA Annual Meeting (Optical Society of America, Washington, DC, 1989), paper WU1.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (17)

Fig. 1
Fig. 1

Sample procedure to calculate membrane potential based on a uniprocessor. The value shown in parenthesis is the number of clock cycles for an Intel 80386 processor. Clock period is 50 ns. Only 45% of the time is used in actual computation.

Fig. 2
Fig. 2

Paradigm for an optical neural network.

Fig. 3
Fig. 3

The ION model: (a) normalized inhibitory (I) element; (b) unnormalized I element; (c) nonlinear (N) element; and (d) the ION structure.

Fig. 4
Fig. 4

Homogeneous case example: typical characteristic of a Hughes liquid crystal light valve serving as both I and N elements. Regions A and C are used for the I and N elements, respectively. Region B serves to separate the I element from the N element.

Fig. 5
Fig. 5

Modeling the device noise of an incoherent optical neuron: I element with (a) drift (vertical/horizontal), (b) gain variation; N element with (c) horizontal drift or variation of the bias; (d) vertical drift, and (e) gain variation.

Fig. 6
Fig. 6

On-center off-surround competitive neural network: (a) network and (b) interconnection weight strengths as a function of distance from a neuron. Interconnections in this network are space invariant.

Fig. 7
Fig. 7

Network responses for different attenuation factors, s1, at the input to the nonlinear inhibitory element. Each horizontal scan line of the figure represents a separate simulation on a 1–D input: (a) input pattern; (b) s1 = 1.0, no attenuation; (c) s1 = 1.5; (d) s1 = 2.5; (e) s1 = 3.0; and (f) s1 = 3.5. The ideal output is essentially identical to (d).

Fig. 8
Fig. 8

Normalized mean square error (nmse) measure of the network response for temporally correlated noise. Three noise sources, N I + , N I *, and N N + are simulated. (a) Normalized mean square error of the net output vs maximum noise perturbation p for correlation periods (T) ranging from 1 to 50. The input level is 0.7. (b) Output nmse plot for different noise perturbations and input levels (T = 50).

Fig. 9
Fig. 9

Simulation results for spatially and temporally correlated noise; sc is the spatial correlation range and T is the temporal correlation period. Three noise sources are simulated simultaneously, each with perturbation p = ±10%. The nmse is the normalized mean square error of the network output. Only spatially correlated noise (T = 1) with (a) sc = 3, nmse = 0.08, (b) sc = 13, nmse = 0.13; spatially and temporally correlated noise (T = 25) with (c) sc = 3, nmse = 0.14, (d) sc 13, nmse 0.18.

Fig. 10
Fig. 10

Effect of device drift in the ION. The drift is uniform over all neuron units. High frequency drift (T = 1) with (a) p = ±10%, nmse = 0.02, (b) p = ±25%, nmse = 0.09; low frequency drift (T = 50) with (c) p = ±10%, nmse = 0.07, (d) p = ±25%, nmse = 0.11.

Fig. 11
Fig. 11

Effect of gain variation of the I element in the ION. This effect is nonuniform with some spatial correlation. High frequency gain variation with (a) T = 1, sc = 3, p = ±10%, nmse = 0.04, (b) T = 1, sc = 3, p = ±25%, nmse = 0.11; low frequency variation with (c) T = 25, sc = 9, p = ±10%, nmse = 0.09, (d) T = 25, sc = 9, p = 25%, nmse = 0.18.

Fig. 12
Fig. 12

Experimental setup of the ION test circuit.

Fig. 13
Fig. 13

LCLV characteristics of the I and N element in the test circuit for V = 5.0 volts, f = 1.5 kHz. The vertical axis is the intensity measured at the LCLV output, when in the system of Fig. 12 with a laser power of 200 mW. The I element is fairly linear within 50% of its operation range. The self-feedback of the N element (b) is necessary to satisfy the ION requirement for this particular device.

Fig. 14
Fig. 14

Experimental results of a single neuron showing N element output (left side of each figure) and I element output (center pixel of each figure). The rightmost pixel is used for alignment. The following data are normalized: (a) Iexc = 1, Iinh = 0; (b) Iexc = 1, Iinh = 1; (c) Iexc = 0, Iinh = 0; (d) Iexc = 0, Iinh = 1; gray level case, with (e) Iexc = 1, Iinh = 0.5; and (f) Iexc = 0.5, Iinh = 0.5.

Fig. 15
Fig. 15

Results of binary subtraction with (a) input patterns, N inputs (left) and I inputs (right), at LCLV input; and (b) outputs, N element outputs, the subtraction result (left), and I element outputs (right). The ideal result is the residual leg of R (top right) and full T (bottom right) of the N element output. (c) Gray-level subtraction showing normalized N input vs I input for a constant N output. The subtraction is quite linear if we only use 50% of the maximum I input (a2).

Fig. 16
Fig. 16

Single layer feedback net using a single spatial light modulator to implement both I and N elements.

Fig. 17
Fig. 17

Conceptual diagram to implement Grossberg’s mass action type neuron based on the ION model utilizing a LCLV.

Equations (36)

Equations on this page are rendered with MathJax. Learn more.

V ^ i = ψ ( j = 1 N W i j V j ) ,
V ^ i = ψ [ j = 1 N ( W i j + W b ) V j ] = ψ [ j = 1 N W i j V j + W b + W b j = 1 N V j ] ,
V ^ i = ψ [ j = 1 N W i j ( V j + V b ) ] = ψ [ j = 1 N W i j V j + V b j = 1 N W i j ] .
V ^ i = ψ [ k W i k V k + j W i j ( 1 - V j ) ] = ψ ( k W i k V k - j W i j V j + j W i j ) ,
V ^ i = ψ [ j = 1 N ( W i j + 1 2 ) V j + j = 1 N 1 2 ( 1 - V j ) ] = ψ [ j = 1 N W i j V j + N 2 ] .
I out ( I ) = 1 - I inh
I out ( N ) = ψ [ I out ( I ) + I exc + I bias - α ] ,
I out ( N ) = ψ ( I exc - I inh ) ,
I out ( I ) = - b 1 a 1 I inh + b 1 for 0 I inh a 1 , I out ( N ) = ψ ( I in ( N ) - α ) for α I in ,
I out ( I ) = b 1 - b 1 - b 0 Δ a I I inh ,
mmse = a m a M [ y ( x ) - y ^ ( x ) ] 2 d x a m a M y 2 ( x ) d x ,
I out = ψ ( { 1 - [ I inh + N b ( I ) + N r ( I ) + N w ( I ) ] } + { I exc + N b ( N ) + N r ( N ) + N w ( N ) } + [ α - 1 ] - α ) .
I out = ψ { [ 1 + N g ( I ) ] [ 1 + N d ( I ) - I inh ] + I exc + N dh ( N ) + ( α - 1 + N bp ) - α } .
I out = ψ { ( 1 + N 1 * ) [ 1 - I inh + N I + ] + I exc + N N + - 1 } ,
1 2 ( 1 + V ^ i ) = ψ [ j = 1 N ( 1 - W i j ) 2 ( 1 - V j ) 2 + j = 1 N ( 1 + W i j ) 2 ( 1 + V j ) 2 ] = ψ [ j = 1 N W i j V j 2 + N 2 ] .
V ^ i = Φ [ j = 1 N W i j V j ] ,
V i = ψ { j A pre a i j V j + k A post a i k V k - d i j A pre c ˜ i j V j - b i k A post c ˜ i k V k - l A l e ˜ i l V l } ,
x ˙ i = - A x i + ( B - x i ) I exc - x i I inh , I exc = j = 1 n 1 ψ ( x j ) C i j + I i , I inh = j = 1 n 2 ϕ ( x j ) D i j + J i ,
x i ( k + 1 ) = x i ( k ) [ 1 - ( A + I exc + I inh ) ] + B I exc .
x i = B I exc A + I exc + I inh .
x ˙ i = - A x i + ( B i - x i ) [ j n 1 ψ ( x j ) C i j + I i ] - x i [ j n 2 ϕ ( y j ) D i j + J i ] ; y ˙ i = - E y i + j n 3 x j F i j ,
0 I inh a 1 ,
0 I exc a 1 ,
α a 1 ( heterogeneous case ) ,
α a 1 + a 2 ( homogeneous case ) .
a 2 I exc + ( α - a 1 ) α .
α a 1 + θ ( heterogeneous case )
α a 1 + a 2 + θ ( homogeneous case )
I j ( out ) = I r + Δ I s V j ,
I i ( in ) = I r N out j = 1 N in w i j + Δ I s N out j = 1 N in w i j V j .
I j ( in ) = I r N out N in w i j ¯ + Δ I s N out w i j ¯ .
N in Δ I s I r ,
I i ( in ) ( max ) = 1 N out w i j ¯ N in ( exc ) + Δ I s N out w i j ¯ N in ( exc ) ,
a 1 + γ s Δ I s N out N in ( exc ) w i j ¯ + ( α - a 1 ) Δ a N + α ,
N out N in ( exc ) γ s w i j ¯ Δ I s Δ a N .
Δ I s N in ( inh ) N out a 1 , Δ I s N in ( exc ) N out a 1 ,

Metrics