Abstract

An all-optical implementation of a feed-forward artificial neural network is presented that uses self-lensing materials in which the index of refraction is irradiance dependent. Many of these types of material have ultrafast response times and permit both weighted connections and nonlinear neuron processing to be implemented with only thin material layers separated by free space. Both neuron processing and weighted interconnections emerge directly from the physical optics of the device. One creates virtual neurons and their connections simply by applying patterns of irradiance to thin layers of the nonlinear media. This is a result of a variation of the refractive-index profile of the self-lensing nonlinear media in response to the applied irradiance. An optical-backpropagation training method for this network is presented. The optical backpropagation is a training method that can be implemented potentially within the same optical device as the forward calculations, although several issues crucial to this possibility remain to be addressed. Such a network was numerically simulated and trained to solve many benchmark classification problems, and some of these results are presented. To demonstrate the feasibility of building such a network, we also describe experimental work in the construction of an optical network trained to perform a logic xnor function. This network, as a proof of concept, uses a relatively slow thermal nonlinear material with ~1-s response time.

© 1995 Optical Society of America

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. P. D. Wasserman, Neural Computing Theory and Practice (Van Nostrand Reinhold, New York, 1989).
  2. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.
  3. M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.
  4. H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
    [CrossRef]
  5. M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.
  6. S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.
  7. J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.
  8. D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
    [CrossRef]
  9. G. J. Dunning, Y. Owechko, B. H. Soffer, “Hybrid optoelectronic neural network using a mutually pumped phase-conjugate mirror,” Opt. Lett. 16, 928–930 (1991).
    [CrossRef] [PubMed]
  10. J. Jang, S. Shin, S. Lee, “Programmable quadratic associative memory using holographic lenslet arrays,” Opt. Lett. 14, 838–840 (1989).
    [CrossRef] [PubMed]
  11. T. S. Yu, X. Yang, S. Yin, D. Gregory, “Mirror-array optical interconnected neural network,” Opt. Lett. 16, 1602–1604 (1991).
    [CrossRef] [PubMed]
  12. D. Casasent, E. Botha, “Optical correlator production system neural net,” Appl. Opt. 31, 1030–1040 (1992).
    [CrossRef] [PubMed]
  13. K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
    [CrossRef] [PubMed]
  14. Using NWorks, An Extended Tutorial for NeuralWorks Professional II/Plus and NeuralWorks Explorer (NeuralWare, Inc., Pittsburgh, Pa., 1991), p. UN-18.
  15. Brainmaker Users Guide and Reference Manual (California Scientific Software, Grass Valley, Calif., 1990), p. 3-3.
  16. H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.
  17. H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
  18. D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
    [CrossRef]

1992 (1)

1991 (2)

1990 (1)

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

1989 (2)

J. Jang, S. Shin, S. Lee, “Programmable quadratic associative memory using holographic lenslet arrays,” Opt. Lett. 14, 838–840 (1989).
[CrossRef] [PubMed]

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

1988 (1)

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

1987 (1)

Anderson, D. R.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Au, W. W. L.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).

Benson, R.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Botha, E.

Casasent, D.

Catro, H.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Denker, J. S.

S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.

Dunning, G. J.

Emerling, M.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Graf, H. P.

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

Gregory, D.

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

Holler, M.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Hoofen, D. E.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Howard, R. E.

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

Hubbard, W. E.

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

Jackel, L. D.

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

Jang, J.

Kaplan, A. E.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Lee, S.

Mackie, S.

S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.

Mead, C.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Moore, P. W. B.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

Nachtigall, P. E.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).

Owechko, Y.

Penner, R. H.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

Psaltis, D.

Roitblat, H. L.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

Sage, J. P.

J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.

Schwartz, D. B.

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

Shin, S.

Sivilotti, M.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Soffer, B. H.

Swartzlander, G. A.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Tam, S.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Wagner, K.

Wasserman, P. D.

P. D. Wasserman, Neural Computing Theory and Practice (Van Nostrand Reinhold, New York, 1989).

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

Withers, R. S.

J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.

Yang, X.

Yin, S.

Yu, T. S.

Appl. Opt. (2)

IEEE Computer (1)

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

IEEE J. Solid-State Circuits (1)

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

Opt. Lett. (1)

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Opt. Lett. (3)

Other (10)

P. D. Wasserman, Neural Computing Theory and Practice (Van Nostrand Reinhold, New York, 1989).

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Using NWorks, An Extended Tutorial for NeuralWorks Professional II/Plus and NeuralWorks Explorer (NeuralWare, Inc., Pittsburgh, Pa., 1991), p. UN-18.

Brainmaker Users Guide and Reference Manual (California Scientific Software, Grass Valley, Calif., 1990), p. 3-3.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.

J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Metrics