Abstract

An all-optical implementation of a feed-forward artificial neural network is presented that uses self-lensing materials in which the index of refraction is irradiance dependent. Many of these types of material have ultrafast response times and permit both weighted connections and nonlinear neuron processing to be implemented with only thin material layers separated by free space. Both neuron processing and weighted interconnections emerge directly from the physical optics of the device. One creates virtual neurons and their connections simply by applying patterns of irradiance to thin layers of the nonlinear media. This is a result of a variation of the refractive-index profile of the self-lensing nonlinear media in response to the applied irradiance. An optical-backpropagation training method for this network is presented. The optical backpropagation is a training method that can be implemented potentially within the same optical device as the forward calculations, although several issues crucial to this possibility remain to be addressed. Such a network was numerically simulated and trained to solve many benchmark classification problems, and some of these results are presented. To demonstrate the feasibility of building such a network, we also describe experimental work in the construction of an optical network trained to perform a logic xnor function. This network, as a proof of concept, uses a relatively slow thermal nonlinear material with ~1-s response time.

© 1995 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. P. D. Wasserman, Neural Computing Theory and Practice (Van Nostrand Reinhold, New York, 1989).
  2. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.
  3. M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.
  4. H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
    [CrossRef]
  5. M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.
  6. S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.
  7. J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.
  8. D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
    [CrossRef]
  9. G. J. Dunning, Y. Owechko, B. H. Soffer, “Hybrid optoelectronic neural network using a mutually pumped phase-conjugate mirror,” Opt. Lett. 16, 928–930 (1991).
    [CrossRef] [PubMed]
  10. J. Jang, S. Shin, S. Lee, “Programmable quadratic associative memory using holographic lenslet arrays,” Opt. Lett. 14, 838–840 (1989).
    [CrossRef] [PubMed]
  11. T. S. Yu, X. Yang, S. Yin, D. Gregory, “Mirror-array optical interconnected neural network,” Opt. Lett. 16, 1602–1604 (1991).
    [CrossRef] [PubMed]
  12. D. Casasent, E. Botha, “Optical correlator production system neural net,” Appl. Opt. 31, 1030–1040 (1992).
    [CrossRef] [PubMed]
  13. K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
    [CrossRef] [PubMed]
  14. Using NWorks, An Extended Tutorial for NeuralWorks Professional II/Plus and NeuralWorks Explorer (NeuralWare, Inc., Pittsburgh, Pa., 1991), p. UN-18.
  15. Brainmaker Users Guide and Reference Manual (California Scientific Software, Grass Valley, Calif., 1990), p. 3-3.
  16. H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.
  17. H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
    [CrossRef] [PubMed]
  18. D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
    [CrossRef]

1992 (1)

1991 (2)

1990 (1)

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

1989 (2)

J. Jang, S. Shin, S. Lee, “Programmable quadratic associative memory using holographic lenslet arrays,” Opt. Lett. 14, 838–840 (1989).
[CrossRef] [PubMed]

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

1988 (1)

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

1987 (1)

Anderson, D. R.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Au, W. W. L.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
[CrossRef] [PubMed]

Benson, R.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Botha, E.

Casasent, D.

Catro, H.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Denker, J. S.

S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.

Dunning, G. J.

Emerling, M.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Graf, H. P.

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

Gregory, D.

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

Holler, M.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Hoofen, D. E.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Howard, R. E.

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

Hubbard, W. E.

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

Jackel, L. D.

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

Jang, J.

Kaplan, A. E.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Lee, S.

Mackie, S.

S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.

Mead, C.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Moore, P. W. B.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
[CrossRef] [PubMed]

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

Nachtigall, P. E.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
[CrossRef] [PubMed]

Owechko, Y.

Penner, R. H.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
[CrossRef] [PubMed]

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

Psaltis, D.

Roitblat, H. L.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
[CrossRef] [PubMed]

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

Sage, J. P.

J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.

Schwartz, D. B.

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

Shin, S.

Sivilotti, M.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Soffer, B. H.

Swartzlander, G. A.

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Tam, S.

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

Wagner, K.

Wasserman, P. D.

P. D. Wasserman, Neural Computing Theory and Practice (Van Nostrand Reinhold, New York, 1989).

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

Withers, R. S.

J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.

Yang, X.

Yin, S.

Yu, T. S.

Appl. Opt. (2)

IEEE Computer (1)

H. P. Graf, L. D. Jackel, W. E. Hubbard, “VLSI implementation of a neural network model,” IEEE Computer (March1988), 41–49.
[CrossRef]

IEEE J. Solid-State Circuits (1)

D. B. Schwartz, R. E. Howard, W. E. Hubbard, “A programmable analog neural network chip,” IEEE J. Solid-State Circuits 24, 313–319 (1989).
[CrossRef]

Opt. Lett. (1)

D. R. Anderson, D. E. Hoofen, G. A. Swartzlander, A. E. Kaplan, “Direct measurement of the transverse velocity of dark spatial solitons,” Opt. Lett. 15, 783–785 (1990).
[CrossRef]

Opt. Lett. (3)

Other (10)

P. D. Wasserman, Neural Computing Theory and Practice (Van Nostrand Reinhold, New York, 1989).

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. Rumelhart, J. McClelland, eds. (MIT Press, Cambridge, Mass., 1986), Vol. 1.

M. Sivilotti, M. Emerling, C. Mead, “A novel associative memory implemented using collective computation,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 11–21.

Using NWorks, An Extended Tutorial for NeuralWorks Professional II/Plus and NeuralWorks Explorer (NeuralWare, Inc., Pittsburgh, Pa., 1991), p. UN-18.

Brainmaker Users Guide and Reference Manual (California Scientific Software, Grass Valley, Calif., 1990), p. 3-3.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, “Natural dolphin echo recognition using an integrator gateway network,” in Advances in Neural Processing Systems (Morgan Kaufmann, San Mateo, Calif., 1991), Vol. 3, pp. 273–281.

H. L. Roitblat, P. W. B. Moore, P. E. Nachtigall, R. H. Penner, W. W. L. Au, “Dolphin echolocation: identification of returning echoes using a counterpropagation network,” Proceedings of IEEE First International Joint Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1989), pp. I-295–I-300; W. W. L. Au, D. W. Martin, “Insights into dolphin sonar discrimination capabilities from human listening experiments,” J. Acoust. Soc. Am. 86, 1662–1669 (1989).
[CrossRef] [PubMed]

M. Holler, S. Tam, H. Catro, R. Benson, “An electrically trainable artificial neural network (ETANN) with 10240 ‘floating gate’ synapses,” in Proceedings of IEEE International Annual Conference on Neural Networks (Institute of Electrical and Electronics Engineering, New York, 1988), Vol. 2, pp. 191–196.

S. Mackie, J. S. Denker, “A digital implementation of a best match classifier,” in IEEE 1988 Custom Integrated Circuits Conference (Institute of Electrical and Electronics Engineering, New York, 1988), pp. 10.4.1–10.4.4.

J. P. Sage, R. S. Withers, “Analog nonvolatile memory for neural network implementations,” in Artificial Neural Networks: Electronic Implementations, N. Morgan, ed. (IEEE Computer Society, Los Alamitos, Calif., 1990), pp. 22–33.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1

Network architecture. Each of the slabs shown is a self-lensing media; the initial slab is the input layer I(x, y), and succeeding slabs are weighting layers [i(x, y)] Each nonlinear layer serves as a plane continuum of neurons. The free-space regions between the slabs serve as connection layers. The output field is O(x, y).

Fig. 2
Fig. 2

Optical backpropagation training. Ei(x, y) are the forward-propagation signals through, and Wi(x, y) are the weighting light distributions at the nonlinear layer i. O(x, y) is the output, and λn+1 is the error signal at the output. The nonlinear layers are separated by ΔLi, and the thickness of layer i is ΔNLi. Also shown are the backpropagated error signals [δi(x, y)] for updating the weights on the nonlinear layers.

Fig. 3
Fig. 3

Confidence ratio versus test data pair for the iris classification. The confidence ratio is defined as the ratio of the correct classification to the total output of the three classification units. The network (a) was trained on 50 training pairs, then (b) tested on an additional 50.

Fig. 4
Fig. 4

Predicted and correct stock price over a period of 20 days. The network was trained on the 20 pairs of training shown here.

Fig. 5
Fig. 5

Same as Fig. 3 but for the dolphin sonar discrimination. The network (a) was trained on 13 training pairs, then (b) tested on these additional 14.

Fig. 6
Fig. 6

Masks used in the experimental setup for spatial modulation of the irradiance profiles to (a) the input layer and (b) the weighting layer. The weighting mask was found from backpropagation training results via computer simulations of the experimental setup.

Tables (1)

Tables Icon

Table 1 Experimental and Numerical Resultsa for an xnor Logic Gate

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

x i = j = 1 , N w i j f j ( x j ) ,
F i ( α ) = E i ( α ) exp ( j k 0 Δ N L i × { n 2 [ | Γ i ( α ) | 2 + | E i ( α ) | 2 ] + n 0 } ) ,
E i + 1 ( β ) = j C i π Ω i F i ( α ) exp [ j C i ( β α ) 2 ] d α ,
C i = k 0 2 Δ L i
H o = 1 2 γ 2 [ γ Ω o O ( β ) O * ( β ) d β D ] 2 ,
λ n + 1 ( β ) = 1 γ O * ( β ) [ D γ Ω o O ( β ) O * ( β ) d β ] .
δ n ( α ) = j C n π Ω o λ n + 1 ( β ) exp [ j C n ( β α ) 2 ] d β ,
λ i ( β ) = δ i ( β ) exp ( j k 0 Δ N L i { n 2 [ | W i ( β ) | 2 + | E i ( β ) | 2 ] + n 0 } ) + k 0 Δ N L i n 2 E i * ( β ) × 2 Im [ δ i ( β ) E i ( β ) exp ( j k 0 Δ N L i { n 2 [ | W i ( β ) | 2 + | E i ( β ) | 2 ] + n 0 } ) ] ,
δ i 1 ( α ) = j C i 1 π Ω i 1 λ i ( β ) exp [ j C i 1 ( β α ) 2 ] d β .
W i new ( α ) = W i old ( α ) + η k 0 Δ N L i n 2 | W i old ( α ) | 2 × Im ( E i ( α ) δ i ( α ) exp { j k 0 Δ N L i n i × [ | W i old ( α ) | 2 + | E i ( α ) | 2 ] } ) ,
λ i ( β ) = δ i ( β ) exp ( j k 0 Δ N L i { n 2 [ | W i ( β ) | 2 + | E i ( β ) | 2 ] + n 0 } ) ,
δ i 1 ( α ) = j C i 1 π Ω i 1 λ i 1 ( β ) exp [ j C i 1 ( β α ) 2 ] d β ,
W i new ( α ) = W i old ( α ) + η k 0 Δ N L i n 2 × | W i old ( α ) | 2 Im [ E i ( α ) λ i 1 ( α ) ] ,

Metrics