Abstract

We analytically determine that the backward-error-propagation learning algorithm has a well-defined region of convergence in neural learning-parameter space for two classes of photorefractive-based optical neural-network architectures. The first class uses electric-field amplitude encoding of signals and weights in a fully coherent system, whereas the second class uses intensity encoding of signals and weights in an incoherent/coherent system. Under typical assumptions on the grating formation in photorefractive materials used in adaptive optical interconnections, we compute weight updates for both classes of architectures. Using these weight updates, we derive a set of conditions that are sufficient for such a network to operate within the region of convergence. The results are verified empirically by simulations of the XOR sample problem. The computed weight updates for both classes of architectures contain two neural learning parameters: a learning-rate coefficient and a weight-decay coefficient. We show that these learning parameters are directly related to two important design parameters: system gain and exposure energy. The system gain determines the ratio of the learning-rate parameter to decay-rate parameter, and the exposure energy determines the size of the decay-rate parameter. We conclude that convergence is guaranteed (assuming no spurious local minima in the error function) by using a sufficiently high gain and a sufficiently low exposure energy per weight update.

© 1996 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. P. J. van Heerden, “Theory of optical information storage in solids,” Appl. Opt. 2, 393–400 (1963).
  2. F. H. Mok, “Angle-multiplexed storage of 5000 holograms in lithium niobate,” Opt. Lett. 18, 915–917 (1993).
  3. K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
  4. D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27,1752–1759 (1988).
  5. P. Yeh, A. E. T. Chiou, J. Hong, “Optical interconnection using photorefractive dynamic holograms,” Appl. Opt. 27, 2093–2095 (1988).
  6. H. Yoshinaga, K. Kitayama, T. Hara, “All-optical error-signal generation for backpropagation learning in optical multilayer neural networks,” Opt. Lett. 14, 262–264 (1989).
  7. E. G. Paek, J. R. Wullert, J. S. Patel, “Holographic implementation of a learning machine based on a multicat-egory perceptron algorithm,” Opt. Lett. 14, 1303–1305 (1989).
  8. D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).
  9. A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).
  10. J. H. Hong, S. Campbell, P. Yeh, “Optical pattern classifier with Perceptron learning,” Appl. Opt. 29, 3019–3025 (1990).
  11. C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).
  12. Y. Owechko, B. H. Soffer, “Optical interconnection method for neural networks using self-pumped phase-conjugate mir-rors,” Opt. Lett. 16, 675–677 (1991).
  13. G. J. Dunning, Y. Owechko, B. H. Soffer, “Hybrid optoelectronic neural networks using a mutually pumped phase-conjugate mirror,” Opt. Lett. 16, 928–930 (1991).
  14. Y. Qiao, D. Psaltis, “Learning algorithms for optical multilayer neural networks,” in Proceedings of the Third International Joint Conference on Neural Networks (IEEE, New York, 1991), Vol. 1, pp. 457–462.
  15. Y. Qiao, D. Psaltis, “Local learning algorithm for optical neural networks,” Appl. Opt. 31, 3285–3288 (1992).
  16. B. K. Jenkins, A. R. Tanguay, Photonic implementations of neural networks, in Neural Networks for Signal Processing, Bart Kosko, ed. (Prentice-Hall, Englewood Cliffs, N.J., 1992), pp. 287–379.
  17. Y. Owechko, “Cascaded-grating holography for artificial neural networks,” Appl. Opt. 32, 1380–1398 (1993).
  18. J. Hong, “Applications of photorefractive crystals for optical neural networks,” Opt. Quantum Electron. 25, 551–568 (1993).
  19. K. Y. Hsu, S. H. Lin, P. Yeh, “Conditional convergence of photorefractive perceptron learning,” Opt. Lett. 18, 2135–2137 (1993).
  20. K. Wagner, T. M. Slagle, “Optical competitive learning with VLSI/liquid-crystal winner-take-all modulators,” Appl. Opt. 32, 1408–1435 (1993).
  21. C.J. Cheng, P. C. Yeh, K. Y. Hsu, “Generalized perceptron learning rule and its implications for photorefractive neural networks,” J. Opt. Soc. Am. B 11, 1619–1624 (1994).
  22. T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).
  23. Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).
  24. A. Goldstein, G. C. Petrisor, B. K. Jenkins, “Gain and exposure schedule to compensate for photorefractive neural-network weight decay,” Opt. Lett. 20, 611–613 (1995).
  25. D. C. Plaut, S. J. Nowlan, G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126 (Department of Computer Science, Carnegie-Mellon, Pittsburgh, Pa., 1986), p.8.
  26. P. Asthana, G. P. Nordin, A. R. Tanguay, B. K. Jenkins, “Analysis of weighted fan-out fan-in volume holographic optical interconnections,” Appl. Opt. 32, 1441–1469 (1993).
  27. N. V. Kukhtarev, “Kinetics of hologram recording and erasurein electrooptic crystals,” Sov. Tech. Phys. Lett. 2, 438–448 (1976).
  28. D. Brady, D. Psaltis, “Information capacity of 3-D holo-graphic data storage,” Opt. Quantum Electron. 25, 597–610 (1993).
  29. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learninginternal representations by error propagation,” in Parallel Dìstrìbuted Processing, D. Rumelhart, J. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, Chap. 8, pp. 318–328.
  30. H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).
  31. D. Brady, D. Psaltis, “Control of volume holograms,” J. Opt. Soc. Am. A 9, 1167–1182 (1992).
  32. C. Slinger, “Analysis ofthe N-to-N volume-holographic neural interconnect,” J. Opt. Soc. Am. A 8, 1074–1081 (1991).
  33. B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).
  34. G. P. Nordin, “Analysis of volume diffraction phenomena forphotonic neural network implementations and stratified volume holographic optical elements,” Ph.D. dissertationUniversity of Southern California, Los Angeles, Calif., 1992), pp. 99–130.
  35. G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.
  36. D. Psaltis, X.-G. Gu, D. Brady, “Fractal sampling grids for holographic interconnections,” in Optical Computing ‘88, P. Chavel, J. W. Goodman, G. Roblin, eds., Proc. Soc. Photo-Opt. Instrum. Eng.963, 468–474 (1988).

1995

1994

C.J. Cheng, P. C. Yeh, K. Y. Hsu, “Generalized perceptron learning rule and its implications for photorefractive neural networks,” J. Opt. Soc. Am. B 11, 1619–1624 (1994).

T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).

1993

1992

1991

1990

D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).

A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).

J. H. Hong, S. Campbell, P. Yeh, “Optical pattern classifier with Perceptron learning,” Appl. Opt. 29, 3019–3025 (1990).

C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).

1989

H. Yoshinaga, K. Kitayama, T. Hara, “All-optical error-signal generation for backpropagation learning in optical multilayer neural networks,” Opt. Lett. 14, 262–264 (1989).

E. G. Paek, J. R. Wullert, J. S. Patel, “Holographic implementation of a learning machine based on a multicat-egory perceptron algorithm,” Opt. Lett. 14, 1303–1305 (1989).

H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).

1988

1987

1976

N. V. Kukhtarev, “Kinetics of hologram recording and erasurein electrooptic crystals,” Sov. Tech. Phys. Lett. 2, 438–448 (1976).

1963

Asthana, P.

P. Asthana, G. P. Nordin, A. R. Tanguay, B. K. Jenkins, “Analysis of weighted fan-out fan-in volume holographic optical interconnections,” Appl. Opt. 32, 1441–1469 (1993).

B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).

Brady, D.

D. Brady, D. Psaltis, “Information capacity of 3-D holo-graphic data storage,” Opt. Quantum Electron. 25, 597–610 (1993).

D. Brady, D. Psaltis, “Control of volume holograms,” J. Opt. Soc. Am. A 9, 1167–1182 (1992).

D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).

D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27,1752–1759 (1988).

D. Psaltis, X.-G. Gu, D. Brady, “Fractal sampling grids for holographic interconnections,” in Optical Computing ‘88, P. Chavel, J. W. Goodman, G. Roblin, eds., Proc. Soc. Photo-Opt. Instrum. Eng.963, 468–474 (1988).

Campbell, S.

Cheng, C.J.

Chiou, A. E. T.

Dunning, G. J.

Galstyan, T.

T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).

Goldstein, A.

Gu, C.

Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).

Gu, X. G.

H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).

Gu, X.-G.

D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).

D. Psaltis, X.-G. Gu, D. Brady, “Fractal sampling grids for holographic interconnections,” in Optical Computing ‘88, P. Chavel, J. W. Goodman, G. Roblin, eds., Proc. Soc. Photo-Opt. Instrum. Eng.963, 468–474 (1988).

Habiby, S. F.

A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).

Hara, T.

H. Yoshinaga, K. Kitayama, T. Hara, “All-optical error-signal generation for backpropagation learning in optical multilayer neural networks,” Opt. Lett. 14, 262–264 (1989).

Hartman, E.

C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).

Hinton, G. E.

D. C. Plaut, S. J. Nowlan, G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126 (Department of Computer Science, Carnegie-Mellon, Pittsburgh, Pa., 1986), p.8.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learninginternal representations by error propagation,” in Parallel Dìstrìbuted Processing, D. Rumelhart, J. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, Chap. 8, pp. 318–328.

Hong, J.

J. Hong, “Applications of photorefractive crystals for optical neural networks,” Opt. Quantum Electron. 25, 551–568 (1993).

Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).

P. Yeh, A. E. T. Chiou, J. Hong, “Optical interconnection using photorefractive dynamic holograms,” Appl. Opt. 27, 2093–2095 (1988).

Hong, J. H.

Hsu, K. Y.

Hubbard, W. M.

A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).

Jenkins, B. K.

A. Goldstein, G. C. Petrisor, B. K. Jenkins, “Gain and exposure schedule to compensate for photorefractive neural-network weight decay,” Opt. Lett. 20, 611–613 (1995).

P. Asthana, G. P. Nordin, A. R. Tanguay, B. K. Jenkins, “Analysis of weighted fan-out fan-in volume holographic optical interconnections,” Appl. Opt. 32, 1441–1469 (1993).

B. K. Jenkins, A. R. Tanguay, Photonic implementations of neural networks, in Neural Networks for Signal Processing, Bart Kosko, ed. (Prentice-Hall, Englewood Cliffs, N.J., 1992), pp. 287–379.

B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).

G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.

Keeler, J. D.

C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).

Kitayama, K.

H. Yoshinaga, K. Kitayama, T. Hara, “All-optical error-signal generation for backpropagation learning in optical multilayer neural networks,” Opt. Lett. 14, 262–264 (1989).

Kukhtarev, N. V.

N. V. Kukhtarev, “Kinetics of hologram recording and erasurein electrooptic crystals,” Sov. Tech. Phys. Lett. 2, 438–448 (1976).

Lee, H.

H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).

Lin, S.

D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).

Lin, S. H.

Marrakchi, A.

A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).

Mok, F. H.

Neurgaonkar, R. R.

Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).

Nordin, G. P.

P. Asthana, G. P. Nordin, A. R. Tanguay, B. K. Jenkins, “Analysis of weighted fan-out fan-in volume holographic optical interconnections,” Appl. Opt. 32, 1441–1469 (1993).

G. P. Nordin, “Analysis of volume diffraction phenomena forphotonic neural network implementations and stratified volume holographic optical elements,” Ph.D. dissertationUniversity of Southern California, Los Angeles, Calif., 1992), pp. 99–130.

G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.

Nowlan, S. J.

D. C. Plaut, S. J. Nowlan, G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126 (Department of Computer Science, Carnegie-Mellon, Pittsburgh, Pa., 1986), p.8.

Owechko, Y.

Paek, E. G.

Patel, J. S.

A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).

E. G. Paek, J. R. Wullert, J. S. Patel, “Holographic implementation of a learning machine based on a multicat-egory perceptron algorithm,” Opt. Lett. 14, 1303–1305 (1989).

Pauliat, G.

T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).

Peterson, C.

C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).

Petrisor, G. C.

A. Goldstein, G. C. Petrisor, B. K. Jenkins, “Gain and exposure schedule to compensate for photorefractive neural-network weight decay,” Opt. Lett. 20, 611–613 (1995).

G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.

B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).

Piazzolla, S.

B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).

G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.

Plaut, D. C.

D. C. Plaut, S. J. Nowlan, G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126 (Department of Computer Science, Carnegie-Mellon, Pittsburgh, Pa., 1986), p.8.

Psaltis, D.

D. Brady, D. Psaltis, “Information capacity of 3-D holo-graphic data storage,” Opt. Quantum Electron. 25, 597–610 (1993).

D. Brady, D. Psaltis, “Control of volume holograms,” J. Opt. Soc. Am. A 9, 1167–1182 (1992).

Y. Qiao, D. Psaltis, “Local learning algorithm for optical neural networks,” Appl. Opt. 31, 3285–3288 (1992).

Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).

D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).

H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).

D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27,1752–1759 (1988).

K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).

Y. Qiao, D. Psaltis, “Learning algorithms for optical multilayer neural networks,” in Proceedings of the Third International Joint Conference on Neural Networks (IEEE, New York, 1991), Vol. 1, pp. 457–462.

D. Psaltis, X.-G. Gu, D. Brady, “Fractal sampling grids for holographic interconnections,” in Optical Computing ‘88, P. Chavel, J. W. Goodman, G. Roblin, eds., Proc. Soc. Photo-Opt. Instrum. Eng.963, 468–474 (1988).

Qiao, Y.

Y. Qiao, D. Psaltis, “Local learning algorithm for optical neural networks,” Appl. Opt. 31, 3285–3288 (1992).

Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).

Y. Qiao, D. Psaltis, “Learning algorithms for optical multilayer neural networks,” in Proceedings of the Third International Joint Conference on Neural Networks (IEEE, New York, 1991), Vol. 1, pp. 457–462.

Redfield, S.

C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).

Roosen, G.

T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learninginternal representations by error propagation,” in Parallel Dìstrìbuted Processing, D. Rumelhart, J. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, Chap. 8, pp. 318–328.

Slagle, T. M.

Slinger, C.

Soffer, B. H.

Tanguay, A. R.

P. Asthana, G. P. Nordin, A. R. Tanguay, B. K. Jenkins, “Analysis of weighted fan-out fan-in volume holographic optical interconnections,” Appl. Opt. 32, 1441–1469 (1993).

B. K. Jenkins, A. R. Tanguay, Photonic implementations of neural networks, in Neural Networks for Signal Processing, Bart Kosko, ed. (Prentice-Hall, Englewood Cliffs, N.J., 1992), pp. 287–379.

B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).

G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.

van Heerden, P. J.

Villing, A.

T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).

Wagner, K.

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learninginternal representations by error propagation,” in Parallel Dìstrìbuted Processing, D. Rumelhart, J. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, Chap. 8, pp. 318–328.

Wullert, J. R.

Yeh, P.

Yeh, P. C.

Yoshinaga, H.

H. Yoshinaga, K. Kitayama, T. Hara, “All-optical error-signal generation for backpropagation learning in optical multilayer neural networks,” Opt. Lett. 14, 262–264 (1989).

Appl. Opt.

J. Appl. Phys.

Y. Qiao, D. Psaltis, C. Gu, J. Hong, P. Yeh, R. R. Neurgaonkar, “Phase-locked sustainment of photorefractive holograms using phase conjugation,” J. Appl. Phys. 70, 4646–4648 (1991).

H. Lee, X. G. Gu, D. Psaltis, “Volume holographic interconnections with maximal capacity and minimal crosstalk,” J. Appl. Phys. 65, 2191–2195 (1989).

J. Opt. Soc. Am. A

J. Opt. Soc. Am. B

Nature (London)

D. Psaltis, D. Brady, X.-G. Gu, S. Lin, “Holography inartificial neural networks,” Nature (London) 343, 325–343 (1990).

Opt. Commun.

T. Galstyan, G. Pauliat, A. Villing, G. Roosen, “Adaptive photorefractive neurons for self-organizing networks,” Opt. Commun. 109, 35–42 (1994).

Opt. Eng.

A. Marrakchi, W. M. Hubbard, S. F. Habiby, J. S. Patel, “Dynamic holographic interconnects with analog weights in photorefractive crystals,” Opt. Eng. 29, 215–224 (1990).

C. Peterson, S. Redfield, J. D. Keeler, E. Hartman, “Optoelectronic implementation of multilayer neural net-works in a single photorefractive crystal,” Opt. Eng. 29, 359–368 (1990).

Opt. Lett.

Opt. Quantum Electron.

D. Brady, D. Psaltis, “Information capacity of 3-D holo-graphic data storage,” Opt. Quantum Electron. 25, 597–610 (1993).

J. Hong, “Applications of photorefractive crystals for optical neural networks,” Opt. Quantum Electron. 25, 551–568 (1993).

Sov. Tech. Phys. Lett.

N. V. Kukhtarev, “Kinetics of hologram recording and erasurein electrooptic crystals,” Sov. Tech. Phys. Lett. 2, 438–448 (1976).

Other

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learninginternal representations by error propagation,” in Parallel Dìstrìbuted Processing, D. Rumelhart, J. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, Chap. 8, pp. 318–328.

D. C. Plaut, S. J. Nowlan, G. E. Hinton, “Experiments on learning by back propagation,” Tech. Rep. CMU-CS-86-126 (Department of Computer Science, Carnegie-Mellon, Pittsburgh, Pa., 1986), p.8.

B. K. Jenkins, G. C. Petrisor, S. Piazzolla, P. Asthana, A. R. Tanguay, “Photonic architecture for neural nets using incoherent coherent holographic interconnections,” in Optical Computing ‘90, J. Tsujinunchi, Y. Ichioka, S. Ishihara, eds., Proc. Soc. Photo-Opt. Instrum. Eng.1359, 317–318 (1990).

G. P. Nordin, “Analysis of volume diffraction phenomena forphotonic neural network implementations and stratified volume holographic optical elements,” Ph.D. dissertationUniversity of Southern California, Los Angeles, Calif., 1992), pp. 99–130.

G. C. Petrisor, S. Piazzolla, G. P. Nordin, B. K. Jenkins, A. R. Tanguay, “Volume holographic interconnections and copying architectures based on incoherent coherent source arrays,” in Fourth International Conference on Holographs Systems, Components, and Apptications, IEEE Conf. Pub. (Institute of Electrical Engineering, London, September1993), pp. 21–26.

D. Psaltis, X.-G. Gu, D. Brady, “Fractal sampling grids for holographic interconnections,” in Optical Computing ‘88, P. Chavel, J. W. Goodman, G. Roblin, eds., Proc. Soc. Photo-Opt. Instrum. Eng.963, 468–474 (1988).

B. K. Jenkins, A. R. Tanguay, Photonic implementations of neural networks, in Neural Networks for Signal Processing, Bart Kosko, ed. (Prentice-Hall, Englewood Cliffs, N.J., 1992), pp. 287–379.

Y. Qiao, D. Psaltis, “Learning algorithms for optical multilayer neural networks,” in Proceedings of the Third International Joint Conference on Neural Networks (IEEE, New York, 1991), Vol. 1, pp. 457–462.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1

Two-layer feedforward neural network. The input layer (layer 0) is used to input patterns to the network. The two computational layers, the hidden layer (layer 1) and the output layer (layer 2), compute outputs given by y i ( l ) = f [ j W i j ( l ) y j ( l 1 ) ] ,in which W i j ( l ) is the i jth component of the total weight matrix, W (l), for layer l.

Fig. 2
Fig. 2

Single-coherent-source architecture. A single coherent source illuminates both the training SLM, SLMδ, and the input SLM, SLM y . In the forward-propagation mode, shutter S1 is closed and E δ i is not defined. In the weight-update mode, shutter S1 is opened.

Fig. 3
Fig. 3

Incoherent/coherent architecture. In this architecture, SLMy is placed in the image plane of an array of individually coherent but mutually incoherent sources and SLMδ is placed in the Fourier plane of this source array. Mutual incoherence of beams is denoted by different values of wavelength λ j . In the forward-propagation mode, shutter S1 is closed and I δ i is not defined. In the weight-update mode, shutter S1 is opened.

Fig. 4
Fig. 4

Dual-rail encoding is used to implement effective bipolar interconnection for networks with unipolar neuron unit outputs. The effective bipolar weight, W i j = W i j + W i j , is computed within the neuron unit.

Fig. 5
Fig. 5

Temporal multiplexing is used to propagate the error signals (δ i ) backward through the network. At time step t +, δ+ is output from the positive channel and δ is output from the negative channel. At time step t , δ is output from the positive channel and δ+ is output from the negative channel. The bipolar product terms W ij δ i are computed in the backward-propagating units by the differencing of d + and d .

Fig. 6
Fig. 6

Theoretically predicted shape of the SCS architecture ROC for small weight-update step size.

Fig. 7
Fig. 7

Theoretically predicted shape of the I/C architecture ROC for small weight-update step size.

Fig. 8
Fig. 8

2:3:1 (two input, three hidden, one output neuron unit) network simulated to map the ROC of the XOR sample problem.

Fig. 9
Fig. 9

Portion of the ROC that corresponds to a small weight-update step size for the SCS architecture.

Fig. 10
Fig. 10

The portion of the ROC that corresponds to a small weight-update step size for the I/C architecture.

Fig. 11
Fig. 11

Complete ROC for the XOR sample problem (SCS architecture).

Fig. 12
Fig. 12

Complete ROC for the XOR sample problem (I/C architecture).

Fig. 13
Fig. 13

Relationship between neural-space parameters (learning-rate parameter α and decay-rate parameter β) and physical-space parameters (maximum physical-space response function differential intensity gain G and exposure energy per update U).

Fig. 14
Fig. 14

(a) Using κpr = 1.0 J/cm2 and ηsat= 1.0, we have mapped the neural-space XOR ROC to physical space for the first layer of the network. The roughness in the lower boundary of the physical-space ROC is due to the irregularity of the sampling grid in physical space after being mapped from the uniform sampling grid in neural space. (b) Physical-space ROC, recomputed by simulation of the same neural network over a uniform sampling grid in physical space.

Equations (71)

Equations on this page are rendered with MathJax. Learn more.

d η ˜ i j ( t ) d t = 1 τ pr [ m i j ( η sat ) 1 / 2 η ˜ i j ( t ) ] ,
m i j = υ E y j * E δ i I total ,
I total = υ 2 E y j 2 + υ 2 E δ i 2 + I bias ,
η i j ( t ) = η ˜ i j 2 ( t ) .
d η i j ( t ) d t = 2 η ˜ i j ( t ) d η ˜ i j ( t ) d t = 2 τ pr { m i j ( η sat ) 1 / 2 [ η i j ( t ) 1 / 2 η i j ( t ) ] } ,
m i j = 2 ( I y j I δ i ) 1 / 2 I total ,
Δ η ˜ i j = Δ t τ pr [ m i j ( η sat ) 1 / 2 η ˜ i j ]
Δ η i j = 2 Δ t τ pr [ m i j ( η sat ) 1 / 2 ( η i j ) 1 / 2 η i j ]
ρ i ( l ) = j W i j ( l ) y j ( l 1 ) ,
y i ( l ) = f ( ρ i ( l ) ) ,
J 0 ( W ) = m = 1 M J 0 ( m ) ( W ) ,
J 0 ( m ) ( W ) = 1 2 i = 1 N ( L ) [ t i ( m ) y i ( L ) ( S ( m ) ) ] 2 ,
W = [ W 1 , 1 ( 1 ) , , W 1 , N ( 1 ) ( 1 ) , W 2 , 1 ( 1 ) , , W N ( 2 ) , N ( 1 ) ( 1 ) , W 1 , 1 ( 2 ) , , W N ( L ) , N ( L 1 ) ( L ) ] T .
Δ W i , j ( l ) = α J 0 ( W ) W i , j ( l ) = α m δ i ( l ) ( S ( m ) ) y j ( l 1 ) ( S ( m ) ) ,
δ i ( l ) ( S ( m ) ) = f ( ρ i ( l ) ( S ( m ) ) ) d i ( l ) ( S ( m ) ) ,
d i ( l ) ( S ( m ) ) = { k = 1 N ( l + 1 ) δ k ( l + 1 ) ( S ( m ) ) W k , i ( l + 1 ) , l < L [ t i ( m ) y i ( L ) ( S ( m ) ) ] , l = L .
f ( ρ ) = 1 1 + e 4 ρ
Δ W i , j ( l ) = α J 0 ( m ) ( W ) W i , j ( l ) = α δ i ( l ) ( S ( m ) ) y j ( l 1 ) ( S ( m ) ) .
E ρ i ( l ) = j η ˜ i j ( l ) E y j ( l 1 ) ,
y j ( l ) = E y j ( l ) E ¯ , δ i ( l ) = E δ i ( l ) E ¯ ,
E y i ( l ) = f slm ( E ρ i ( l ) ) = E ¯ 1 + exp [ γ E ρ i ( l ) ] .
G ˜ scs = E y i ( l ) E ρ i ( l ) E ρ i ( l ) = 0 = E ¯ γ 4 ,
γ = 4 G ˜ SCS E ¯ .
E y i ( l ) E ¯ = 1 1 + exp { 4 j G ˜ SCS η ˜ i j ( l ) [ E y j ( l 1 ) / E ¯ ] } ,
y i ( l ) = 1 1 + exp [ 4 j G ˜ SCS η ˜ i j ( l ) y j ( l 1 ) ]
W i j ( l ) = G ˜ SCS η ˜ i j ( l ) .
W i j ( l ) = ( G SCS ) 1 / 2 η ˜ i j ( l ) ,
I ρ i ( l ) = j η i j ( l ) I y j ( l 1 ) ,
y j ( l ) = I y j ( l ) I ¯ , δ i ( l ) = I δ i ( l ) I ¯ .
W i j ( l ) = G ic η i j ( l ) ,
W i j = W i j + W i j .
δ i ( l ) + = { δ i ( l ) , δ i ( l ) > 0 0 , δ i ( l ) 0 ,
δ i ( l ) = { 0 , δ i ( l ) 0 δ i ( l ) , δ i ( l ) < 0 .
d i ( l ) + ( S ( m ) ) = k = 1 N ( l + 1 ) δ k ( l + 1 ) + ( S ( m ) ) W k , i ( l + 1 ) + + k = 1 N ( l + 1 ) δ k ( l + 1 ) ( S ( m ) ) W k , i ( l + 1 ) ,
d i ( l ) ( S ( m ) ) = k = 1 N ( l + 1 ) δ k ( l + 1 ) ( S ( m ) ) W k , i ( l + 1 ) + + k = 1 N ( l + 1 ) δ k ( l + 1 ) + ( S ( m ) ) W k , i ( l + 1 ) .
d i ( l ) ( S ( m ) ) = d i ( l ) + ( S ( m ) ) d i ( l ) ( S ( m ) ) .
I total ( l ) = [ N ( l 1 ) + 2 N ( l ) ] I ¯
m i j ( l ) = 2 y j δ i [ N ( l 1 ) + 2 N ( l ) ] ,
m i j ( l ) = 2 [ y j δ i / N ( l 1 ) ] 1 / 2 N ( l 1 ) + 2 N ( l ) ,
Δ W i j = Δ W i j + Δ W i j .
Δ W i j = α scs y j ( δ i + δ i ) Outer Product β scs W i j Decay ,
α scs = 2 Δ t I ¯ ( η sat ) 1 / 2 ( G scs ) 1 / 2 κ pr ,
β scs = Δ t I ¯ [ N ( l 1 ) + 2 N ( l ) ] κ pr .
Δ W i j = α ic ( y j ) 1 / 2 [ ( W i j + ) 1 / 2 ( δ i + ) 1 / 2 ( W i j ) 1 / 2 ( δ i ) 1 / 2 ] Outer Product β ic W i j , Decay
α ic = 4 Δ t I ¯ ( η sat ) 1 / 2 ( G ic ) 1 / 2 κ pr [ N ( l 1 ) ] 1 / 2 ,
β ic = 2 Δ t I ¯ [ N ( l 1 ) + 2 N ( l ) ] κ pr .
W ¯ scs = α scs β scs
W ¯ ic = ( α ic β ic ) 2
d J 0 ( W ) d t = [ J 0 ( W ) ] T dW d t
J 0 ( W + Δ W ) = J 0 ( W ) + [ J 0 ( W ) ] T Δ W + H . O . T ,
Δ J 0 ( W ) = J 0 ( W + Δ W ) J 0 ( W ) = [ J 0 ( W ) ] T Δ W + H . O . T.
[ J 0 ( W ) ] T Δ W > H . O . T ,
( Δ W ) T [ J 0 ( W ) ] > 0 .
[ J 0 ( W ) ] i j ( l ) = J 0 ( W ) W i , j ( l ) = m δ i ( l ) y j ( l 1 ) .
α scs β scs > l , i j ( m δ i y j ) W i j l , i , j m ( δ i y j ) 2 .
S 0 { W 0 W 0 < } ,
S { W W S c } { W J 0 ( W ) max W S 0 [ J 0 ( W ) ] } .
α scs β scs > l , i j ( m δ i y j ) W i j l , i j m ( δ i y j ) 2 , W S ,
α ic β ic > i j ( m δ i y j ) W i j i j m [ δ i y j 3 / 2 ( W ˜ i j ) 1 / 2 ] ,
W ˜ i j = { W i j + , δ i > 0 W i j , δ i < 0 .
if  W i j > 0 , then  W i j + W i j , W i j 0 ;     if  W i j < 0 , then  W i j W i j , W i j + 0 .
α ic β ic > i j ( m δ i y j ) W i j i j m [ δ i y j 3 / 2 ( W ^ i j ) 1 / 2 ] ,
W ^ i j = { W i j , sgn ( δ i ) = sgn ( W i j ) 0 , sgn ( δ i ) sgn ( W i j ) .
α ic β ic > i j ( m δ i y j ) W i j i j m [ δ i y j 3 / 2 ( W ^ i j ) 1 / 2 ] , W  ∈  S .
U = I total ( l ) Δ t = Δ t I ¯ [ N ( l 1 ) + 2 N ( l ) ] ,
G scs = ( α scs β scs ) 2 [ N ( l 1 ) + 2 N ( l ) ] 2 4 η sat ,
U scs = κ pr β scs .
G i c = ( α i c β i c ) 2 [ N ( l 1 ) + 2 N ( l ) ] 2 N ( l 1 ) 4 η sat ,
U ic = κ pr β ic 2 .
U scs ( l i ) = U scs ( l j ) , G scs ( l i ) = G scs ( l j ) [ N ( l i 1 ) + 2 N ( l i ) ] 2 [ N ( l j 1 ) + 2 N ( l j ) ] 2 .
U ic ( l i ) = U ic ( l j ) , G ic ( l i ) = G ic ( l j ) [ N ( l i 1 ) + 2 N ( l i ) ] 2 N ( l i 1 ) [ N ( l j 1 ) + 2 N ( l j ) ] 2 N ( l j 1 ) .

Metrics