Abstract

The finite impulse response neural network is described in detail. Different algorithms capable of temporal back-propagation are considered, including a novel modification to the conventional algorithm, called the delayed-feedback back-propagation algorithm. We present and analyze different optoelectronic processors making use of adaptive volume holograms and three-dimensional optical processing. Two single-layer architectures are presented: the input delay plane architecture and the output delay plane architecture. By combining them it is possible to implement both forward and backward propagation in two multi-layer architectures: the first making use of the conventional temporal back-propagation and the second making use of delayed-feedback back-propagation.

© 2002 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
    [CrossRef]
  2. K. J. Lang, A. H. Waibel, G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks 3, 23–43 (1990).
    [CrossRef]
  3. E. A. Wan, “Time Series Prediction by Using a Connectionist Network with Internal Delay Lines,” in Time Series Prediction: Forecasting the Future and Understanding the Past, SFI studies in the science of complexity, A. S. Weigend, N. A. Gershenfeld, eds., (Addison-Wesley, Reading, Mass., 1993).
  4. H. Gleisner, H. Lundstedt, P. Wintoft, “Predicting geomagnetic storms from solar-wind data using time-delay neural networks,” Annales Geophysicae 14, 676–686 (1996).
    [CrossRef]
  5. D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
    [CrossRef]
  6. F. Lavagetto, “Time-Delay Neural Networks For Estimating Lip Movements From Speech Analysis—A Useful Tool In Audio-Video Synchronization,” IEEE Trans. Circuits Systems For Video Technology 7, 786–800 (1997).
    [CrossRef]
  7. P. E. X. Silveira, G. S. Pati, K. H. Wagner, “Optical implementation of a single-layer finite impulse response neural network,” in Proc. Int. Conf. on Optics in Computing, R. A. Lessard, T. V. Galstian, eds., 4089, 656–667 (2000).
  8. P. E. X. Silveira, G. S. Pati, K. H. Wagner, to be published in Applied Optics.
  9. P. S. R. Diniz, in Adaptive Filtering: Algorithms and Practical Implementation (Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997), Chap. 1, pp. 8–13.
  10. L. J. G. B. Widrow, P. E. Mantey, B. B. Goode, “Adaptive Antenna Systems,” Proc. IEEE 55, 2143–2161 (1967).
    [CrossRef]
  11. A. Cauchy, “Méthode Générale pour la Résolution des Systémes d’équations Simultanées,” Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences 25, 536–538 (1847).
  12. B. Widrow, S. D. Stearns, Adaptive Signal Processing (Prentice Hall, Englewood Cliffs, N.J., 1985).
  13. A. V. VanderLugt, “Adaptive Optical Processor,” Appl. Opt. 21, 4005–4011 (1982).
    [CrossRef]
  14. D. Psaltis, J. Hong, “Adaptive acoustooptic filter,” Appl. Opt. 23, 3475–3481 (1984).
    [CrossRef] [PubMed]
  15. J. Rhodes, “Adaptive filter with a time-domain implementation using correlation cancellation loops,” Appl. Opt. 22, 282–287 (1983).
    [CrossRef] [PubMed]
  16. R. M. Montgomery, M. R. Lange, “Photorefractive adaptive filter structure with 40-dB interference rejection,” Appl. Opt. 30, 2844–2849 (1991).
    [CrossRef] [PubMed]
  17. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representation by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds., (MIT Press, Cambridge, Mass., 1986), Chap. 8.
  18. D. Psaltis, Y. Qiao, “Adaptive multilayer optical networks,” Prog. Opt. 31, 227–261 (1993).
    [CrossRef]
  19. M. A. Neifeld, D. Psaltis, “Optical implementations of radial basis classifiers,” Appl. Opt. 32, 1370–1379 (1993).
    [CrossRef] [PubMed]
  20. K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
    [CrossRef] [PubMed]
  21. K. Wagner, T. M. Slagle, “Optical competitive learning with VLSI liquid-crystal winner-take-all modulators,” Appl. Opt. 32, 1408–1435 (1993).
    [CrossRef] [PubMed]
  22. H. Toyoda, N. Mukohzaka, Y. Suzuki, M. Ishikawa, “Adaptive optical-processing system with optical associative memory,” Appl. Opt. 32, 1354–1358 (1993).
    [CrossRef] [PubMed]
  23. P. Lalanne, P. Chavel, J. Taboury, “Optical inner-product implementation of neural networks models,” Appl. Opt. 28, 377–385 (1989).
    [CrossRef] [PubMed]
  24. P. E. X. Silveira, K. H. Wagner, “Time delay optical neural network,” Proc. Int. Conf. on Optics in Computing 34, 266–269 (1998).
  25. G. Zhou, D. Z. Anderson, “Acoustic signal recognition with a photorefractive time-delay neural network,” Opt. Lett. 19, 655–657 (1994).
    [CrossRef] [PubMed]
  26. E. A. Wan, “Temporal backpropagation for FIR neural networks,” in Proc. Int. Joint Conf. Neural Networks, pp. I 575–580 (Omnipress, Madison, Wisc., 1990).
  27. P. Werbos, Ph.D. dissertation (Harvard University, Cambridge, Mass., 1974).
  28. K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).
  29. G. Kriehn, A. Kiruluta, P. E. X. Silveira, S. Weaver, S. Kraut, K. Wagner, R. T. Weverka, L. Griffiths, “Optical BEAMTAP beam-forming and jammer-nulling system for broadband phased-array antennas,” Appl. Opt. 39, 212–230 (2000).
    [CrossRef]
  30. D. R. Pape, “Multichannel Bragg cells: design, performance, and applications,” Opt. Eng. 31, 2148–2158 (1992).
    [CrossRef]
  31. G. Zhou, Ph.D. dissertation, (University of Colorado at Boulder, Boulder, Colo.1994).
  32. P. E. X. Silveira, K. H. Wagner, “Optical finite impulse response neural networks,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE, 3804, 72–81 (1999).
    [CrossRef]
  33. C. Bishop, in Neural Networks for Pattern Recognition (Clarendon Press, 1995), Chap. 9, pp. 338–340.
  34. A. J. M. Kiruluta, G. Kriehn, P. E. X. Silveira, K. H. Wagner, “Adaptive beamforming with TDI CCD-based true-time-delay processing,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE3804,(1999).
    [CrossRef]
  35. T. Merlet, D. Dolfi, J.-P. Huignard, “A traveling fringes photodetector for microwave signals,” IEEE J. Quantum Electron. 32, 778–783 (1996).
    [CrossRef]
  36. R. T. Weverka, K. Wagner, A. Sarto, “Photorefractive processing for large adaptive phased-arrays,” Appl. Opt. 35, 1344–1366 (1996).
    [CrossRef] [PubMed]
  37. A. W. Sarto, K. H. Wagner, R. T. Weverka, S. Weaver, E. K. Walge, “Wide angular aperture holograms in photorefractive crystals by the use of orthogonally polarized write and read beams,” Appl. Opt. 35, 5765–5775 (1996).
    [CrossRef] [PubMed]
  38. A. Sarto, Ph.D. dissertation (University of Colorado at Boulder, Boulder, Colo., 1996).

2000 (1)

1998 (1)

P. E. X. Silveira, K. H. Wagner, “Time delay optical neural network,” Proc. Int. Conf. on Optics in Computing 34, 266–269 (1998).

1997 (2)

D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
[CrossRef]

F. Lavagetto, “Time-Delay Neural Networks For Estimating Lip Movements From Speech Analysis—A Useful Tool In Audio-Video Synchronization,” IEEE Trans. Circuits Systems For Video Technology 7, 786–800 (1997).
[CrossRef]

1996 (4)

H. Gleisner, H. Lundstedt, P. Wintoft, “Predicting geomagnetic storms from solar-wind data using time-delay neural networks,” Annales Geophysicae 14, 676–686 (1996).
[CrossRef]

T. Merlet, D. Dolfi, J.-P. Huignard, “A traveling fringes photodetector for microwave signals,” IEEE J. Quantum Electron. 32, 778–783 (1996).
[CrossRef]

R. T. Weverka, K. Wagner, A. Sarto, “Photorefractive processing for large adaptive phased-arrays,” Appl. Opt. 35, 1344–1366 (1996).
[CrossRef] [PubMed]

A. W. Sarto, K. H. Wagner, R. T. Weverka, S. Weaver, E. K. Walge, “Wide angular aperture holograms in photorefractive crystals by the use of orthogonally polarized write and read beams,” Appl. Opt. 35, 5765–5775 (1996).
[CrossRef] [PubMed]

1994 (1)

1993 (4)

1992 (1)

D. R. Pape, “Multichannel Bragg cells: design, performance, and applications,” Opt. Eng. 31, 2148–2158 (1992).
[CrossRef]

1991 (1)

1990 (1)

K. J. Lang, A. H. Waibel, G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks 3, 23–43 (1990).
[CrossRef]

1989 (2)

A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
[CrossRef]

P. Lalanne, P. Chavel, J. Taboury, “Optical inner-product implementation of neural networks models,” Appl. Opt. 28, 377–385 (1989).
[CrossRef] [PubMed]

1987 (1)

1984 (1)

1983 (1)

1982 (1)

1967 (1)

L. J. G. B. Widrow, P. E. Mantey, B. B. Goode, “Adaptive Antenna Systems,” Proc. IEEE 55, 2143–2161 (1967).
[CrossRef]

1847 (1)

A. Cauchy, “Méthode Générale pour la Résolution des Systémes d’équations Simultanées,” Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences 25, 536–538 (1847).

Anderson, D. Z.

Bishop, C.

C. Bishop, in Neural Networks for Pattern Recognition (Clarendon Press, 1995), Chap. 9, pp. 338–340.

Cauchy, A.

A. Cauchy, “Méthode Générale pour la Résolution des Systémes d’équations Simultanées,” Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences 25, 536–538 (1847).

Chavel, P.

Clouse, D. S.

D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
[CrossRef]

Cottrell, G. W.

D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
[CrossRef]

Diniz, P. S. R.

P. S. R. Diniz, in Adaptive Filtering: Algorithms and Practical Implementation (Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997), Chap. 1, pp. 8–13.

Dolfi, D.

T. Merlet, D. Dolfi, J.-P. Huignard, “A traveling fringes photodetector for microwave signals,” IEEE J. Quantum Electron. 32, 778–783 (1996).
[CrossRef]

Giles, C. L.

D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
[CrossRef]

Gleisner, H.

H. Gleisner, H. Lundstedt, P. Wintoft, “Predicting geomagnetic storms from solar-wind data using time-delay neural networks,” Annales Geophysicae 14, 676–686 (1996).
[CrossRef]

Goode, B. B.

L. J. G. B. Widrow, P. E. Mantey, B. B. Goode, “Adaptive Antenna Systems,” Proc. IEEE 55, 2143–2161 (1967).
[CrossRef]

Griffiths, L.

G. Kriehn, A. Kiruluta, P. E. X. Silveira, S. Weaver, S. Kraut, K. Wagner, R. T. Weverka, L. Griffiths, “Optical BEAMTAP beam-forming and jammer-nulling system for broadband phased-array antennas,” Appl. Opt. 39, 212–230 (2000).
[CrossRef]

K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).

Hanazawa, T.

A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
[CrossRef]

Hinton, G. E.

K. J. Lang, A. H. Waibel, G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks 3, 23–43 (1990).
[CrossRef]

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representation by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds., (MIT Press, Cambridge, Mass., 1986), Chap. 8.

Hong, J.

Horne, B. G.

D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
[CrossRef]

Huignard, J.-P.

T. Merlet, D. Dolfi, J.-P. Huignard, “A traveling fringes photodetector for microwave signals,” IEEE J. Quantum Electron. 32, 778–783 (1996).
[CrossRef]

Ishikawa, M.

Kiruluta, A.

Kiruluta, A. J. M.

A. J. M. Kiruluta, G. Kriehn, P. E. X. Silveira, K. H. Wagner, “Adaptive beamforming with TDI CCD-based true-time-delay processing,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE3804,(1999).
[CrossRef]

Kraut, S.

G. Kriehn, A. Kiruluta, P. E. X. Silveira, S. Weaver, S. Kraut, K. Wagner, R. T. Weverka, L. Griffiths, “Optical BEAMTAP beam-forming and jammer-nulling system for broadband phased-array antennas,” Appl. Opt. 39, 212–230 (2000).
[CrossRef]

K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).

Kriehn, G.

G. Kriehn, A. Kiruluta, P. E. X. Silveira, S. Weaver, S. Kraut, K. Wagner, R. T. Weverka, L. Griffiths, “Optical BEAMTAP beam-forming and jammer-nulling system for broadband phased-array antennas,” Appl. Opt. 39, 212–230 (2000).
[CrossRef]

A. J. M. Kiruluta, G. Kriehn, P. E. X. Silveira, K. H. Wagner, “Adaptive beamforming with TDI CCD-based true-time-delay processing,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE3804,(1999).
[CrossRef]

Lalanne, P.

Lang, K. J.

K. J. Lang, A. H. Waibel, G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks 3, 23–43 (1990).
[CrossRef]

A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
[CrossRef]

Lange, M. R.

Lavagetto, F.

F. Lavagetto, “Time-Delay Neural Networks For Estimating Lip Movements From Speech Analysis—A Useful Tool In Audio-Video Synchronization,” IEEE Trans. Circuits Systems For Video Technology 7, 786–800 (1997).
[CrossRef]

Lundstedt, H.

H. Gleisner, H. Lundstedt, P. Wintoft, “Predicting geomagnetic storms from solar-wind data using time-delay neural networks,” Annales Geophysicae 14, 676–686 (1996).
[CrossRef]

Mantey, P. E.

L. J. G. B. Widrow, P. E. Mantey, B. B. Goode, “Adaptive Antenna Systems,” Proc. IEEE 55, 2143–2161 (1967).
[CrossRef]

Merlet, T.

T. Merlet, D. Dolfi, J.-P. Huignard, “A traveling fringes photodetector for microwave signals,” IEEE J. Quantum Electron. 32, 778–783 (1996).
[CrossRef]

Montgomery, R. M.

Mukohzaka, N.

Neifeld, M. A.

Pape, D. R.

D. R. Pape, “Multichannel Bragg cells: design, performance, and applications,” Opt. Eng. 31, 2148–2158 (1992).
[CrossRef]

Pati, G. S.

P. E. X. Silveira, G. S. Pati, K. H. Wagner, to be published in Applied Optics.

P. E. X. Silveira, G. S. Pati, K. H. Wagner, “Optical implementation of a single-layer finite impulse response neural network,” in Proc. Int. Conf. on Optics in Computing, R. A. Lessard, T. V. Galstian, eds., 4089, 656–667 (2000).

Psaltis, D.

Qiao, Y.

D. Psaltis, Y. Qiao, “Adaptive multilayer optical networks,” Prog. Opt. 31, 227–261 (1993).
[CrossRef]

Rhodes, J.

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representation by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds., (MIT Press, Cambridge, Mass., 1986), Chap. 8.

Sarto, A.

R. T. Weverka, K. Wagner, A. Sarto, “Photorefractive processing for large adaptive phased-arrays,” Appl. Opt. 35, 1344–1366 (1996).
[CrossRef] [PubMed]

A. Sarto, Ph.D. dissertation (University of Colorado at Boulder, Boulder, Colo., 1996).

K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).

Sarto, A. W.

Shikano, G. E. H. K.

A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
[CrossRef]

Silveira, P. E. X.

G. Kriehn, A. Kiruluta, P. E. X. Silveira, S. Weaver, S. Kraut, K. Wagner, R. T. Weverka, L. Griffiths, “Optical BEAMTAP beam-forming and jammer-nulling system for broadband phased-array antennas,” Appl. Opt. 39, 212–230 (2000).
[CrossRef]

P. E. X. Silveira, K. H. Wagner, “Time delay optical neural network,” Proc. Int. Conf. on Optics in Computing 34, 266–269 (1998).

P. E. X. Silveira, G. S. Pati, K. H. Wagner, “Optical implementation of a single-layer finite impulse response neural network,” in Proc. Int. Conf. on Optics in Computing, R. A. Lessard, T. V. Galstian, eds., 4089, 656–667 (2000).

P. E. X. Silveira, G. S. Pati, K. H. Wagner, to be published in Applied Optics.

P. E. X. Silveira, K. H. Wagner, “Optical finite impulse response neural networks,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE, 3804, 72–81 (1999).
[CrossRef]

A. J. M. Kiruluta, G. Kriehn, P. E. X. Silveira, K. H. Wagner, “Adaptive beamforming with TDI CCD-based true-time-delay processing,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE3804,(1999).
[CrossRef]

Slagle, T. M.

Stearns, S. D.

B. Widrow, S. D. Stearns, Adaptive Signal Processing (Prentice Hall, Englewood Cliffs, N.J., 1985).

Suzuki, Y.

Taboury, J.

Toyoda, H.

VanderLugt, A. V.

Wagner, K.

Wagner, K. H.

P. E. X. Silveira, K. H. Wagner, “Time delay optical neural network,” Proc. Int. Conf. on Optics in Computing 34, 266–269 (1998).

A. W. Sarto, K. H. Wagner, R. T. Weverka, S. Weaver, E. K. Walge, “Wide angular aperture holograms in photorefractive crystals by the use of orthogonally polarized write and read beams,” Appl. Opt. 35, 5765–5775 (1996).
[CrossRef] [PubMed]

A. J. M. Kiruluta, G. Kriehn, P. E. X. Silveira, K. H. Wagner, “Adaptive beamforming with TDI CCD-based true-time-delay processing,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE3804,(1999).
[CrossRef]

P. E. X. Silveira, K. H. Wagner, “Optical finite impulse response neural networks,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE, 3804, 72–81 (1999).
[CrossRef]

K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).

P. E. X. Silveira, G. S. Pati, K. H. Wagner, to be published in Applied Optics.

P. E. X. Silveira, G. S. Pati, K. H. Wagner, “Optical implementation of a single-layer finite impulse response neural network,” in Proc. Int. Conf. on Optics in Computing, R. A. Lessard, T. V. Galstian, eds., 4089, 656–667 (2000).

Waibel, A. H.

K. J. Lang, A. H. Waibel, G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks 3, 23–43 (1990).
[CrossRef]

A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
[CrossRef]

Walge, E. K.

Wan, E. A.

E. A. Wan, “Time Series Prediction by Using a Connectionist Network with Internal Delay Lines,” in Time Series Prediction: Forecasting the Future and Understanding the Past, SFI studies in the science of complexity, A. S. Weigend, N. A. Gershenfeld, eds., (Addison-Wesley, Reading, Mass., 1993).

E. A. Wan, “Temporal backpropagation for FIR neural networks,” in Proc. Int. Joint Conf. Neural Networks, pp. I 575–580 (Omnipress, Madison, Wisc., 1990).

Weaver, S.

Werbos, P.

P. Werbos, Ph.D. dissertation (Harvard University, Cambridge, Mass., 1974).

Weverka, R.

K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).

Weverka, R. T.

Widrow, B.

B. Widrow, S. D. Stearns, Adaptive Signal Processing (Prentice Hall, Englewood Cliffs, N.J., 1985).

Widrow, L. J. G. B.

L. J. G. B. Widrow, P. E. Mantey, B. B. Goode, “Adaptive Antenna Systems,” Proc. IEEE 55, 2143–2161 (1967).
[CrossRef]

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representation by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds., (MIT Press, Cambridge, Mass., 1986), Chap. 8.

Wintoft, P.

H. Gleisner, H. Lundstedt, P. Wintoft, “Predicting geomagnetic storms from solar-wind data using time-delay neural networks,” Annales Geophysicae 14, 676–686 (1996).
[CrossRef]

Zhou, G.

Annales Geophysicae (1)

H. Gleisner, H. Lundstedt, P. Wintoft, “Predicting geomagnetic storms from solar-wind data using time-delay neural networks,” Annales Geophysicae 14, 676–686 (1996).
[CrossRef]

Appl. Opt. (12)

A. V. VanderLugt, “Adaptive Optical Processor,” Appl. Opt. 21, 4005–4011 (1982).
[CrossRef]

D. Psaltis, J. Hong, “Adaptive acoustooptic filter,” Appl. Opt. 23, 3475–3481 (1984).
[CrossRef] [PubMed]

J. Rhodes, “Adaptive filter with a time-domain implementation using correlation cancellation loops,” Appl. Opt. 22, 282–287 (1983).
[CrossRef] [PubMed]

R. M. Montgomery, M. R. Lange, “Photorefractive adaptive filter structure with 40-dB interference rejection,” Appl. Opt. 30, 2844–2849 (1991).
[CrossRef] [PubMed]

M. A. Neifeld, D. Psaltis, “Optical implementations of radial basis classifiers,” Appl. Opt. 32, 1370–1379 (1993).
[CrossRef] [PubMed]

K. Wagner, D. Psaltis, “Multilayer optical learning networks,” Appl. Opt. 26, 5061–5076 (1987).
[CrossRef] [PubMed]

K. Wagner, T. M. Slagle, “Optical competitive learning with VLSI liquid-crystal winner-take-all modulators,” Appl. Opt. 32, 1408–1435 (1993).
[CrossRef] [PubMed]

H. Toyoda, N. Mukohzaka, Y. Suzuki, M. Ishikawa, “Adaptive optical-processing system with optical associative memory,” Appl. Opt. 32, 1354–1358 (1993).
[CrossRef] [PubMed]

P. Lalanne, P. Chavel, J. Taboury, “Optical inner-product implementation of neural networks models,” Appl. Opt. 28, 377–385 (1989).
[CrossRef] [PubMed]

G. Kriehn, A. Kiruluta, P. E. X. Silveira, S. Weaver, S. Kraut, K. Wagner, R. T. Weverka, L. Griffiths, “Optical BEAMTAP beam-forming and jammer-nulling system for broadband phased-array antennas,” Appl. Opt. 39, 212–230 (2000).
[CrossRef]

R. T. Weverka, K. Wagner, A. Sarto, “Photorefractive processing for large adaptive phased-arrays,” Appl. Opt. 35, 1344–1366 (1996).
[CrossRef] [PubMed]

A. W. Sarto, K. H. Wagner, R. T. Weverka, S. Weaver, E. K. Walge, “Wide angular aperture holograms in photorefractive crystals by the use of orthogonally polarized write and read beams,” Appl. Opt. 35, 5765–5775 (1996).
[CrossRef] [PubMed]

Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences (1)

A. Cauchy, “Méthode Générale pour la Résolution des Systémes d’équations Simultanées,” Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences 25, 536–538 (1847).

IEEE J. Quantum Electron. (1)

T. Merlet, D. Dolfi, J.-P. Huignard, “A traveling fringes photodetector for microwave signals,” IEEE J. Quantum Electron. 32, 778–783 (1996).
[CrossRef]

IEEE Trans. Acoust. Speech Signal Process. (1)

A. H. Waibel, T. Hanazawa, G. E. H. K. Shikano, K. J. Lang, “Phoneme Recognition Using Time-Delay Neural Networks,” IEEE Trans. Acoust. Speech Signal Process. 37, 328–339 (1989).
[CrossRef]

IEEE Trans. Circuits Systems For Video Technology (1)

F. Lavagetto, “Time-Delay Neural Networks For Estimating Lip Movements From Speech Analysis—A Useful Tool In Audio-Video Synchronization,” IEEE Trans. Circuits Systems For Video Technology 7, 786–800 (1997).
[CrossRef]

IEEE Trans. Neural Netw. (1)

D. S. Clouse, C. L. Giles, B. G. Horne, G. W. Cottrell, “Time-Delay Neural Networks—Representation and Induction of Finite-State Machines,” IEEE Trans. Neural Netw. 8, 1065–1070 (1997).
[CrossRef]

Neural Networks (1)

K. J. Lang, A. H. Waibel, G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neural Networks 3, 23–43 (1990).
[CrossRef]

Opt. Eng. (1)

D. R. Pape, “Multichannel Bragg cells: design, performance, and applications,” Opt. Eng. 31, 2148–2158 (1992).
[CrossRef]

Opt. Lett. (1)

Proc. IEEE (1)

L. J. G. B. Widrow, P. E. Mantey, B. B. Goode, “Adaptive Antenna Systems,” Proc. IEEE 55, 2143–2161 (1967).
[CrossRef]

Proc. Int. Conf. on Optics in Computing (1)

P. E. X. Silveira, K. H. Wagner, “Time delay optical neural network,” Proc. Int. Conf. on Optics in Computing 34, 266–269 (1998).

Prog. Opt. (1)

D. Psaltis, Y. Qiao, “Adaptive multilayer optical networks,” Prog. Opt. 31, 227–261 (1993).
[CrossRef]

Other (14)

B. Widrow, S. D. Stearns, Adaptive Signal Processing (Prentice Hall, Englewood Cliffs, N.J., 1985).

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representation by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds., (MIT Press, Cambridge, Mass., 1986), Chap. 8.

E. A. Wan, “Time Series Prediction by Using a Connectionist Network with Internal Delay Lines,” in Time Series Prediction: Forecasting the Future and Understanding the Past, SFI studies in the science of complexity, A. S. Weigend, N. A. Gershenfeld, eds., (Addison-Wesley, Reading, Mass., 1993).

P. E. X. Silveira, G. S. Pati, K. H. Wagner, “Optical implementation of a single-layer finite impulse response neural network,” in Proc. Int. Conf. on Optics in Computing, R. A. Lessard, T. V. Galstian, eds., 4089, 656–667 (2000).

P. E. X. Silveira, G. S. Pati, K. H. Wagner, to be published in Applied Optics.

P. S. R. Diniz, in Adaptive Filtering: Algorithms and Practical Implementation (Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997), Chap. 1, pp. 8–13.

E. A. Wan, “Temporal backpropagation for FIR neural networks,” in Proc. Int. Joint Conf. Neural Networks, pp. I 575–580 (Omnipress, Madison, Wisc., 1990).

P. Werbos, Ph.D. dissertation (Harvard University, Cambridge, Mass., 1974).

K. H. Wagner, S. Kraut, L. Griffiths, S. Weaver, R. Weverka, A. Sarto, “Efficient true-time-delay adaptive array processing,” in Proc. SPIERadar Processing, Technology, and Applications, W. J. Miceli, ed., 2845, (1996).

G. Zhou, Ph.D. dissertation, (University of Colorado at Boulder, Boulder, Colo.1994).

P. E. X. Silveira, K. H. Wagner, “Optical finite impulse response neural networks,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE, 3804, 72–81 (1999).
[CrossRef]

C. Bishop, in Neural Networks for Pattern Recognition (Clarendon Press, 1995), Chap. 9, pp. 338–340.

A. J. M. Kiruluta, G. Kriehn, P. E. X. Silveira, K. H. Wagner, “Adaptive beamforming with TDI CCD-based true-time-delay processing,” in Algorithms, Devices, and Systems for Optical Information Processing III, B. Javidi, D. Psaltis, eds., Proc. SPIE3804,(1999).
[CrossRef]

A. Sarto, Ph.D. dissertation (University of Colorado at Boulder, Boulder, Colo., 1996).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

Two representations of a continuous time FIR filter. (a) Space-integrating version, in which the input signal is first delayed, then weighted and summed. (b) Time-integrating version, in which the weighted input signal is summed in a traveling delay line. Both versions perform exactly the same operation: a continuously flowing windowed convolution between the weights and the input signal.

Fig. 2
Fig. 2

An FIRNN depicted as a neural network in which its interconnects are composed of FIR filters. N l and T l denote the number of nodes and time delays present at layer l. s i l (k) is the input node i to layer l, and σ j l+1(k) is the output summation node j from all FIRs from layer l. A sigmoidal non-linear function is shown in this case.

Fig. 3
Fig. 3

Signal flow in the last layer of an FIRNN during (a) forward propagation and (b) backward propagation. The weights shown in (a) and (b) are exactly the same, but signals flow in opposite direction in each case. Z -1 represents a discrete time delay.

Fig. 4
Fig. 4

The single-layer FIRNN. (a) Conventional LMS: each input node is applied to a tap delay line. Each delayed input is multiplied by its respective weight and the partial products are summed at the output, which is subtracted from a desired signal producing an error signal. The error signal is locally multiplied by the delayed input and integrated at each weight. (b) BEAMTAP version: the inputs are simultaneously multiplied by all the weights at every time step. The resulting products are time delayed and summed by the output scrolling delay line accumulator, producing the final output, which is subtracted from a desired signal. The resulting error signal is delayed by another delay line and its delayed versions are multiplied by a delayed version of the input signal and locally integrated at each weight.

Fig. 5
Fig. 5

SLM frame captured in the image plane, displaying a continuous broadband binary chirp.

Fig. 6
Fig. 6

Fourier plane representation of the modulated signal from the SLM. It is a long and thin line with constant tilt, showing that the signal is temporally broadband, and that there is a linear delay between SLM columns. The periodic sampling along the slice is due to the double repetition of the signal within the SLM aperture.

Fig. 7
Fig. 7

Single-layer FIRNN architectures. Solid lines represent optic signals and dashed lines represent electronic signals. (a), Input delay plane architecture. A scrolling SLM modulates an incoming beam with delayed versions of the input vector s¯, producing spatio-temporal input plane s¯¯ which is diffracted by the PR volume hologram. The diffracted beam is spatially integrated onto the output detector array and the output non-linear function is electronically implemented. The δ¯ error term is used to modulate a reference beam which is imaged through the PRC onto the detector so that in the PRC it interferes with the input beam at the dynamic hologram, modifying the interconnection grating. (b) Output delay plane, or BEAMTAP architecture. The input vector is used to modulate an optic beam at the input, which is diffracted by a volume hologram and detected by a TDI-CCD. The holographic diffraction performs the space integration while the TDI-CCD performs a temporal convolution as it shifts and accumulates the detected signals. The error signal is propagated through the velocity matched 2-D scrolling SLM, producing the plane δ¯¯, which is imaged through the PRC onto the TDI-CCD so that it interferes in the PRC with the input signal, modifying the interconnection grating.

Fig. 8
Fig. 8

Two-layer optical architecture based on Wan’s temporal back-propagation learning algorithm. Optic signals are represented by solid lines and electronic signals are represented by dashed lines. Operations inside the gray box are performed electronically, possibly using smart-pixel arrays. The input delay plane architecture is used for forward propagation and the output delay plane architecture is used for backward propagation of the output δ terms. The inset describes how the input half-plane s¯¯ (0)(k) is combined with the half-plane s¯¯ (0)(k - T), the latter used for updating the PR dynamic hologram read-out by the first.

Fig. 9
Fig. 9

Two-layer optical architecture based on the delayed-feedback back-propagation learning algorithm. Again, operations inside the gray box are performed electronically, possibly by smart-pixel devices. The output plane architecture is used for forward propagation and the input plane architecture is used for backward propagation of the output δ terms. The combiner used is the same described in the previous architecture.

Fig. 10
Fig. 10

Photorefractive grating formation and readout in FIRNN multi-layer architectures. SLM 1 is imaged pixel by pixel onto detector array 2 and SLM 2 is imaged pixel by pixel onto detector array 1. The dimension of the SLMs and the detectors (1-D or 2-D) vary according to which architecture is being analyzed (conventional or delayed-feedback). However, the diagram and its analysis apply to both architectures.

Equations (63)

Equations on this page are rendered with MathJax. Learn more.

σjl+1κ=i=1Nlτ=0Tl wji,τl|κsilκ-τ,
sjl+1κ=fσjl+1κ,
s¯l+1κ=fw¯¯¯l : s¯¯lκ,
C=κ=1K e¯κ2.
wji,τlκ+1=wji,τlκ-η Cwji,τl,
δjlκ=Cσjlκ.
wji,τL-1-nκ+1=wji,τL-1-nκ-ηδjL-nκ-Υn×siL-1-nκ-Υn-τ,
Υn=0n=0Υn-1+TL-n0<nL.
δjL-nκ-Υn=-2fσjLκ ejκn=0fσjL-nκ-Υnm=1NL-n+1c=0TL-n wmj,cL-n δmL-n+1κ-Υn+c1nL-1.
Δw¯¯¯L-1-nκ=-ηδ¯L-nκ-Υns¯¯L-1-nκ-Υn,
δ¯L-nκ-Υn=-2fσ¯Lκe¯κn=0fσ¯L-nκ-Υnw¯¯¯L-n : δ¯¯L-n+1κ-Υn1nL-1.
δ¯¯L-n+1κ-Υn=δ¯L-n+1κ-Υn+1δ¯L-n+1κ-Υn+1+1δ¯L-n+1κ-Υn-1δ¯L-n+1κ-Υn
ojκ=i=1Nτ=0T wji,τκsiκ-τ.
wji,τκ+1=wji,τκ-ηδjκsiκ-τ,
wji,τκ+1=wji,τκ+2ηejκ-T+τsiκ-T.
ojκ=i=1Nτ=0T wji,τκ-τsiκ-τ.
sjl+1κ=fi=1Nτ=0T wji,τlκ-τsilκ-τ.
wji,τL-n-1κ+1=wji,τL-n-1κ-ηδjL-nκ-Υn+1+τsiL-n-1κ-Υn+1,
δjL-nκ-Υn+1+τ=-2fsjLκ-TL-1+τejκ-TL-1+τn=0fsjL-nκ-Υn+1+τm=1NL-n+1c=0TL-n wjm,cL-nδmL-n+1κ-Υn+1+c+τ1nL-1.
Δw¯¯¯L-1-nκ=-ηδ¯¯L-nκ-Υns¯L-1-nκ-Υn,
Ei,τAy, z, t=ηSE0 expiωt+kx× z-iΔzΔz  y-τΔyΔy× t-κ-τΔtΔtsiκ-τ+c.c.
EAy, z, κ=i=1N0τ=0T0 Ei,τAy, z, t.
σjκ=Zoηdi,τ Gji,τκEi,τAκ+Er2,
EjBx, y, t=ηmE0gywyexpiωt-kz× x-jΔxΔx  t-κΔtΔtδjκ+c.c.,
EBx, y, κ=j=0N1 EjBx, y, t.
Ġji,τκ=βEjBκEi,τA*κ-Gji,τκτprI,
EiAy, z, t=ηmE0gywyexpiwt+kx× z-iΔzΔz  t-κΔtΔtsiκ+c.c.,
EAy, z, t=i=1N0 EiAy, z, t.
aj,γκ=ηdτ=0γi=1N1 Gji,T-γ+τκ+T-γ-τ×Ei,τAκ+Er2Δt.
σjκ  aj,Tκ=Er*ηdi,τ Gji,τκ-τEi,τAκ+c.c.+bias.
Ej,τBx, y, t=ηmE0gywyexpiωt-kz× x-jΔxΔx× t-κ-τΔtΔtδjκ-τ,
EBx, y, κ=j=1N1τ=0T0 Ej,τBx, y, t.
G˙ji,τκ=βEj,τBκEiA*κ-T-Gji,τκτprI,
σj2κ=Z0ηdi,τ Gji,τ2κEi,τAκ+Er2,
Ġji,τ2κ=β2EjBκEi,τA*κ-Gji,τκτpr2I,
ai,γκ=ηdτ=0γj=1N2 Ej,τBκGji,T-γ+τ2*κ+T-γ-τ+Er2Δt.
ai,Tκ=ηdErj,τ Ej,τBκGji,τ2*κ-τ+c.c.+bias.
wji,τlκ+1=wji,τlκ+Δwji,τlκ-η Cwji,τl,
Δwji,τlκ=-η Cσjl+1κ-ξlσjl+1κ-ξlwji,τl,
δjlκ-ξl=Cσjlκ-ξl,
σjl+1κ-ξlwji,τl=j=1Nlτ=0Tl wji,τlsilκ-τ-ξlwji,τl=silκ-τ-ξl.
δjLκ-ξl=CσjLκ-ξl, =ej2κ-ξlσjLκ-ξl,
δjLκ-ξl=djκ-ξl-fσjLκ-ξl2σjLκ-ξl=dj2κ-ξl-2djκ-ξlfσjLκ-ξl+f2σjLκ-ξlσjLκ-ξl, =-2fσjLκ-ξldjκ-ξl-fσjLκ-ξlejκ-ξl, =-2fσjLκ-ξlejκ-ξl.
δjlk-ξl=m=1Nl+1t=-+Cσml+1t-ξlσml+1t-ξl+1σjlk-ξl+1=m=1N1+1t=-+ δml+1t-ξl+1j=1Nlτ=0Tl wmj,τl sjlt-τ-ξlsjlκ-ξlsjlκ-ξlσjlκ-ξl,
δjlκ-ξl=m=1Nl+1t=κκ+Tl wmj,t-κlδml+1×t-ξl+1fsjlκ-ξl =fσjlκ-ξlm=1Nl+1c=0Tl wmj,clδml+1×κ+c-ξl+1,
ξl=ΥL-l+1-τ,
Δwji,τL-n-1κ=-ηδjL-nκ-Υn+1+τsiL-n-1κ-Υn+1
δjL-nκ-Υn+1+τ=-2fσjLκ-TL-1+τejκ-TL-1+τn=0fσjL-nκ-Υn+1+τm=1NL-n+1c=0TL-n wmj,cL-nδmL-n+1κ-Υn+1+c+τ1nL-1.
EAx, y, z; t=Dxy,z:y1,z13y1,z1:y0,z0EAz0; tδy0.
EBx, y, z; t=Dzx,y:x1,y11x1,y1:x0,y0EBx0, y0; t.
2x2,y2:x1,y11x1,y1:x0,y0Mx2,y2:x0,y0,
4y2,z2:z1,y13z1,y1:z0,y0My2,z2:z0,y0,
Dzx,y:x,yEx, y, z=0; t=expi 2πλ ziλz  Ex, y, z=0; texpiπλzx-x2+y-y2dxdy,
Dz*x,y:x,yD-zx,y:x,y,
D-zx,y:x,yDzx,y:x,y.
Gx, y, z; t=β -t EA*x, y, z; tEBx, y, z; t×exp-t-tτprdt,
Edfx2, y2; t=2x2,y2:x1,y10L1 D-zx1,y1:x,yEAx, y, z, tGx, y, z, tdz,
Edfx2, y2; t=β2x2,y2:x1,y10L1 D-zx1,y1:x,yDxy,z:y1,z13y1,z1:y0,z0×EAz0; tδy0×-tDxy,z:y1,z13y1,z1:y0,z0*EA*z0;tδy0×Dzx,y:x1,y11x1,y1:x0,y0 EBx0, y0; te-t-tτprdtdz.
Edfx2, y2; t=β2x2,y2:x1,y10L1 D-zx1,y1:x,yEAz0; t×-t EA*z0; t×EBx, y, z; te- t-tτprdtdz.
Edfx2, y2; t=β2x2,y2:x1,y10L1 D-zx1,y1:x,yEAz0; t-t EA*z0; t×Dzx,y:x1,y11x1,y1:x0,y0EBx0, y0; te- t-tτprdtdz.
Edfx2, y2; t=βMx2,y2:x0,y00W1EAz0; t-tEA* z0; t×EBx0, y0; texp-t-tτprdtdz0,
Edby2, z2; t=4y2,z2:z1,y10L2 D-xy1,z1:y,zEBx, y, z, tG*x, y, z, tdx,
Edbz2; t=βMy2,z2:y0,z00Wy20Wx2 EBx0, y0; t×-t EB*x0, y0; tEAz0; tδy0×exp-t-tτprdtdx0, dy0,

Metrics