Abstract

In the first part of this paper we present frequency multiplexed raster (FMR) optical implementation of neural networks. A hidden difficulty for hardware (optical and electronic) implementation is that the dimension of the synaptic matrix is twice that of the input and output matrices or vectors. For 2-D images, which is we believe one of the greatest potentialities of neural networks, the synaptic matrix is 4-D and cannot be directly implemented in optics. We propose FMR as a method to fold this matrix into a 2-D format. In the second part of this paper we describe the system built in our laboratory showing the feasibility of FMR optical neural networks. The system is built from an optical input module, a fixed synaptic matrix coded on a transparency, a CCD camera, and a microcomputer which performs the thresholding and feedback operations. In a later stage the fixed matrix will be replaced by a programmable matrix.

© 1989 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. 79, 2254 (1982); “Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
    [CrossRef]
  2. S. Venkatesh, D. Psaltis, “Information Storage and Retrieval in the Associative Nets,” IEEE Trans. Inf. Theory, submitted.
  3. L. Personnaz, I. Guyon, G. Dreyfus, “Information Storage and Retrieval in Spin-Like Neural Networks,” J. Phys. Paris Lett. 46, L359 (1985);L. Personnaz, I. Guyon, G. Dreyfus, G. Toulouse, “A Biologically Constrained Learning Mechanism in Networks of Formal Neurons,” J. Stat. Phys. 43, 411 (1986).
    [CrossRef]
  4. A. D. Maruani, R. C. Chevallier, G. Y. Sirat, “Information Retrieval in Neural Networks. Eigenproblems in Neural Networks,” Rev. Phys. Appl. 22, 1321 (1987).
    [CrossRef]
  5. G. Y. Sirat, R. C. Chevallier, A. D. Maruani, “Grey Level Neural Networks,” Appl. Opt.28, 414 (1989), submitted;A. J. Noest, “Discrete-State Phasor Neural Networks,” Phys. Rev. A, submitted;A. J. Noest, “Associative Memory in Sparse Phasor,” Europhys. Lett., in press.
    [CrossRef] [PubMed]
  6. D. Psaltis, N. Farhat, “Optical Information Processing Based on an Associative-Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett. 10, 98 (1985).
    [CrossRef] [PubMed]
  7. N. Farhat, D. Psaltis, “Architectures for Optical Implementation of 2-D Content Addressable Memories,” in Technical Digest of the Annual Meeting (Optical Society of America, Washington, DC, 1985), paperWT3.
  8. E. E. Fournier d’Albe, British Pat. 233, 746 (15 May1925).
  9. W. T. Rhodes, “Acousto-Optic Devices Applied to Image Processing,” Proc. Soc. Photo-Opt. Instrum. Eng. 180, 143 (1979).
  10. G. Y. Sirat, “Imaging System Based on Frequency Multiplexing Utilizing the Piezo-Electric-Elasto-Optic Effect,” Ph.D. Thesis, Hebrew University (Mar.1983);N. Ben-Yosef, G. Sirat, “Real-Time Spatial Filtering Utilizing the Piezo-ElectricElasto-Optic Effect,” Opt. Acta 29, 419 (1982).
    [CrossRef]
  11. N. Ben-Yosef, G. Sirat, “Signal to Noise Ratio in a Frequency Multiplexing Imager,” IEEE J. Quantum Electron. QE-20, (1984).
  12. D. Psaltis et al., “Shift Invariance in Optical Associative Memories,” Proc. Soc. Photo-Opt. Instrum. Eng.625, (1986).

1987 (1)

A. D. Maruani, R. C. Chevallier, G. Y. Sirat, “Information Retrieval in Neural Networks. Eigenproblems in Neural Networks,” Rev. Phys. Appl. 22, 1321 (1987).
[CrossRef]

1986 (1)

D. Psaltis et al., “Shift Invariance in Optical Associative Memories,” Proc. Soc. Photo-Opt. Instrum. Eng.625, (1986).

1985 (2)

D. Psaltis, N. Farhat, “Optical Information Processing Based on an Associative-Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett. 10, 98 (1985).
[CrossRef] [PubMed]

L. Personnaz, I. Guyon, G. Dreyfus, “Information Storage and Retrieval in Spin-Like Neural Networks,” J. Phys. Paris Lett. 46, L359 (1985);L. Personnaz, I. Guyon, G. Dreyfus, G. Toulouse, “A Biologically Constrained Learning Mechanism in Networks of Formal Neurons,” J. Stat. Phys. 43, 411 (1986).
[CrossRef]

1984 (1)

N. Ben-Yosef, G. Sirat, “Signal to Noise Ratio in a Frequency Multiplexing Imager,” IEEE J. Quantum Electron. QE-20, (1984).

1982 (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. 79, 2254 (1982); “Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
[CrossRef]

1979 (1)

W. T. Rhodes, “Acousto-Optic Devices Applied to Image Processing,” Proc. Soc. Photo-Opt. Instrum. Eng. 180, 143 (1979).

1925 (1)

E. E. Fournier d’Albe, British Pat. 233, 746 (15 May1925).

Ben-Yosef, N.

N. Ben-Yosef, G. Sirat, “Signal to Noise Ratio in a Frequency Multiplexing Imager,” IEEE J. Quantum Electron. QE-20, (1984).

Chevallier, R. C.

A. D. Maruani, R. C. Chevallier, G. Y. Sirat, “Information Retrieval in Neural Networks. Eigenproblems in Neural Networks,” Rev. Phys. Appl. 22, 1321 (1987).
[CrossRef]

G. Y. Sirat, R. C. Chevallier, A. D. Maruani, “Grey Level Neural Networks,” Appl. Opt.28, 414 (1989), submitted;A. J. Noest, “Discrete-State Phasor Neural Networks,” Phys. Rev. A, submitted;A. J. Noest, “Associative Memory in Sparse Phasor,” Europhys. Lett., in press.
[CrossRef] [PubMed]

Dreyfus, G.

L. Personnaz, I. Guyon, G. Dreyfus, “Information Storage and Retrieval in Spin-Like Neural Networks,” J. Phys. Paris Lett. 46, L359 (1985);L. Personnaz, I. Guyon, G. Dreyfus, G. Toulouse, “A Biologically Constrained Learning Mechanism in Networks of Formal Neurons,” J. Stat. Phys. 43, 411 (1986).
[CrossRef]

Farhat, N.

D. Psaltis, N. Farhat, “Optical Information Processing Based on an Associative-Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett. 10, 98 (1985).
[CrossRef] [PubMed]

N. Farhat, D. Psaltis, “Architectures for Optical Implementation of 2-D Content Addressable Memories,” in Technical Digest of the Annual Meeting (Optical Society of America, Washington, DC, 1985), paperWT3.

Fournier d’Albe, E. E.

E. E. Fournier d’Albe, British Pat. 233, 746 (15 May1925).

Guyon, I.

L. Personnaz, I. Guyon, G. Dreyfus, “Information Storage and Retrieval in Spin-Like Neural Networks,” J. Phys. Paris Lett. 46, L359 (1985);L. Personnaz, I. Guyon, G. Dreyfus, G. Toulouse, “A Biologically Constrained Learning Mechanism in Networks of Formal Neurons,” J. Stat. Phys. 43, 411 (1986).
[CrossRef]

Hopfield, J. J.

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. 79, 2254 (1982); “Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
[CrossRef]

Maruani, A. D.

A. D. Maruani, R. C. Chevallier, G. Y. Sirat, “Information Retrieval in Neural Networks. Eigenproblems in Neural Networks,” Rev. Phys. Appl. 22, 1321 (1987).
[CrossRef]

G. Y. Sirat, R. C. Chevallier, A. D. Maruani, “Grey Level Neural Networks,” Appl. Opt.28, 414 (1989), submitted;A. J. Noest, “Discrete-State Phasor Neural Networks,” Phys. Rev. A, submitted;A. J. Noest, “Associative Memory in Sparse Phasor,” Europhys. Lett., in press.
[CrossRef] [PubMed]

Personnaz, L.

L. Personnaz, I. Guyon, G. Dreyfus, “Information Storage and Retrieval in Spin-Like Neural Networks,” J. Phys. Paris Lett. 46, L359 (1985);L. Personnaz, I. Guyon, G. Dreyfus, G. Toulouse, “A Biologically Constrained Learning Mechanism in Networks of Formal Neurons,” J. Stat. Phys. 43, 411 (1986).
[CrossRef]

Psaltis, D.

D. Psaltis et al., “Shift Invariance in Optical Associative Memories,” Proc. Soc. Photo-Opt. Instrum. Eng.625, (1986).

D. Psaltis, N. Farhat, “Optical Information Processing Based on an Associative-Memory Model of Neural Nets with Thresholding and Feedback,” Opt. Lett. 10, 98 (1985).
[CrossRef] [PubMed]

N. Farhat, D. Psaltis, “Architectures for Optical Implementation of 2-D Content Addressable Memories,” in Technical Digest of the Annual Meeting (Optical Society of America, Washington, DC, 1985), paperWT3.

S. Venkatesh, D. Psaltis, “Information Storage and Retrieval in the Associative Nets,” IEEE Trans. Inf. Theory, submitted.

Rhodes, W. T.

W. T. Rhodes, “Acousto-Optic Devices Applied to Image Processing,” Proc. Soc. Photo-Opt. Instrum. Eng. 180, 143 (1979).

Sirat, G.

N. Ben-Yosef, G. Sirat, “Signal to Noise Ratio in a Frequency Multiplexing Imager,” IEEE J. Quantum Electron. QE-20, (1984).

Sirat, G. Y.

A. D. Maruani, R. C. Chevallier, G. Y. Sirat, “Information Retrieval in Neural Networks. Eigenproblems in Neural Networks,” Rev. Phys. Appl. 22, 1321 (1987).
[CrossRef]

G. Y. Sirat, R. C. Chevallier, A. D. Maruani, “Grey Level Neural Networks,” Appl. Opt.28, 414 (1989), submitted;A. J. Noest, “Discrete-State Phasor Neural Networks,” Phys. Rev. A, submitted;A. J. Noest, “Associative Memory in Sparse Phasor,” Europhys. Lett., in press.
[CrossRef] [PubMed]

G. Y. Sirat, “Imaging System Based on Frequency Multiplexing Utilizing the Piezo-Electric-Elasto-Optic Effect,” Ph.D. Thesis, Hebrew University (Mar.1983);N. Ben-Yosef, G. Sirat, “Real-Time Spatial Filtering Utilizing the Piezo-ElectricElasto-Optic Effect,” Opt. Acta 29, 419 (1982).
[CrossRef]

Venkatesh, S.

S. Venkatesh, D. Psaltis, “Information Storage and Retrieval in the Associative Nets,” IEEE Trans. Inf. Theory, submitted.

British Pat. (1)

E. E. Fournier d’Albe, British Pat. 233, 746 (15 May1925).

IEEE J. Quantum Electron. (1)

N. Ben-Yosef, G. Sirat, “Signal to Noise Ratio in a Frequency Multiplexing Imager,” IEEE J. Quantum Electron. QE-20, (1984).

J. Phys. Paris Lett. (1)

L. Personnaz, I. Guyon, G. Dreyfus, “Information Storage and Retrieval in Spin-Like Neural Networks,” J. Phys. Paris Lett. 46, L359 (1985);L. Personnaz, I. Guyon, G. Dreyfus, G. Toulouse, “A Biologically Constrained Learning Mechanism in Networks of Formal Neurons,” J. Stat. Phys. 43, 411 (1986).
[CrossRef]

Opt. Lett. (1)

Proc. Natl. Acad. Sci. (1)

J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. 79, 2254 (1982); “Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
[CrossRef]

Proc. Soc. Photo-Opt. Instrum. Eng. (2)

W. T. Rhodes, “Acousto-Optic Devices Applied to Image Processing,” Proc. Soc. Photo-Opt. Instrum. Eng. 180, 143 (1979).

D. Psaltis et al., “Shift Invariance in Optical Associative Memories,” Proc. Soc. Photo-Opt. Instrum. Eng.625, (1986).

Rev. Phys. Appl. (1)

A. D. Maruani, R. C. Chevallier, G. Y. Sirat, “Information Retrieval in Neural Networks. Eigenproblems in Neural Networks,” Rev. Phys. Appl. 22, 1321 (1987).
[CrossRef]

Other (4)

G. Y. Sirat, R. C. Chevallier, A. D. Maruani, “Grey Level Neural Networks,” Appl. Opt.28, 414 (1989), submitted;A. J. Noest, “Discrete-State Phasor Neural Networks,” Phys. Rev. A, submitted;A. J. Noest, “Associative Memory in Sparse Phasor,” Europhys. Lett., in press.
[CrossRef] [PubMed]

S. Venkatesh, D. Psaltis, “Information Storage and Retrieval in the Associative Nets,” IEEE Trans. Inf. Theory, submitted.

G. Y. Sirat, “Imaging System Based on Frequency Multiplexing Utilizing the Piezo-Electric-Elasto-Optic Effect,” Ph.D. Thesis, Hebrew University (Mar.1983);N. Ben-Yosef, G. Sirat, “Real-Time Spatial Filtering Utilizing the Piezo-ElectricElasto-Optic Effect,” Opt. Acta 29, 419 (1982).
[CrossRef]

N. Farhat, D. Psaltis, “Architectures for Optical Implementation of 2-D Content Addressable Memories,” in Technical Digest of the Annual Meeting (Optical Society of America, Washington, DC, 1985), paperWT3.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

Principle of time multiplexed raster and frequency multiplexed raster mapping.

Fig. 2
Fig. 2

Effects of HF and LF operators.

Fig. 3
Fig. 3

Principle of an optical implementation of the FMR scheme.

Fig. 4
Fig. 4

High frequency input module.

Fig. 5
Fig. 5

System.

Fig. 6
Fig. 6

FMR coding, raster mapping, and Fourier transform.

Fig. 7
Fig. 7

Above, The FMR coding as a sampled Fourier transform; below, blow up of off-line sampling.

Equations (67)

Equations on this page are rendered with MathJax. Learn more.

V ^ i = j   =   0 N     1 T i j V j ( μ ) .
R a b c d = m   =   1 M V a c ( m ) V b d ( m ) .
V ^ b d = a c R a b c d X a c .
R ( I , J ) = FMR ( R abcd ) ,
R ( I , J ) = FMR ( R a b c d ) = a   =   0 N     1   b   =   0 N     1   c   =   0 N     1   d   =   0 N     1 R a b c d × exp μ [ ( a + N b ) I + ( c + N d ) J ]  for  0 I , J N 2 1 ,
X E F = HF ( X e f ) ,
{ X E F = N 4 X e f   if E = N e  and  F = N f , X E F = 0   otherwise ,
X E F = LF ( X e f ) ,
{ X E F = N 4 X e f   if  0 E , F N 1 , X E F = 0   otherwise  .
V ^ b d = ILF ( DFT { FMR  [ R a b c d ]  @ DFT  [ HF ( X a c ) ] } ) ,
V ^ b d = IHF ( DFT { FMR [ R a b c d ]  @  DFT  [ LF ( X a c ) ] } ) ,
A ( I ) = FMR ( A i j ) = i   =   0 N     1   j   =   0 N     1 A ij  exp [ μ ( i + N j ) I ]  ,
A ( I ) = N 2 I   =   0 N 2     1 A ( I )  exp [ μ Π ]  for  0 I N 2 1.
A ( I ) = i   =   0 N     1   j   =   0 N     1 A ij δ [ I ( i + N j ) ]  ,
A ( 0 ) = A 00 , A ( 1 ) = A 10 , A ( 2 ) = A 20 , A ( N ) = A 01 , A ( N + 1 ) = A 11 , A ( N 2 1 ) = A N     1 N     1  .
A ( i , j ) = i = 0 N 1 j = 0 N 1 A i j  exp N μ ( i i + j j ) .
{ A i j = A i j if  0 i , j N 1 , A i j = 0   otherwise,
A ( i , j ) = i   =   0 N 2 1 j   =   0 N 2 1 A i j  exp μ ( i i + j j ) ,
A ( i , j ) = i   =   0 N     1 j   =   0 N     1 A i j  exp μ ( i i + j j ) .
{ i = I 0 I N 2 1 , j = N I mod  . N 2 ,
X N k = N 2 x k for  k = 0 , 1 , , N 1 ,
X k = N 2 x k for  k = 0 , 1 , , N 1 ,
X ( K ) = k   =   0 N     1 x k exp μ N k K for K = 0 , 1 , , N 2 1 ,
X ( K ) = k   =   0 N     1 x k  exp  μ k K  for  K = 0 , 1 , , N 2 1.
X ( K ) = k   =   0 N 2     1 X k  exp  μ k K ,
X ( K ) = k   =   0 N 2     1 X k  exp  μ k K ,
B ( I ) = A ( I ) x ( I ) ,
B ( I ) = A ( I ) x ( I ) .
C ( I ) = A ( I ) x ( I ) x ( I ) .
B ( I ) = i   =   0 N     1 j   =   0 N     1 k   =   0 N     1 A i j x k  exp μ [ i + ( j k ) N ] I ,
B ( I ) = i   =   0 N     1 α   =   0 N     1 k   =   0 N     1 A j ( k   +   α ) x k  exp μ [ i + N α ] I
B i α = k   =   0 N     1 A i ( k   +   α ) x k  ,
B ( I ) = i   =   0 N     1 α   =   0 N     1 B i α  exp μ [ i + N α ] I ,
B α j = i   =   0 N     1 A ( i   +   α ) j x i ,
B ( I ) = j   =   0 N     1 α   =   0 N     1 B α j  exp μ [ α + N j ] I ,
C α β = i   =   0 N     1 j   =   0 N     1 A ( i   +   α ) ( j   +   β ) x i x j ,
C ( I ) = β   =   0 N     1 α   =   0 N     1 c β α  exp μ [ β + N α ] I ,
c α = DE [ C β α ] = C α α ;
R ( I , J ) = a   =   0 N     i b   =   0 N     1 c   =   0 N     1 d   =   0 N     1 R a b c d × exp μ [ ( a + N b ) I + ( c + N d ) J ]
{ X E F = N 4 X e f      if E = N e and F = N f , X E F = 0    otherwise ,
{ X E F = N 4 X e f   if  0 E , F N 1 , X E F = 0   otherwise,
X ( E , F ) = E   =   0 N 2     1 F   =   0 N 2     1 X E F  exp μ ( E E + F F ) ,
X ( E , F ) = e f X e f  exp μ N ( e E + f F ) ,
X ( E , F ) = E   =   0 N 2     1 F   =   0 N 2     1 X E F exp μ ( E E + F F ) ,
X ( E , F ) = e f X e f exp μ ( e E + f F )  .
U ( I , J ) = R ( I , J ) X ( I , J )
U ( I , J ) = R ( I , J ) X ( I , J ) ,
C ( I , J ) = R ( I , J ) X ( I , J ) X ( I , J ) ,
U a α c γ = e   =   0 N     1 f   =   0 N     1 R a  ( e   +   α ) c ( f   +   γ ) X e f ,
U ( I , J ) = a   =   0 N     1 α   =   0 N     1 c   =   0 N     1 γ   =   0 N     1 U a α c γ ×  exp μ [ ( a + N α ) I + ( c + N γ ) J ]  ,
U I J = 1 N 4 I   =   0 N 2     1 J   =   0 N 2     1 U ( I , J )  exp μ ( I I + J J ) .
U I J = a   =   0 N     1 α   =   0 N     1 c   =   0 N     1 γ = 0 N 1 U a α c γ δ [ I ( a + N α ) ] δ [ J ( c + N γ ) ] .
a = I C = J α = γ = 0 ,
U I J = U a 0 c 0 .
U I J = e   =   0 N     1 f   =   0 N     1 R a e c f X e f .
U I J = U a 0 c 0 = U a c = a   =   0 N     1 c   =   0 N     1 R a b c d X a c  for  I = b and J = d ,
U β b δ d = e   =   0 N     1 f   =   0 N     1 R ( e   +   β ) b ( f   +   δ ) d X e f .
U ( I , J ) = R ( I , J ) X ( I , J ) ,
U ( J , J ) = β   =   0 N     1 b   =   0 N     1 δ   =   0 N     1 d   =   0 N     1 U β b δ d ×  exp μ [ ( β + N b ) I + ( δ + N d ) J ]  ,
U I J = 1 N 4 I   =   0 N 2     1 J   =   0 N 2     1 U ( I  , J )  exp μ [ I I + J J ] ,
U I J = β   =   0 N     1 b   =   0 N     1 δ   =   0 N     1 d   =   0 N     1 U β b δ d δ [ β I + N b ] δ [ N d + δ J ] .
b = p d = q β = δ = 0 ,
U I J = a = 0 N 1 c = 0 N 1 R a b c d X a c for I = N b  and  J = N d .
C α ε γ ϕ = e   =   0 N     1 f   =   0 N     1 a   =   0 N     1 c   =   0 N     1 R ( a   +   α ) ( e   +   ε ) ( c   +   γ ) ( f   +   ϕ ) X e f X a c ,
C ( I , J ) = α   =   0 N     1 ε   =   0 N     1 γ = 0 N     1 ϕ   =   0 N     1 c α ε γ ϕ ×  exp μ [ ( α + N ε ) I + ( γ + N ϕ ) J ] ;
C I J = 1 N 4 I   =   0 N 2     1 J   =   0 N 2     1 C ( I , J )  exp μ ( I I + J J ) .
C α γ = DE  [ C α ε γ ϕ ] = C α α γ γ ;

Metrics