Abstract

This paper proposes a parallel distributed processing model with local space-invariant interconnections, which is more readily implemented by optics and is able to classify patterns correctly, even if they have been shifted or distorted. Error backpropagation is used as a training algorithm. Computer simulation results presented indicate that the processing is effective and the network can deal with the shifted or distorted patterns. Moreover, the optical implementation architecture using matched filters for the model is discussed.

© 1990 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. Q. Tian, Y. Fainman, Z. H. Gu, S. H. Lee, “Comparison of Statistical Pattern-Recognition Algorithms for Hybrid Processing. I. Linear-Mapping Algorithms,” J. Opt. Soc. Am. A 5, 1655–1669 (1988).
    [CrossRef]
  2. M. Minsky, S. Papert, Perceptrons (MIT Press, Cambridge, MA, 1969).
  3. B. Widrow, R. Winter, “Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition,” Computer 21, 25–39 (1988).
    [CrossRef]
  4. T. Kohonen, “The “Neural” Phonetic Typewriter,” Computer 21, 11–22 (1988).
    [CrossRef]
  5. G. A. Carpenter, S. Grossberg, “The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network,” Computer 21, 77–88 (1988).
    [CrossRef]
  6. D. E. Rumelhart et al., Parallel Distributed Processing (MIT Press, Cambridge, MA, 1986).
  7. K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 86–97 (1987).
  8. A. D. Fisher, W. L. Lippincott, J. N. Lee, “Optical Implementations of Associative Networks with Versatile Adaptive Learning Capabilities,” Appl. Opt. 26, 5039–5054 (1987).
    [CrossRef] [PubMed]
  9. M. R. Feldman, S. C. Esener, C. C. Guest, S. H. Lee, “Comparison Between Optical and Electrical Interconnects Based on Power and Speed Considerations,” Appl. Opt. 27, 1742–1751 (1988).
    [CrossRef] [PubMed]
  10. D. Casasent, “Optical Associative Processors for Visual Perception,” Proc. Soc. Photo-Opt. Instrum. Eng. 882, 47 (1986).
  11. K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Patter Recognition,” IEEE Trans. Sys. Man Cybernet. SMC-13, 826–834 (1983).
    [CrossRef]
  12. D. H. Hubel, T. N. Wiesel, “Receptive Fields, Binocular Interaction and Functional Architecture in Cat’s Visual Cortex,” J. Physiol. London 160, 106–154 (1962).
  13. D. H. Hubel, T. N. Wiesel, “Receptive Fields and Functional Architecture in Two Nonstriated Visual Area (18 and 19) of the Cat,” J. Neurophysiol. 28, 229–289 (1965).
    [PubMed]
  14. J. W. Goodman, Introduction to Fourier Optics (Wiley, New York, 1968).

1988

Q. Tian, Y. Fainman, Z. H. Gu, S. H. Lee, “Comparison of Statistical Pattern-Recognition Algorithms for Hybrid Processing. I. Linear-Mapping Algorithms,” J. Opt. Soc. Am. A 5, 1655–1669 (1988).
[CrossRef]

B. Widrow, R. Winter, “Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition,” Computer 21, 25–39 (1988).
[CrossRef]

T. Kohonen, “The “Neural” Phonetic Typewriter,” Computer 21, 11–22 (1988).
[CrossRef]

G. A. Carpenter, S. Grossberg, “The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network,” Computer 21, 77–88 (1988).
[CrossRef]

M. R. Feldman, S. C. Esener, C. C. Guest, S. H. Lee, “Comparison Between Optical and Electrical Interconnects Based on Power and Speed Considerations,” Appl. Opt. 27, 1742–1751 (1988).
[CrossRef] [PubMed]

1987

1986

D. Casasent, “Optical Associative Processors for Visual Perception,” Proc. Soc. Photo-Opt. Instrum. Eng. 882, 47 (1986).

1983

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Patter Recognition,” IEEE Trans. Sys. Man Cybernet. SMC-13, 826–834 (1983).
[CrossRef]

1965

D. H. Hubel, T. N. Wiesel, “Receptive Fields and Functional Architecture in Two Nonstriated Visual Area (18 and 19) of the Cat,” J. Neurophysiol. 28, 229–289 (1965).
[PubMed]

1962

D. H. Hubel, T. N. Wiesel, “Receptive Fields, Binocular Interaction and Functional Architecture in Cat’s Visual Cortex,” J. Physiol. London 160, 106–154 (1962).

Carpenter, G. A.

G. A. Carpenter, S. Grossberg, “The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network,” Computer 21, 77–88 (1988).
[CrossRef]

Casasent, D.

D. Casasent, “Optical Associative Processors for Visual Perception,” Proc. Soc. Photo-Opt. Instrum. Eng. 882, 47 (1986).

Esener, S. C.

Fainman, Y.

Feldman, M. R.

Fisher, A. D.

Fukushima, K.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Patter Recognition,” IEEE Trans. Sys. Man Cybernet. SMC-13, 826–834 (1983).
[CrossRef]

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics (Wiley, New York, 1968).

Grossberg, S.

G. A. Carpenter, S. Grossberg, “The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network,” Computer 21, 77–88 (1988).
[CrossRef]

Gu, Z. H.

Guest, C. C.

Hubel, D. H.

D. H. Hubel, T. N. Wiesel, “Receptive Fields and Functional Architecture in Two Nonstriated Visual Area (18 and 19) of the Cat,” J. Neurophysiol. 28, 229–289 (1965).
[PubMed]

D. H. Hubel, T. N. Wiesel, “Receptive Fields, Binocular Interaction and Functional Architecture in Cat’s Visual Cortex,” J. Physiol. London 160, 106–154 (1962).

Ito, T.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Patter Recognition,” IEEE Trans. Sys. Man Cybernet. SMC-13, 826–834 (1983).
[CrossRef]

Kohonen, T.

T. Kohonen, “The “Neural” Phonetic Typewriter,” Computer 21, 11–22 (1988).
[CrossRef]

Lee, J. N.

Lee, S. H.

Lippincott, W. L.

Minsky, M.

M. Minsky, S. Papert, Perceptrons (MIT Press, Cambridge, MA, 1969).

Miyake, S.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Patter Recognition,” IEEE Trans. Sys. Man Cybernet. SMC-13, 826–834 (1983).
[CrossRef]

Papert, S.

M. Minsky, S. Papert, Perceptrons (MIT Press, Cambridge, MA, 1969).

Psaltis, D.

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 86–97 (1987).

Rumelhart, D. E.

D. E. Rumelhart et al., Parallel Distributed Processing (MIT Press, Cambridge, MA, 1986).

Tian, Q.

Wagner, K.

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 86–97 (1987).

Widrow, B.

B. Widrow, R. Winter, “Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition,” Computer 21, 25–39 (1988).
[CrossRef]

Wiesel, T. N.

D. H. Hubel, T. N. Wiesel, “Receptive Fields and Functional Architecture in Two Nonstriated Visual Area (18 and 19) of the Cat,” J. Neurophysiol. 28, 229–289 (1965).
[PubMed]

D. H. Hubel, T. N. Wiesel, “Receptive Fields, Binocular Interaction and Functional Architecture in Cat’s Visual Cortex,” J. Physiol. London 160, 106–154 (1962).

Winter, R.

B. Widrow, R. Winter, “Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition,” Computer 21, 25–39 (1988).
[CrossRef]

Appl. Opt.

Computer

B. Widrow, R. Winter, “Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition,” Computer 21, 25–39 (1988).
[CrossRef]

T. Kohonen, “The “Neural” Phonetic Typewriter,” Computer 21, 11–22 (1988).
[CrossRef]

G. A. Carpenter, S. Grossberg, “The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network,” Computer 21, 77–88 (1988).
[CrossRef]

IEEE Trans. Sys. Man Cybernet.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Patter Recognition,” IEEE Trans. Sys. Man Cybernet. SMC-13, 826–834 (1983).
[CrossRef]

J. Neurophysiol.

D. H. Hubel, T. N. Wiesel, “Receptive Fields and Functional Architecture in Two Nonstriated Visual Area (18 and 19) of the Cat,” J. Neurophysiol. 28, 229–289 (1965).
[PubMed]

J. Opt. Soc. Am. A

J. Physiol. London

D. H. Hubel, T. N. Wiesel, “Receptive Fields, Binocular Interaction and Functional Architecture in Cat’s Visual Cortex,” J. Physiol. London 160, 106–154 (1962).

Proc. Soc. Photo-Opt. Instrum. Eng.

K. Wagner, D. Psaltis, “Multilayer Optical Learning Networks,” Proc. Soc. Photo-Opt. Instrum. Eng. 752, 86–97 (1987).

D. Casasent, “Optical Associative Processors for Visual Perception,” Proc. Soc. Photo-Opt. Instrum. Eng. 882, 47 (1986).

Other

M. Minsky, S. Papert, Perceptrons (MIT Press, Cambridge, MA, 1969).

D. E. Rumelhart et al., Parallel Distributed Processing (MIT Press, Cambridge, MA, 1986).

J. W. Goodman, Introduction to Fourier Optics (Wiley, New York, 1968).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

PDP model with local space-invariant interconnection and constrained receptive fields.

Fig. 2
Fig. 2

Space-invariant interconnection.

Fig. 3
Fig. 3

Training set.

Fig. 4
Fig. 4

Responses of each layer to the input pattern a.

Fig. 5
Fig. 5

Inner products between each pair of the connection patterns in (a) the fan-out field and (b) the fan-in field.

Fig. 6
Fig. 6

Responses of the system measured at every 5°. Unit i denotes each decision unit in the last layer: (a) to the class of a; (b) to the class of b; (c) to the class of c; (d) to the class of d.

Fig. 7
Fig. 7

Some examples of the test patterns which are recognized correctly. The last two patterns are deformed by the impulse noises whose levels are bright and Gaussian variable, respectively. To make it clear, the penultimate one is displayed reversely.

Fig. 8
Fig. 8

Some examples of the test patterns which are not recognized correctly by our model.

Fig. 9
Fig. 9

Optical architecture: L, Fourier transform lenses; H, hologram; SLM, spatial light modulators; LA, Fourier transform lens arrays; HA, hologram array; M, masks; PDA, photodiode array.

Fig. 10
Fig. 10

Getting the net input of the second layer by taking the cross correlation between the input and connection patterns without crosstalk. The symbol ⊗ denotes operation of the cross correlation.

Tables (1)

Tables Icon

Table I Structure of the Network Used In Our Simulation

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

w ( i , j , x , y ) = w ( 1 - x ; j - y ) ,
net p l + 1 ( i , j ) = p 0 m n w p 0 , p l ( m , n ) × o p 0 l ( i + m + s l , j + n + s l ) + b p l + 1
s l = ½ ( I l - I l + 1 - M l ) , o p l + 1 ( i , j ) = f [ net p l + 1 ( i , j ) ] ,
w p 0 , p L - 1 ( m , n ) = { w L - 1 if p 0 = p , 0 else ,
Δ w p 0 , p l ( m , n ) = η i j δ p l + 1 ( i , j ) × o p 0 l ( i + m + s l , j + n + s l ) ,
Δ b p l = η i j δ p l ( i , j ) ,
δ p 0 l ( i , j ) = f [ net p 0 l ( i , j ) ] p m n × [ δ p l + 1 ( i - m - s l , j - n - s l ) w p 0 , p l ( m , n ) ] ;
δ p L = f ( net p L ) ( t p - o p L ) ,
w 1 ( m , n ) = w 1 , 1 1 ( m - m 0 , n + n 0 ) + w 1 , 2 1 ( m + m 0 , n + n 0 ) + w 1 , 3 1 ( m - m 0 , n - n 0 ) + w 1 , 4 1 ( m + m 0 , n - n 0 ) ,
I ( x 2 , y 2 ) = | W 1 ( x 2 λ f , y 2 λ f ) + exp [ - i 2 π ( α x 2 + β y 2 ) ] | 2 , α = cos θ x λ ,             β = cos θ y λ ,
α > 1 λ f [ 3 ( I 1 + 2 M 1 ) 2 + I 1 ] ,             β = 0.
( α 1 , β 1 ) = ( α + m 0 λ f , β - n 0 λ f ) , ( α 2 , β 2 ) = ( α - m 0 λ f , β - n 0 λ f ) , ( α 3 , β 3 ) = ( α + m 0 λ f , β + n 0 λ f ) , ( α 4 , β 4 ) = ( α - m 0 λ f , β + n 0 λ f ) , α > 1 λ f [ 3 ( I 2 + 2 M 2 ) 2 + I 2 ] ,             β = 0 ,

Metrics