Abstract

This paper presents a comparison between the field of artificial neural network and the field of image processing and pattern recognition. It shows that some of the adaptive processing algorithms for pattern recognition and image processing, in terms of neural networks, can be seen as adaptive heteroassociative and autoassociative memories, respectively. The similarities and differences between these two fields are addressed.

© 1989 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition,” IEEE Trans. Syst. Man Cybern. SMC-13, 826 (1983).
    [CrossRef]
  2. D. E. Rumelhart, D. Zipser, “Feature Discovery by Competitive Learning,” Cognitive Sci. 9, 75 (1985).
    [CrossRef]
  3. G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vis. Graph. Image Proc. 37, 54 (1987).
    [CrossRef]
  4. Y. Z. Zhou, R. Chellappa, B. K. Jenkins, “A Novel Approach to Image Restoration Based on a Neural Network,” Presented at IEEE First Annual International Conference on Neural Network, San Diego, CA., June 1987.
  5. R. P. Lippmann, “An Introduction to Computing with Neural Nets,” IEEE Trans. Acoust. Speech Signal Process. ASSP Magazine, 4, 4 (1987).
  6. P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing, Snowbird, UT, AIP Conf. Proc. 151 (1986).
  7. K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, New York, 1972), Chap. 7.
  8. Z-H. Gu, J. R. Leger, S. H. Lee, “Optical Implementation of the Least-Square Linear Mapping Technique for Image Classification,” J. Opt. Soc. Am. 72, 787 (1982).
    [CrossRef]
  9. J. R. Leger, S. H. Lee, “Image Classification by An Optical Implementation of the Fukunaga-Koontz Transformation,” J. Opt. Soc. Am. 72, 556 (1982).
    [CrossRef]
  10. A. Papoulis, “A New Algorithm in Spectral Analysis and Band-Limited Extrapolation,” IEEE Trans. Circuit Syst. CAS-22, 735 (1976).
  11. J. A. Cadzow, “An Extrapolation Procedure for Band-Limited Signal,” IEEE Trans. Acoust. Speech Signal Process. ASSP-27, 4 (1979).
    [CrossRef]
  12. A. V. Oppenheim, M. H. Hayes, J. S. Lim, “Iterative Procedures for Signal Reconstruction from Phase,” Proc. Soc. Photo-Opt. Instrum. Eng. 231, 121 (1980).
  13. D. Youla, “Generalized Image Restoration by the Method of Alternating Orthogonal Projections,” IEEE Trans. Circuit Syst. CAS-25, 694 (1978).
    [CrossRef]
  14. M. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
    [CrossRef]
  15. N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469 (1985).
    [CrossRef] [PubMed]
  16. J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
    [CrossRef] [PubMed]
  17. A. D. Fisher, C. L. Giles, “Optical Adaptive Associative Computer Architectures,” in Proceedings, IEEE COMPCON (Spring1985), pp. 342–344.
  18. R. Hecht-Nielsen, “Performance Limits of Optical, Electro-optical, and Electronic Neurocomputers,” Proc. Soc. Photo-Opt. Instrum. Eng. 634, 277 (1986).
  19. M. Takeda, J. W. Goodman, “Neural Network for Computation: Number Representation and Programming Complexity,” Appl. Opt. 25, 3033 (1986).
    [CrossRef] [PubMed]
  20. T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, Berlin, 1984), Chap. 5.

1987 (2)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vis. Graph. Image Proc. 37, 54 (1987).
[CrossRef]

R. P. Lippmann, “An Introduction to Computing with Neural Nets,” IEEE Trans. Acoust. Speech Signal Process. ASSP Magazine, 4, 4 (1987).

1986 (2)

R. Hecht-Nielsen, “Performance Limits of Optical, Electro-optical, and Electronic Neurocomputers,” Proc. Soc. Photo-Opt. Instrum. Eng. 634, 277 (1986).

M. Takeda, J. W. Goodman, “Neural Network for Computation: Number Representation and Programming Complexity,” Appl. Opt. 25, 3033 (1986).
[CrossRef] [PubMed]

1985 (3)

A. D. Fisher, C. L. Giles, “Optical Adaptive Associative Computer Architectures,” in Proceedings, IEEE COMPCON (Spring1985), pp. 342–344.

N. H. Farhat, D. Psaltis, A. Prata, E. Paek, “Optical Implementation of the Hopfield Model,” Appl. Opt. 24, 1469 (1985).
[CrossRef] [PubMed]

D. E. Rumelhart, D. Zipser, “Feature Discovery by Competitive Learning,” Cognitive Sci. 9, 75 (1985).
[CrossRef]

1984 (1)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
[CrossRef] [PubMed]

1983 (2)

M. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition,” IEEE Trans. Syst. Man Cybern. SMC-13, 826 (1983).
[CrossRef]

1982 (2)

1980 (1)

A. V. Oppenheim, M. H. Hayes, J. S. Lim, “Iterative Procedures for Signal Reconstruction from Phase,” Proc. Soc. Photo-Opt. Instrum. Eng. 231, 121 (1980).

1979 (1)

J. A. Cadzow, “An Extrapolation Procedure for Band-Limited Signal,” IEEE Trans. Acoust. Speech Signal Process. ASSP-27, 4 (1979).
[CrossRef]

1978 (1)

D. Youla, “Generalized Image Restoration by the Method of Alternating Orthogonal Projections,” IEEE Trans. Circuit Syst. CAS-25, 694 (1978).
[CrossRef]

1976 (1)

A. Papoulis, “A New Algorithm in Spectral Analysis and Band-Limited Extrapolation,” IEEE Trans. Circuit Syst. CAS-22, 735 (1976).

Cadzow, J. A.

J. A. Cadzow, “An Extrapolation Procedure for Band-Limited Signal,” IEEE Trans. Acoust. Speech Signal Process. ASSP-27, 4 (1979).
[CrossRef]

Carpenter, G. A.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vis. Graph. Image Proc. 37, 54 (1987).
[CrossRef]

Chellappa, R.

Y. Z. Zhou, R. Chellappa, B. K. Jenkins, “A Novel Approach to Image Restoration Based on a Neural Network,” Presented at IEEE First Annual International Conference on Neural Network, San Diego, CA., June 1987.

Cohen, M.

M. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

Farhat, N. H.

Fisher, A. D.

A. D. Fisher, C. L. Giles, “Optical Adaptive Associative Computer Architectures,” in Proceedings, IEEE COMPCON (Spring1985), pp. 342–344.

Fukunaga, K.

K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, New York, 1972), Chap. 7.

Fukushima, K.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition,” IEEE Trans. Syst. Man Cybern. SMC-13, 826 (1983).
[CrossRef]

Giles, C. L.

A. D. Fisher, C. L. Giles, “Optical Adaptive Associative Computer Architectures,” in Proceedings, IEEE COMPCON (Spring1985), pp. 342–344.

Goodman, J. W.

Grant, P. M.

P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing, Snowbird, UT, AIP Conf. Proc. 151 (1986).

Grossberg, S.

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vis. Graph. Image Proc. 37, 54 (1987).
[CrossRef]

M. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

Gu, Z-H.

Hayes, M. H.

A. V. Oppenheim, M. H. Hayes, J. S. Lim, “Iterative Procedures for Signal Reconstruction from Phase,” Proc. Soc. Photo-Opt. Instrum. Eng. 231, 121 (1980).

Hecht-Nielsen, R.

R. Hecht-Nielsen, “Performance Limits of Optical, Electro-optical, and Electronic Neurocomputers,” Proc. Soc. Photo-Opt. Instrum. Eng. 634, 277 (1986).

Hopfield, J. J.

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
[CrossRef] [PubMed]

Ito, T.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition,” IEEE Trans. Syst. Man Cybern. SMC-13, 826 (1983).
[CrossRef]

Jenkins, B. K.

Y. Z. Zhou, R. Chellappa, B. K. Jenkins, “A Novel Approach to Image Restoration Based on a Neural Network,” Presented at IEEE First Annual International Conference on Neural Network, San Diego, CA., June 1987.

Kohonen, T.

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, Berlin, 1984), Chap. 5.

Lee, S. H.

Leger, J. R.

Lim, J. S.

A. V. Oppenheim, M. H. Hayes, J. S. Lim, “Iterative Procedures for Signal Reconstruction from Phase,” Proc. Soc. Photo-Opt. Instrum. Eng. 231, 121 (1980).

Lippmann, R. P.

R. P. Lippmann, “An Introduction to Computing with Neural Nets,” IEEE Trans. Acoust. Speech Signal Process. ASSP Magazine, 4, 4 (1987).

Miyake, S.

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition,” IEEE Trans. Syst. Man Cybern. SMC-13, 826 (1983).
[CrossRef]

Oppenheim, A. V.

A. V. Oppenheim, M. H. Hayes, J. S. Lim, “Iterative Procedures for Signal Reconstruction from Phase,” Proc. Soc. Photo-Opt. Instrum. Eng. 231, 121 (1980).

Paek, E.

Papoulis, A.

A. Papoulis, “A New Algorithm in Spectral Analysis and Band-Limited Extrapolation,” IEEE Trans. Circuit Syst. CAS-22, 735 (1976).

Prata, A.

Psaltis, D.

Rumelhart, D. E.

D. E. Rumelhart, D. Zipser, “Feature Discovery by Competitive Learning,” Cognitive Sci. 9, 75 (1985).
[CrossRef]

Sage, J. P.

P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing, Snowbird, UT, AIP Conf. Proc. 151 (1986).

Takeda, M.

Youla, D.

D. Youla, “Generalized Image Restoration by the Method of Alternating Orthogonal Projections,” IEEE Trans. Circuit Syst. CAS-25, 694 (1978).
[CrossRef]

Zhou, Y. Z.

Y. Z. Zhou, R. Chellappa, B. K. Jenkins, “A Novel Approach to Image Restoration Based on a Neural Network,” Presented at IEEE First Annual International Conference on Neural Network, San Diego, CA., June 1987.

Zipser, D.

D. E. Rumelhart, D. Zipser, “Feature Discovery by Competitive Learning,” Cognitive Sci. 9, 75 (1985).
[CrossRef]

Appl. Opt. (2)

Cognitive Sci. (1)

D. E. Rumelhart, D. Zipser, “Feature Discovery by Competitive Learning,” Cognitive Sci. 9, 75 (1985).
[CrossRef]

Comput. Vis. Graph. Image Proc. (1)

G. A. Carpenter, S. Grossberg, “A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine,” Comput. Vis. Graph. Image Proc. 37, 54 (1987).
[CrossRef]

IEEE Trans. Acoust. Speech Signal Process. (1)

J. A. Cadzow, “An Extrapolation Procedure for Band-Limited Signal,” IEEE Trans. Acoust. Speech Signal Process. ASSP-27, 4 (1979).
[CrossRef]

IEEE Trans. Acoust. Speech Signal Process. ASSP Magazine (1)

R. P. Lippmann, “An Introduction to Computing with Neural Nets,” IEEE Trans. Acoust. Speech Signal Process. ASSP Magazine, 4, 4 (1987).

IEEE Trans. Circuit Syst. (2)

A. Papoulis, “A New Algorithm in Spectral Analysis and Band-Limited Extrapolation,” IEEE Trans. Circuit Syst. CAS-22, 735 (1976).

D. Youla, “Generalized Image Restoration by the Method of Alternating Orthogonal Projections,” IEEE Trans. Circuit Syst. CAS-25, 694 (1978).
[CrossRef]

IEEE Trans. Syst. Man Cybern. (2)

M. Cohen, S. Grossberg, “Absolute Stability of Global Pattern Formation and Parallel Memory Storage by Competitive Neural Networks,” IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).
[CrossRef]

K. Fukushima, S. Miyake, T. Ito, “Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition,” IEEE Trans. Syst. Man Cybern. SMC-13, 826 (1983).
[CrossRef]

J. Opt. Soc. Am. (2)

Proc. Natl. Acad. Sci. (1)

J. J. Hopfield, “Neurons with Graded Response have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl. Acad. Sci. 81, 3088 (1984).
[CrossRef] [PubMed]

Proc. Soc. Photo-Opt. Instrum. Eng. (2)

A. V. Oppenheim, M. H. Hayes, J. S. Lim, “Iterative Procedures for Signal Reconstruction from Phase,” Proc. Soc. Photo-Opt. Instrum. Eng. 231, 121 (1980).

R. Hecht-Nielsen, “Performance Limits of Optical, Electro-optical, and Electronic Neurocomputers,” Proc. Soc. Photo-Opt. Instrum. Eng. 634, 277 (1986).

Proceedings, IEEE COMPCON (1)

A. D. Fisher, C. L. Giles, “Optical Adaptive Associative Computer Architectures,” in Proceedings, IEEE COMPCON (Spring1985), pp. 342–344.

Other (4)

T. Kohonen, Self-Organization and Associative Memory (Springer-Verlag, Berlin, 1984), Chap. 5.

P. M. Grant, J. P. Sage, “A Comparison of Neural Network and Matched Filter Processing for Detecting Lines in Images,” in Neural Networks for Computing, Snowbird, UT, AIP Conf. Proc. 151 (1986).

K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, New York, 1972), Chap. 7.

Y. Z. Zhou, R. Chellappa, B. K. Jenkins, “A Novel Approach to Image Restoration Based on a Neural Network,” Presented at IEEE First Annual International Conference on Neural Network, San Diego, CA., June 1987.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Equations (11)

Equations on this page are rendered with MathJax. Learn more.

y = f ( x ) ,
f k + 1 = f k - [ f k ( x k ) - Γ ( x k ] G ,
y = Θ ( Mx ) , x R n , y R p ,
M k + 1 = M k - [ Θ ( M k x k ) - Γ ( x k ) ] G k ,
y = H x + n ,
x ^ k + 1 = y + Q a P a x ^ k , x ^ 1 = y ,
lim k x ^ k = x .
E k = x ^ k - x 2 ,
x ^ k + 1 = Φ x 0 + Λ Ω x ^ k ,             x ^ 1 = x 0 ,
lim k x ^ k = x .
E k = 1 2 i = 1 N j = 1 N T i j V i k V j k - i = 1 N I i V i k .

Metrics