Abstract

The cascaded correlator architecture comprises a series of traditional linear correlators separated by nonlinear threshold functions, trained with neural-network techniques. We investigate the shift-invariant classification performance of cascaded correlators in comparison with optimum Bayes classifiers. Inputs are formulated as randomly generated sample members of known statistical class distributions. It is shown that when the separability of true and false classes is varied in both the first and the second orders, the two-stage cascaded correlator shows performance similar to that of the optimum quadratic Bayes classifier throughout the studied range. It is shown that this is due to the similar decision boundaries implemented by the two nonlinear classifiers.

© 2000 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, New York, 1972).
  2. A. B. VanderLugt, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
    [CrossRef]
  3. D. Casasent, “Unified synthetic discriminant function computation formulation,” Appl. Opt. 23, 1620–1627 (1984).
    [CrossRef]
  4. H. J. Caulfield, W. T. Maloney, “Improved discrimination in optical character recognition,” Appl. Opt. 8, 2354–2356 (1969).
    [CrossRef] [PubMed]
  5. B. V. K. Vijaya Kumar, “Minimum variance synthetic discriminant functions,” J. Opt. Soc. Am. A 3, 1579–1584 (1986).
    [CrossRef]
  6. B. Javidi, “Nonlinear joint power spectrum based optical correlation,” Appl. Opt. 28, 2358–2367 (1989).
    [CrossRef] [PubMed]
  7. S. Reed, J. Coupland, “Cascaded linear shift invariant processing to improve discrimination and noise tolerance in optical pattern recognition,” in Optical Pattern Recognition IX, D. P. Casasent, T. Chao, eds., Proc. SPIE3386, 272–283 (1998).
    [CrossRef]
  8. S. Reed, J. Coupland, “Rotation invariance considerations in cascaded linear shift invariant processing,” Asian J. Phys. 8, 421–429 (1999).
  9. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McLelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.
  10. S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
    [CrossRef] [PubMed]
  11. E. Barnard, E. C. Botha, “Back-propagation uses prior information efficiently,” IEEE Trans. Neural Networks 4, 794–802 (1993).
    [CrossRef]
  12. M. D. Richard, R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Comput. 3, 461–483 (1991).
    [CrossRef]
  13. F. Kanaya, S. Miyake, “Bayes statistical behaviour and valid generalization of pattern classifying neural networks,” IEEE Trans. Neural Networks 2, 471–475 (1991).
    [CrossRef]
  14. We have prepared another study entitled, “Cascaded linear shift-invariant processing in optical pattern recognition.”
  15. M. Soriano, C. Saloma, “Improved classification robustness for noisy cell images represented as principal-component projections in a hybrid recognition system,” Appl. Opt. 37, 3628–3638 (1998).
    [CrossRef]
  16. S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
    [CrossRef] [PubMed]
  17. H. L. Van Trees, Detection, Estimation, and Modulation Theory: Part I (Wiley, New York, 1968).

1999

S. Reed, J. Coupland, “Rotation invariance considerations in cascaded linear shift invariant processing,” Asian J. Phys. 8, 421–429 (1999).

1998

1995

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

1993

E. Barnard, E. C. Botha, “Back-propagation uses prior information efficiently,” IEEE Trans. Neural Networks 4, 794–802 (1993).
[CrossRef]

1991

M. D. Richard, R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Comput. 3, 461–483 (1991).
[CrossRef]

F. Kanaya, S. Miyake, “Bayes statistical behaviour and valid generalization of pattern classifying neural networks,” IEEE Trans. Neural Networks 2, 471–475 (1991).
[CrossRef]

1989

1986

1984

1983

S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef] [PubMed]

1969

1964

A. B. VanderLugt, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
[CrossRef]

Barnard, E.

E. Barnard, E. C. Botha, “Back-propagation uses prior information efficiently,” IEEE Trans. Neural Networks 4, 794–802 (1993).
[CrossRef]

Botha, E. C.

E. Barnard, E. C. Botha, “Back-propagation uses prior information efficiently,” IEEE Trans. Neural Networks 4, 794–802 (1993).
[CrossRef]

Casasent, D.

Caulfield, H. J.

Chien, M. V.

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

Coupland, J.

S. Reed, J. Coupland, “Rotation invariance considerations in cascaded linear shift invariant processing,” Asian J. Phys. 8, 421–429 (1999).

S. Reed, J. Coupland, “Cascaded linear shift invariant processing to improve discrimination and noise tolerance in optical pattern recognition,” in Optical Pattern Recognition IX, D. P. Casasent, T. Chao, eds., Proc. SPIE3386, 272–283 (1998).
[CrossRef]

Freedman, M. T.

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

Fukunaga, K.

K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, New York, 1972).

Gelatt, C. D.

S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef] [PubMed]

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McLelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Javidi, B.

Kanaya, F.

F. Kanaya, S. Miyake, “Bayes statistical behaviour and valid generalization of pattern classifying neural networks,” IEEE Trans. Neural Networks 2, 471–475 (1991).
[CrossRef]

Kirkpatrick, S.

S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef] [PubMed]

Lin, J.-S.

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

Lippmann, R. P.

M. D. Richard, R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Comput. 3, 461–483 (1991).
[CrossRef]

Lo, S.-C. B.

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

Lou, S.-L. A.

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

Maloney, W. T.

Miyake, S.

F. Kanaya, S. Miyake, “Bayes statistical behaviour and valid generalization of pattern classifying neural networks,” IEEE Trans. Neural Networks 2, 471–475 (1991).
[CrossRef]

Mun, S. K.

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

Reed, S.

S. Reed, J. Coupland, “Rotation invariance considerations in cascaded linear shift invariant processing,” Asian J. Phys. 8, 421–429 (1999).

S. Reed, J. Coupland, “Cascaded linear shift invariant processing to improve discrimination and noise tolerance in optical pattern recognition,” in Optical Pattern Recognition IX, D. P. Casasent, T. Chao, eds., Proc. SPIE3386, 272–283 (1998).
[CrossRef]

Richard, M. D.

M. D. Richard, R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Comput. 3, 461–483 (1991).
[CrossRef]

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McLelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Saloma, C.

Soriano, M.

Van Trees, H. L.

H. L. Van Trees, Detection, Estimation, and Modulation Theory: Part I (Wiley, New York, 1968).

VanderLugt, A. B.

A. B. VanderLugt, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
[CrossRef]

Vecchi, M. P.

S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef] [PubMed]

Vijaya Kumar, B. V. K.

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McLelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Appl. Opt.

Asian J. Phys.

S. Reed, J. Coupland, “Rotation invariance considerations in cascaded linear shift invariant processing,” Asian J. Phys. 8, 421–429 (1999).

IEEE Trans. Inf. Theory

A. B. VanderLugt, “Signal detection by complex matched spatial filtering,” IEEE Trans. Inf. Theory 10, 139–145 (1964).
[CrossRef]

IEEE Trans. Med. Imaging

S.-C. B. Lo, S.-L. A. Lou, J.-S. Lin, M. T. Freedman, M. V. Chien, S. K. Mun, “Artificial convolution neural network techniques and applications for lung nodule detection,” IEEE Trans. Med. Imaging 14, 711–718 (1995).
[CrossRef] [PubMed]

IEEE Trans. Neural Networks

E. Barnard, E. C. Botha, “Back-propagation uses prior information efficiently,” IEEE Trans. Neural Networks 4, 794–802 (1993).
[CrossRef]

F. Kanaya, S. Miyake, “Bayes statistical behaviour and valid generalization of pattern classifying neural networks,” IEEE Trans. Neural Networks 2, 471–475 (1991).
[CrossRef]

J. Opt. Soc. Am. A

Neural Comput.

M. D. Richard, R. P. Lippmann, “Neural network classifiers estimate Bayesian a posteriori probabilities,” Neural Comput. 3, 461–483 (1991).
[CrossRef]

Science

S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, “Optimization by simulated annealing,” Science 220, 671–680 (1983).
[CrossRef] [PubMed]

Other

H. L. Van Trees, Detection, Estimation, and Modulation Theory: Part I (Wiley, New York, 1968).

We have prepared another study entitled, “Cascaded linear shift-invariant processing in optical pattern recognition.”

K. Fukunaga, Introduction to Statistical Pattern Recognition (Academic, New York, 1972).

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McLelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

S. Reed, J. Coupland, “Cascaded linear shift invariant processing to improve discrimination and noise tolerance in optical pattern recognition,” in Optical Pattern Recognition IX, D. P. Casasent, T. Chao, eds., Proc. SPIE3386, 272–283 (1998).
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Bayesian classification of a two-class problem.

Fig. 2
Fig. 2

Shift-invariant class representation.

Fig. 3
Fig. 3

Shift-invariant two-class classification problem.

Fig. 4
Fig. 4

Schematic of the cascaded correlator architecture.

Fig. 5
Fig. 5

Representation of sample training images for classification at Bhattacharyya distance values μ1 = 0, μ2 = 2.3: (a) true-class image and (b) false-class image.

Fig. 6
Fig. 6

Cross section through the origin of the sample final-stage non-training-set cascade output for separability μ1 = 2.3, μ2 = 0: (a) true class and (b) false class.

Fig. 7
Fig. 7

Mean performance under varying separability conditions of the linear Bayes and the single correlator classifiers.

Fig. 8
Fig. 8

Performance under varying separability conditions of the quadratic Bayes classifier and the two-stage cascaded correlator: (a) mean performance and (b) difference of the quadratic Bayes classifier and the cascaded correlator.

Fig. 9
Fig. 9

Sample class distributions μ1 = 0.121, μ2 = 0.119 and resulting decision boundaries of the cascaded correlator and the quadratic Bayes technique.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

hX=-logepX|ω1+logepX|ω2ω1ω2logeP1P2,
hX=12X-M1TΣ1-1X-M1-12X-M2T×Σ2-1X-M2+12loge|Σ1||Σ2|ω1ω2logeP1P2,
hX=12 Σ1+12 Σ2-1M2-M1TXω1ω2 v0,
fx=x2,  x2t, fx=0, x2<t,
μ=μ1+μ2,μ1=18M2-M1TΣ1+Σ22-1M2-M1μ2=12logeΣ1+Σ22Σ1Σ2.
M1=1.74.2,  Σ1=0.205000.205, M2=24,  Σ2=0.2160.1590.1590.216,

Metrics