Abstract

We describe an algorithm for the detection and clutter rejection of military vehicles in forward-looking infrared (FLIR) imagery. The detection algorithm is designed to be a prescreener that selects regions for further analysis and uses a spatial anomaly approach that looks for target-sized regions of the image that differ in texture, brightness, edge strength, or other spatial characteristics. The features are linearly combined to form a confidence image that is thresholded to find likely target locations. The clutter rejection portion uses target-specific information extracted from training samples to reduce the false alarms of the detector. The outputs of the clutter rejecter and detector are combined by a higher-level evidence integrator to improve performance over simple concatenation of the detector and clutter rejecter. The algorithm has been applied to a large number of FLIR imagery sets, and some of these results are presented here.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. Bhanu, T. Jones, “Image understanding research for automatic target recognition,” IEEE Aerosp. Electron. Syst. Magazine, October1993, pp. 15–22.
  2. B. Bhanu, “Automatic target recognition: state of the art survey,” IEEE Trans. Aerospace Elect. Sys. 22, 364–379 (1986).
    [CrossRef]
  3. L. Chan, S. Der, N. Nasrabadi, “Multistage Infrared Target Detection,” Opt. Eng. 42, 2746–2754 (2003).
    [CrossRef]
  4. M. W. Roth, “Survey of neural network technology for automatic target recognition,” IEEE Trans. Neural Netw. 1, 28–43 (1990).
    [CrossRef] [PubMed]
  5. R. Hecht-Nielsen, Y.-T. Zhou, “VARTAC: a foveal active vision ATR system,” Neural Netw. 8, 1309–1321 (1995).
    [CrossRef]
  6. L. Wang, S. Der, N. Nasrabadi, “Automatic target recognition using a feature-decomposition and data-decomposition modular neural network,” IEEE Trans. Image Process. 7, 1113–1121 (1998).
    [CrossRef]
  7. J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
    [CrossRef]
  8. P. Domingos, M. Pazzani, “On the optimality of the simple Bayesian classifier under zero-one loss,” Mach. Learn. 29, 103–130 (1997).
    [CrossRef]
  9. R. Brooks, “How to build complete creatures rather than isolated cognitive simulators,” in Architectures for Intelligence, K. VanLehn, ed., (Erlbaum, Hillsdale, N.J., 1989), pp. 225–239.
  10. L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
    [CrossRef]
  11. I. T. Jolliffe, Principal Component Analysis (Springer-Verlag, New York, 1986).
    [CrossRef]
  12. S. Fahlman, “Faster learning variations on back-propagation: an empirical study,” Proceedings of the 1988 Connectionist Models Summer School (Morgan Kaufmann, San Francisco, Calif., 1988), pp. 38–51.
  13. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd Ed. (Prentice-Hall, Englewood Cliffs, N.J., 1999).

2003 (1)

L. Chan, S. Der, N. Nasrabadi, “Multistage Infrared Target Detection,” Opt. Eng. 42, 2746–2754 (2003).
[CrossRef]

1998 (3)

L. Wang, S. Der, N. Nasrabadi, “Automatic target recognition using a feature-decomposition and data-decomposition modular neural network,” IEEE Trans. Image Process. 7, 1113–1121 (1998).
[CrossRef]

J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
[CrossRef]

L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

1997 (1)

P. Domingos, M. Pazzani, “On the optimality of the simple Bayesian classifier under zero-one loss,” Mach. Learn. 29, 103–130 (1997).
[CrossRef]

1995 (1)

R. Hecht-Nielsen, Y.-T. Zhou, “VARTAC: a foveal active vision ATR system,” Neural Netw. 8, 1309–1321 (1995).
[CrossRef]

1993 (1)

B. Bhanu, T. Jones, “Image understanding research for automatic target recognition,” IEEE Aerosp. Electron. Syst. Magazine, October1993, pp. 15–22.

1990 (1)

M. W. Roth, “Survey of neural network technology for automatic target recognition,” IEEE Trans. Neural Netw. 1, 28–43 (1990).
[CrossRef] [PubMed]

1986 (1)

B. Bhanu, “Automatic target recognition: state of the art survey,” IEEE Trans. Aerospace Elect. Sys. 22, 364–379 (1986).
[CrossRef]

Bhanu, B.

B. Bhanu, T. Jones, “Image understanding research for automatic target recognition,” IEEE Aerosp. Electron. Syst. Magazine, October1993, pp. 15–22.

B. Bhanu, “Automatic target recognition: state of the art survey,” IEEE Trans. Aerospace Elect. Sys. 22, 364–379 (1986).
[CrossRef]

Brooks, R.

R. Brooks, “How to build complete creatures rather than isolated cognitive simulators,” in Architectures for Intelligence, K. VanLehn, ed., (Erlbaum, Hillsdale, N.J., 1989), pp. 225–239.

Chan, L.

L. Chan, S. Der, N. Nasrabadi, “Multistage Infrared Target Detection,” Opt. Eng. 42, 2746–2754 (2003).
[CrossRef]

Der, S.

L. Chan, S. Der, N. Nasrabadi, “Multistage Infrared Target Detection,” Opt. Eng. 42, 2746–2754 (2003).
[CrossRef]

L. Wang, S. Der, N. Nasrabadi, “Automatic target recognition using a feature-decomposition and data-decomposition modular neural network,” IEEE Trans. Image Process. 7, 1113–1121 (1998).
[CrossRef]

Domingos, P.

P. Domingos, M. Pazzani, “On the optimality of the simple Bayesian classifier under zero-one loss,” Mach. Learn. 29, 103–130 (1997).
[CrossRef]

Duin, R.

J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
[CrossRef]

Fahlman, S.

S. Fahlman, “Faster learning variations on back-propagation: an empirical study,” Proceedings of the 1988 Connectionist Models Summer School (Morgan Kaufmann, San Francisco, Calif., 1988), pp. 38–51.

Hatef, M.

J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
[CrossRef]

Haykin, S.

S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd Ed. (Prentice-Hall, Englewood Cliffs, N.J., 1999).

Hecht-Nielsen, R.

R. Hecht-Nielsen, Y.-T. Zhou, “VARTAC: a foveal active vision ATR system,” Neural Netw. 8, 1309–1321 (1995).
[CrossRef]

Itti, L.

L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

Jolliffe, I. T.

I. T. Jolliffe, Principal Component Analysis (Springer-Verlag, New York, 1986).
[CrossRef]

Jones, T.

B. Bhanu, T. Jones, “Image understanding research for automatic target recognition,” IEEE Aerosp. Electron. Syst. Magazine, October1993, pp. 15–22.

Kittler, J.

J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
[CrossRef]

Koch, C.

L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

Matas, J.

J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
[CrossRef]

Nasrabadi, N.

L. Chan, S. Der, N. Nasrabadi, “Multistage Infrared Target Detection,” Opt. Eng. 42, 2746–2754 (2003).
[CrossRef]

L. Wang, S. Der, N. Nasrabadi, “Automatic target recognition using a feature-decomposition and data-decomposition modular neural network,” IEEE Trans. Image Process. 7, 1113–1121 (1998).
[CrossRef]

Niebur, E.

L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

Pazzani, M.

P. Domingos, M. Pazzani, “On the optimality of the simple Bayesian classifier under zero-one loss,” Mach. Learn. 29, 103–130 (1997).
[CrossRef]

Roth, M. W.

M. W. Roth, “Survey of neural network technology for automatic target recognition,” IEEE Trans. Neural Netw. 1, 28–43 (1990).
[CrossRef] [PubMed]

Wang, L.

L. Wang, S. Der, N. Nasrabadi, “Automatic target recognition using a feature-decomposition and data-decomposition modular neural network,” IEEE Trans. Image Process. 7, 1113–1121 (1998).
[CrossRef]

Zhou, Y.-T.

R. Hecht-Nielsen, Y.-T. Zhou, “VARTAC: a foveal active vision ATR system,” Neural Netw. 8, 1309–1321 (1995).
[CrossRef]

IEEE Aerosp. Electron. Syst. Magazine (1)

B. Bhanu, T. Jones, “Image understanding research for automatic target recognition,” IEEE Aerosp. Electron. Syst. Magazine, October1993, pp. 15–22.

IEEE Trans. Aerospace Elect. Sys. (1)

B. Bhanu, “Automatic target recognition: state of the art survey,” IEEE Trans. Aerospace Elect. Sys. 22, 364–379 (1986).
[CrossRef]

IEEE Trans. Image Process. (1)

L. Wang, S. Der, N. Nasrabadi, “Automatic target recognition using a feature-decomposition and data-decomposition modular neural network,” IEEE Trans. Image Process. 7, 1113–1121 (1998).
[CrossRef]

IEEE Trans. Neural Netw. (1)

M. W. Roth, “Survey of neural network technology for automatic target recognition,” IEEE Trans. Neural Netw. 1, 28–43 (1990).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

L. Itti, C. Koch, E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

IEEE Trans. Pattern Anal. Machine Intell. (1)

J. Kittler, M. Hatef, R. Duin, J. Matas, “On combining classifiers,” IEEE Trans. Pattern Anal. Machine Intell. 20, 226–239 (1998).
[CrossRef]

Mach. Learn. (1)

P. Domingos, M. Pazzani, “On the optimality of the simple Bayesian classifier under zero-one loss,” Mach. Learn. 29, 103–130 (1997).
[CrossRef]

Neural Netw. (1)

R. Hecht-Nielsen, Y.-T. Zhou, “VARTAC: a foveal active vision ATR system,” Neural Netw. 8, 1309–1321 (1995).
[CrossRef]

Opt. Eng. (1)

L. Chan, S. Der, N. Nasrabadi, “Multistage Infrared Target Detection,” Opt. Eng. 42, 2746–2754 (2003).
[CrossRef]

Other (4)

R. Brooks, “How to build complete creatures rather than isolated cognitive simulators,” in Architectures for Intelligence, K. VanLehn, ed., (Erlbaum, Hillsdale, N.J., 1989), pp. 225–239.

I. T. Jolliffe, Principal Component Analysis (Springer-Verlag, New York, 1986).
[CrossRef]

S. Fahlman, “Faster learning variations on back-propagation: an empirical study,” Proceedings of the 1988 Connectionist Models Summer School (Morgan Kaufmann, San Francisco, Calif., 1988), pp. 38–51.

S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd Ed. (Prentice-Hall, Englewood Cliffs, N.J., 1999).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (32)

Fig. 1
Fig. 1

Typical algorithm structure.

Fig. 2
Fig. 2

Easy image from data set.

Fig. 3
Fig. 3

Results on previous image.

Fig. 4
Fig. 4

Feature 0 on previous image.

Fig. 5
Fig. 5

Feature 1 on previous image.

Fig. 6
Fig. 6

Feature 2 on previous image.

Fig. 7
Fig. 7

Feature 3 on previous image.

Fig. 8
Fig. 8

Feature 4 on previous image.

Fig. 9
Fig. 9

Feature 5 on previous image.

Fig. 10
Fig. 10

Feature 6 on previous image.

Fig. 11
Fig. 11

Histogram of feature image for feature 0.

Fig. 12
Fig. 12

Histogram of feature image for feature 1.

Fig. 13
Fig. 13

Histogram of feature image for feature 2.

Fig. 14
Fig. 14

Histogram of feature image for feature 3.

Fig. 15
Fig. 15

Histogram of feature image for feature 4.

Fig. 16
Fig. 16

Histogram of feature image for feature 5.

Fig. 17
Fig. 17

Histogram of feature image for feature 6.

Fig. 18
Fig. 18

ROC curve on spring grassland imagery. The horizontal axis gives the average number of false alarms per frame; the vertical axis is the target detection probability.

Fig. 19
Fig. 19

ROC curve of individual features, spring grassland imagery.

Fig. 20
Fig. 20

Feature 2 image on previous image at scale 0.

Fig. 21
Fig. 21

Feature 2 image on previous image at scale 1.

Fig. 22
Fig. 22

Feature 2 image on previous image at scale 2.

Fig. 23
Fig. 23

Feature 2 image on previous image at scale 3.

Fig. 24
Fig. 24

Feature 2 image on previous image at scale 4.

Fig. 25
Fig. 25

Feature 2 image on previous image at scale 5.

Fig. 26
Fig. 26

Feature 2 image on previous image at scale 6.

Fig. 27
Fig. 27

Four different architectures of the proposed multi-stage target detector.

Fig. 28
Fig. 28

First 100 most dominant 20 × 40 PCA eigenvectors extracted from the target chips.

Fig. 29
Fig. 29

EIGMLP that consists of a transformation layer and a back-end MLP.

Fig. 30
Fig. 30

Pair of FLIR scenes with different characteristics.

Fig. 31
Fig. 31

Detection performance for the training (above) and testing (bottom) set.

Fig. 32
Fig. 32

At 0.9 false alarms per frame, the detections provided by (left) RGD and (right) ECMLP for the left-hand image in Fig. 30. Rectangular box, acceptance window that encompasses the lone target in this image; crosses, detected locations.

Tables (3)

Tables Icon

Table 1 Co-occurrence of Target Detections and False Alarms

Tables Icon

Table 2 Co-occurrence of Target Detections and False Alarms

Tables Icon

Table 3 Co-occurrence of Target Detections and False Alarms

Equations (21)

Equations on this page are rendered with MathJax. Learn more.

Fi,j0=maxk,lNini, jfk, l,
Fi,j1=1nink,lNini,j fk, l-1noutk,lNouti,j fk, l,
Fi,j2=1nink,lNini,j Gini, j-1noutk,lNouti,j Gouti, j,
Gini, j=Ginhi, j+Ginvi, j,
Ginhi, j=|fi, j-fi, j+1|,
Ginvi, j=|fi, j-fi+1, j|,
Fi,j3=Lini, jnin-Louti, jnout,
Lini, j=k,lNini,j |fk, l-μini, j|,
μini, j=1nink,lNini, j fk, l,
Fi,j4=1nink,lNini,j Hini, j-1noutk,lNouti, j Houti, j,
Hini, j=Hinhi, j+Hinvi, j,
Hinhi, j=|k-i|<l |fk, j-fk, j+1|,
Hinvi, j=|k-j|<l |fi, k-fi+k|,
Hinvi, j=|k-i|<l |fk, j-fk, j-1|,
Hinhi, j=|k-j|<l |fi, k-fi+1, k|,
Hinvi, j=|k-i|<l |fk, j-fk, j+1|,
Fi,jm,N=Fi,jm-μmσm,
μm=1Mallk,l Fk,lm,
σm=1Mallk,lFk,lm-μm2.
Gi,j=m=06 ωmFi,jm,N,
SGi,j=1-expαGi,j,

Metrics