Abstract

This work describes several approaches to the estimation of target detection and identification probabilities as a function of target range. A Bayesian approach to estimation is adopted, whereby the posterior probability distributions associated with these probabilities are analytically derived. The parameter posteriors are then used to develop credible intervals quantifying the degree of uncertainty in the parameter estimates. In our first approach we simply show how these credible intervals evolve as a function of range. A second approach, also following the Bayesian philosophy, attempts to directly estimate the parameterized performance curves. This second approach makes efficient use of the available data and yields a distribution of probability versus range curves. Finally, we demonstrate both approaches using experimental data collected from wide field-of-view imagers focused on maritime targets.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. G. Driggers, J. S. Taylor, and K. Krapels, “Probability of identification cycle criterion (N50/N90) for underwater mine target acquisition,” Opt. Eng. 46, 033201 (2007).
    [CrossRef]
  2. S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
    [CrossRef]
  3. R. H. Vollmerhausen, E. Jacobs, and R. G. Driggers, “New metric for predicting target acquisition performance,” Opt. Eng. 43, 2806–2818 (2004).
    [CrossRef]
  4. A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics, 3rd ed. (McGraw-Hill, 1974).
  5. J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
    [CrossRef]
  6. W. A. Link and R. J. Barker, Bayesian Inference with Ecological Examples (Academic, 2010).
  7. P. Walley, “Inferences from multinomial data: learning about a bag of marbles,” J. R. Statist. Soc. B 58, 3–57 (1996).
  8. W. L. Quirin, Probability and Statistics (Harper & Row, 1978).
  9. R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
    [CrossRef]
  10. V. Dhar and Z. Khan, “Comparison of modeled atmosphere-dependent range performance of long-wave and mid-wave ir iamgers,” Infrared Phys. Technol. 51, 520–527 (2008).
    [CrossRef]
  11. W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika 57, 97–109 (1970).
    [CrossRef]
  12. J. M. Nichols, M. Currie, F. Bucholtz, and W. A. Link, “Bayesian estimation of weak material dispersion: theory and experiment,” Opt. Express 18, 2076–2089 (2010).
    [CrossRef]
  13. A. N. Kolmogorov, Foundations of Probability (Chelsea, 1956).

2010

2008

V. Dhar and Z. Khan, “Comparison of modeled atmosphere-dependent range performance of long-wave and mid-wave ir iamgers,” Infrared Phys. Technol. 51, 520–527 (2008).
[CrossRef]

2007

R. G. Driggers, J. S. Taylor, and K. Krapels, “Probability of identification cycle criterion (N50/N90) for underwater mine target acquisition,” Opt. Eng. 46, 033201 (2007).
[CrossRef]

2006

S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
[CrossRef]

J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
[CrossRef]

2004

R. H. Vollmerhausen, E. Jacobs, and R. G. Driggers, “New metric for predicting target acquisition performance,” Opt. Eng. 43, 2806–2818 (2004).
[CrossRef]

1998

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

1996

P. Walley, “Inferences from multinomial data: learning about a bag of marbles,” J. R. Statist. Soc. B 58, 3–57 (1996).

1970

W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika 57, 97–109 (1970).
[CrossRef]

Barker, R. J.

W. A. Link and R. J. Barker, Bayesian Inference with Ecological Examples (Academic, 2010).

Boes, D. C.

A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics, 3rd ed. (McGraw-Hill, 1974).

Bucholtz, F.

Cox, P. G.

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

Currie, M.

Deaver, D. M.

J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
[CrossRef]

Dhar, V.

V. Dhar and Z. Khan, “Comparison of modeled atmosphere-dependent range performance of long-wave and mid-wave ir iamgers,” Infrared Phys. Technol. 51, 520–527 (2008).
[CrossRef]

Driggers, R. G.

R. G. Driggers, J. S. Taylor, and K. Krapels, “Probability of identification cycle criterion (N50/N90) for underwater mine target acquisition,” Opt. Eng. 46, 033201 (2007).
[CrossRef]

R. H. Vollmerhausen, E. Jacobs, and R. G. Driggers, “New metric for predicting target acquisition performance,” Opt. Eng. 43, 2806–2818 (2004).
[CrossRef]

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

Edwards, T. C.

S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
[CrossRef]

Graybill, F. A.

A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics, 3rd ed. (McGraw-Hill, 1974).

Hastings, W. K.

W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika 57, 97–109 (1970).
[CrossRef]

Hixon, J. G.

S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
[CrossRef]

Jacobs, E.

R. H. Vollmerhausen, E. Jacobs, and R. G. Driggers, “New metric for predicting target acquisition performance,” Opt. Eng. 43, 2806–2818 (2004).
[CrossRef]

Khan, Z.

V. Dhar and Z. Khan, “Comparison of modeled atmosphere-dependent range performance of long-wave and mid-wave ir iamgers,” Infrared Phys. Technol. 51, 520–527 (2008).
[CrossRef]

Kolmogorov, A. N.

A. N. Kolmogorov, Foundations of Probability (Chelsea, 1956).

Krapels, K.

R. G. Driggers, J. S. Taylor, and K. Krapels, “Probability of identification cycle criterion (N50/N90) for underwater mine target acquisition,” Opt. Eng. 46, 033201 (2007).
[CrossRef]

S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
[CrossRef]

Leachtenauer, J.

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

Link, W. A.

Mood, A. M.

A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics, 3rd ed. (McGraw-Hill, 1974).

Moyer, S.

S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
[CrossRef]

Nichols, J. M.

O’Conner, J. D.

J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
[CrossRef]

O’Shea, P.

J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
[CrossRef]

Palmer, J. E.

J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
[CrossRef]

Quirin, W. L.

W. L. Quirin, Probability and Statistics (Harper & Row, 1978).

Scribner, D. A.

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

Taylor, J. S.

R. G. Driggers, J. S. Taylor, and K. Krapels, “Probability of identification cycle criterion (N50/N90) for underwater mine target acquisition,” Opt. Eng. 46, 033201 (2007).
[CrossRef]

Vollmerhausen, R.

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

Vollmerhausen, R. H.

R. H. Vollmerhausen, E. Jacobs, and R. G. Driggers, “New metric for predicting target acquisition performance,” Opt. Eng. 43, 2806–2818 (2004).
[CrossRef]

Walley, P.

P. Walley, “Inferences from multinomial data: learning about a bag of marbles,” J. R. Statist. Soc. B 58, 3–57 (1996).

Biometrika

W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika 57, 97–109 (1970).
[CrossRef]

Infrared Phys. Technol.

V. Dhar and Z. Khan, “Comparison of modeled atmosphere-dependent range performance of long-wave and mid-wave ir iamgers,” Infrared Phys. Technol. 51, 520–527 (2008).
[CrossRef]

J. R. Statist. Soc. B

P. Walley, “Inferences from multinomial data: learning about a bag of marbles,” J. R. Statist. Soc. B 58, 3–57 (1996).

Opt. Eng.

R. G. Driggers, J. S. Taylor, and K. Krapels, “Probability of identification cycle criterion (N50/N90) for underwater mine target acquisition,” Opt. Eng. 46, 033201 (2007).
[CrossRef]

S. Moyer, J. G. Hixon, T. C. Edwards, and K. Krapels, “Probability of identification of small hand-held objects for electro-optic forward-looking infrared systems,” Opt. Eng. 45, 063201 (2006).
[CrossRef]

R. H. Vollmerhausen, E. Jacobs, and R. G. Driggers, “New metric for predicting target acquisition performance,” Opt. Eng. 43, 2806–2818 (2004).
[CrossRef]

R. G. Driggers, P. G. Cox, J. Leachtenauer, R. Vollmerhausen, and D. A. Scribner, “Targeting and intelligence electro-optical recognition modeling: a juxtaposition of the probabilities of discrimination and the general image quality equation,” Opt. Eng. 37, 789–797 (1998).
[CrossRef]

Opt. Express

Proc. SPIE

J. D. O’Conner, P. O’Shea, J. E. Palmer, and D. M. Deaver, “Standard target sets for field sensor performance measurements,” Proc. SPIE 6207, 62070U (2006).
[CrossRef]

Other

W. A. Link and R. J. Barker, Bayesian Inference with Ecological Examples (Academic, 2010).

W. L. Quirin, Probability and Statistics (Harper & Row, 1978).

A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics, 3rd ed. (McGraw-Hill, 1974).

A. N. Kolmogorov, Foundations of Probability (Chelsea, 1956).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1.
Fig. 1.

Noninformative Dirichlet prior (α1s=α2s=α3s=1) for a two-class problem.

Fig. 2.
Fig. 2.

Estimated posterior probability distributions for identifying ship class 1, ship class 2, and no ship (left to right), respectively, given that the true ship class is 1. Estimates are based on the data in Table 1 and a noninformative Dirichlet prior.

Fig. 3.
Fig. 3.

Estimated posterior probability distributions for probability of detection and probability of false alarm (left, right, resp.). Estimates are based on the data in Table 1 and a noninformative Dirichlet prior.

Fig. 4.
Fig. 4.

Predicted range performance of an MWIR camera with 1 mrad instantaneous FOV obtained using NVTherm modeling software. Shown are the predicted probability of detection and the probability of identification curves associated with a 4m×7m target. Also shown are the logistic function approximations [Eq. (21)] obtained by least-squares fit. The logistic model is able to capture the basic detection and identification performance as a function of range and requires only two parameters be estimated.

Fig. 5.
Fig. 5.

Estimated range performance based on the observer counts in Table 3. The Markov chains associated with parameters a1, a2 are shown on the left. The estimated range performance curve is shown along with the upper and lower bounds associated with the 95% credible interval. Also shown are the range-specific MLEs for comparison.

Fig. 6.
Fig. 6.

(Left) Sample screenshot from the experiment. The top window shows the entire FOV, while the subwindows each show 1/3 of the FOV and are displayed at the resolution of the monitor. The same display format was used for the MWIR imager with the image appearing in grayscale instead of color. (Right) Target set consisting of (clockwise from top left) Boston Whaler, fishing boat, and LCM.

Fig. 7.
Fig. 7.

Visible (left) and MWIR (right) cameras used in this study.

Fig. 8.
Fig. 8.

Posterior distributions for probability of correctly identifying a (left) Boston Whaler, (middle) fishing boat, and (right) LCM at 550 m.

Fig. 9.
Fig. 9.

95% credible intervals for estimated probability of identification. Results are given for both the visible panoramic camera (top row) and the MWIR panoramic camera (bottom row). All estimates assume noninformative Dirichlet prior.

Fig. 10.
Fig. 10.

Markov chains and the associated PDFs for a1, a2 generated from the MCMC algorithm applied to the visible camera response data with the Boston Whaler as the target. The algorithm assumed uniform priors for each parameter and used B=20,000 burn-in iterations to ensure stationarity of the resulting Markov chains. The final 10,000 samples were retained and used to form the posterior probability distributions (right column).

Fig. 11.
Fig. 11.

95% credible intervals for estimated probability of identification range curves. Results are given for both the visible panoramic camera (top row) and the MWIR panoramic camera (bottom row). All estimates assume uniform prior on the variables a1s, a2s. For comparison, the range-specific MLE point estimates are also shown (+).

Fig. 12.
Fig. 12.

95% credible intervals for estimated probability of detection range curves. Results are given for both the visible panoramic camera (top row) and the MWIR panoramic camera (bottom row). All estimates assume uniform prior on the variables a1s, a2s. For comparison, the range-specific MLE point estimates are also shown (+).

Tables (3)

Tables Icon

Table 1. Example Data Set for an S=2 Class Target Identification Problem

Tables Icon

Table 2. MLEs Associated with the Data Set Given in Table 1

Tables Icon

Table 3. Counts Associated with a Hypothetical Target Identification Experiment

Equations (32)

Equations on this page are rendered with MathJax. Learn more.

fK(ks|ps)=(1i=1Spis)1i=1Skisi=1Spiskis,
kis(t)={1:i=j0:otherwise
fKs(ks|ps)=Ns!(Nst=1Nsi=1Skis(t))!i=1S(t=1NSkis(t))!×t=1Ns(1i=1Spis)1i=1Skis(t)i=1Spiskis(t)=Ns!(Nst=1Nsi=1Skis(t))!i=1S(t=1NSkis(t))!×(1i=1Spis)Nst=1Nsi=1Skis(t)i=1Spist=1NSkis(t)=Ns!i=0Sk˜is!(1i=1Spis)k˜0si=1Spisk˜is,
log(Ns!i=0Sk˜is!)+(Nsi=1Sk˜is)log(1i=1Spis)+i=1Sk˜islog(pis),
0=pis{log(Ns!i=0Sk˜is!)}(Nsi=1Sk˜is)(1i=1Spis)+(k˜ispis)(k˜ispis)=(Nsi=1Sk˜is)(1i=1Spis).
k¯isk˜is/Ns,
(k¯ispis)=(1i=1Sk¯is)(1i=1Spis).
p^is=k¯is,
fPs(ps|ks)=fK(ks|ps)fπ(ps)Rs+1fK(ks|ps)fπ(ps)dps,
fπ(ps|αs)=1B(αs)i=0Spisαis1=1B(αs)(1i=1Spis)α0s1i=1Spisαis1Dir(ps|αs),
B(αs)i=0SΓ(αis)Γ(i=0Sαis)
fPs(ps|k˜s)Dir(ps|αs+k˜s).
fPis(pis)=RsfPs(ps)dps.
fPis(pis)=1B(k˜is+αis,αs*k˜isαis)pisk˜is+αis1(1pis)αs*k˜isαis1Beta(k˜is+αis,αs*k˜isαis),
p^is=k˜is+αisαs*2.
F(x|a,b)0xBeta(a,b),
CL(1ϵ)F1(ϵ2|a,b),CU(1ϵ)F1(1ϵ2|a,b).
CL(0.95)=F1(0.025,k11+α11,α1*k11),CU(0.95)=F1(0.975,k11+α11,α1*k11).
0.30p^110.86
fPD(pD)=1B(13,4)pD12(1pD)3,
fPfa(pfa)=1B(3,4)pfa2(1pfa)3.
p(r)=11+exp(a1ra2),
fKs(ks|a)=j=1RNjs!k˜0s(rj)!k˜ss(rj)!(1(11+exp(a1srja2s)))k˜0s(rj)×(11+exp(a1srja2s))k˜ss(rj).
FP(p)Pr(Pp),
fP(p)limdp0FP(p+dp)FP(p)dp=FP(p)p,
fP(p)dp=1,fP(p)0,0p1.
fK(ki)Pr(K=ki).
ifK(ki)=1,fK(ki)0i.
r=fAis(ais*)g(ais(j1)|ais*)fAis(ais(j1))g(ais*|ais(j1)).
g(ais*|ais(j1))=12C,|ais*ais(i1)|<C,
asi=(a1s,a2s,,a(i1)s,a(i+1)s,,aSs).
fAs(ais(j)|ai=ai(j1)).

Metrics