Abstract

The ideal-observer performance, as measured by the area under the receiver’s operating characteristic curve, is computed for six examples of signal-detection tasks. Exact values for this quantity, as well as approximations based on the signal-to-noise ratio of the log likelihood and the likelihood-generating function, are found. The noise models considered are normal, exponential, Poisson, and two-sided exponential. The signal may affect the mean or the variance in each case. It is found that the approximation from the likelihood-generating function tracks well with the exact area, whereas the log-likelihood signal-to-noise approximation can fail badly. The signal-to-noise ratio of the likelihood ratio itself is also computed for each example to demonstrate that it is not a good measure of ideal-observer performance.

© 2000 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. H. Barrett, C. K. Abbey, E. Clarkson, “Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions,” J. Opt. Soc. Am. A 15, 1520–1535 (1998).
    [CrossRef]
  2. D. M. Green, J. A. Swets, Signal Detection Theory and Psychophysics (Wiley, New York, 1966).
  3. J. H. Shapiro, “Bounds on the area under the ROC curve,” J. Opt. Soc. Am. A 16, 53–57 (1999).
    [CrossRef]
  4. R. F. Wagner, D. G. Brown, C. E. Metz, “On the multiplex advantage of coded source/aperture photon imaging,” in Digital Radiography, W. R. Brody, ed., Proc. SPIE314, 72–76 (1981).
    [CrossRef]
  5. J. J. Heine, S. R. Deans, L. P. Clarke, “Multiresolution probability analysis of random fields,” J. Opt. Soc. Am. A 16, 6–16 (1999).
    [CrossRef]
  6. H. H. Barrett, C. K. Abbey, E. Clarkson, “Some unlikely properties of the likelihood ratio and its logarithm,” in Medical Imaging 1998: Image Perception, H. L. Kundel, ed., Proc. SPIE3340, 65–77 (1998).
  7. D. G. Brown, M. F. Insana, M. Tapiovara, “Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown,” J. Acoust. Soc. Am. 97, 397–398 (1995).
    [CrossRef]

1999 (2)

1998 (1)

1995 (1)

D. G. Brown, M. F. Insana, M. Tapiovara, “Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown,” J. Acoust. Soc. Am. 97, 397–398 (1995).
[CrossRef]

Abbey, C. K.

H. H. Barrett, C. K. Abbey, E. Clarkson, “Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions,” J. Opt. Soc. Am. A 15, 1520–1535 (1998).
[CrossRef]

H. H. Barrett, C. K. Abbey, E. Clarkson, “Some unlikely properties of the likelihood ratio and its logarithm,” in Medical Imaging 1998: Image Perception, H. L. Kundel, ed., Proc. SPIE3340, 65–77 (1998).

Barrett, H. H.

H. H. Barrett, C. K. Abbey, E. Clarkson, “Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions,” J. Opt. Soc. Am. A 15, 1520–1535 (1998).
[CrossRef]

H. H. Barrett, C. K. Abbey, E. Clarkson, “Some unlikely properties of the likelihood ratio and its logarithm,” in Medical Imaging 1998: Image Perception, H. L. Kundel, ed., Proc. SPIE3340, 65–77 (1998).

Brown, D. G.

D. G. Brown, M. F. Insana, M. Tapiovara, “Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown,” J. Acoust. Soc. Am. 97, 397–398 (1995).
[CrossRef]

R. F. Wagner, D. G. Brown, C. E. Metz, “On the multiplex advantage of coded source/aperture photon imaging,” in Digital Radiography, W. R. Brody, ed., Proc. SPIE314, 72–76 (1981).
[CrossRef]

Clarke, L. P.

Clarkson, E.

H. H. Barrett, C. K. Abbey, E. Clarkson, “Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions,” J. Opt. Soc. Am. A 15, 1520–1535 (1998).
[CrossRef]

H. H. Barrett, C. K. Abbey, E. Clarkson, “Some unlikely properties of the likelihood ratio and its logarithm,” in Medical Imaging 1998: Image Perception, H. L. Kundel, ed., Proc. SPIE3340, 65–77 (1998).

Deans, S. R.

Green, D. M.

D. M. Green, J. A. Swets, Signal Detection Theory and Psychophysics (Wiley, New York, 1966).

Heine, J. J.

Insana, M. F.

D. G. Brown, M. F. Insana, M. Tapiovara, “Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown,” J. Acoust. Soc. Am. 97, 397–398 (1995).
[CrossRef]

Metz, C. E.

R. F. Wagner, D. G. Brown, C. E. Metz, “On the multiplex advantage of coded source/aperture photon imaging,” in Digital Radiography, W. R. Brody, ed., Proc. SPIE314, 72–76 (1981).
[CrossRef]

Shapiro, J. H.

Swets, J. A.

D. M. Green, J. A. Swets, Signal Detection Theory and Psychophysics (Wiley, New York, 1966).

Tapiovara, M.

D. G. Brown, M. F. Insana, M. Tapiovara, “Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown,” J. Acoust. Soc. Am. 97, 397–398 (1995).
[CrossRef]

Wagner, R. F.

R. F. Wagner, D. G. Brown, C. E. Metz, “On the multiplex advantage of coded source/aperture photon imaging,” in Digital Radiography, W. R. Brody, ed., Proc. SPIE314, 72–76 (1981).
[CrossRef]

J. Acoust. Soc. Am. (1)

D. G. Brown, M. F. Insana, M. Tapiovara, “Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown,” J. Acoust. Soc. Am. 97, 397–398 (1995).
[CrossRef]

J. Opt. Soc. Am. A (3)

Other (3)

D. M. Green, J. A. Swets, Signal Detection Theory and Psychophysics (Wiley, New York, 1966).

R. F. Wagner, D. G. Brown, C. E. Metz, “On the multiplex advantage of coded source/aperture photon imaging,” in Digital Radiography, W. R. Brody, ed., Proc. SPIE314, 72–76 (1981).
[CrossRef]

H. H. Barrett, C. K. Abbey, E. Clarkson, “Some unlikely properties of the likelihood ratio and its logarithm,” in Medical Imaging 1998: Image Perception, H. L. Kundel, ed., Proc. SPIE3340, 65–77 (1998).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1
Fig. 1

Exact AUCΛ and the approximations to AUCΛ that are derived from SNRλ and SNR G(0) for example 2 (independent exponential noise) with M = 3. These AUC values are plotted versus the inverse contrast ratio (background to signal). On the left-hand side of the graph the curve that represents the exact value is below the curve for SNR G(0) and above the curve for SNRλ. The curve for the exact value is below the curves for both approximations on the right-hand side of the graph.

Fig. 2
Fig. 2

Exact AUCλ and the approximations to this value that are derived from SNRλ and SNR G(0) for example 3 (Poisson noise) with Mb = 0.1. These AUC values are plotted versus the signal strength, which represents the mean number of photons from the signal in the data. On the left-hand side of the graph the curve for the exact value is below the curve for the SNRλ approximation, which in turn is below the curve for the SNR G(0) approximation. The curve for the exact value is between the curves for the two approximations on the right-hand side.

Fig. 3
Fig. 3

Exact AUCΛ and the approximations to this number that are derived from SNRλ and SNR G(0) for example 4 (Laplacian noise) with M = 1. These AUC values are plotted versus the relative signal strength (signal/noise level). On the left-hand side of the graph the curve for the exact value is below the curve for the SNR G(0) approximation, which in turn is below the curve for the SNRλ approximation. The curve for the exact value is between the curves for the two approximations on the right-hand side.

Equations (99)

Equations on this page are rendered with MathJax. Learn more.

TPFt0=prD2|H2=t0prt|H2dt,FPFt0=prD2|H1=t0prt|H1dt,FNFt0=prD1|H2=-t0prt|H2dt,TNFt0=prD1|H1=-t0prt|H1dt.
AUCt=01TPFdFPF=--TPFtFPFtdt.
ω1=ωt1=- ωtprt|H1dt,ω2=ωt2=- ωtprt|H2dt.
ψ1ξ=exp-2πiξt1,ψ2ξ=exp-2πiξt2,
AUCt=12+12πi P - ψ1ξψ2*ξdξξ.
Λg=prg|H2prg|H1,
AUCΛ=1-120FPFΛ2dΛ.
AUCΛ=0 p1xx p2ydydx =0 p1xx yp1ydydx.
x yp1ydy=-yP˜1yx+x P˜1ydy.
yP˜1yy zp1zdz=y p2zdz.
AUCΛ=0 xp1xP˜1x+0x p1xP˜1ydydx.
0 xp1xP˜1xdx=-12 xP˜12x0+120 P˜12xdx.
0x p1xP˜1ydydx=00y p1xP˜1ydxdy=01-P˜1yP˜1ydy=1-0 P˜12ydy.
ψ1ξ=exp-2πiξλ1,ψ2ξ=exp-2πiξλ2=expλexp-2πiξλ1,
AUCΛ=AUCλ=12+12πi P - ψ1ξψ1*ξ-i2πdξξ.
M1β=expβλ1=Λβ1,M2β=expβλ2=Λβ2.
AUCΛ=12+12πi P - M1iαM11-iαdαα.
AUCΛ=1-14π- M112+iαM112-iαdαα2+1/4.
AUCΛ=1-12π0M112+iα2dαα2+1/4.
M1β=expββ-1Gβ-1/2,M2β=expββ+1Gβ+1/2.
AUCΛ=1-12π0exp-2α2+14×ReGiαdαα2+1/4.
SNRAUC=2 erf-12AUCΛ-1.
SNRλ=2λ2-λ1var1λ+var2λ1/2.
SNRG0=2G01/2.
G0=-4 log M11/2=-4 log-prΛ|H1prΛ|H21/2dΛ.
dBρ1, ρ2=-log ρ1xρ2x1/2dx.
G0=-4 logΛ1/2g1=-4 logMprg|H2prg|H11/2prg|H1dg=-4 logMprg|H1prg|H21/2dg,
AUCΛ=12+12erf12SNRAUC
AUCΛ12+12erf12SNRλ
AUCΛ12+12erf12SNRG0
SNRΛ=2M12-1M12-1+M13-M1221/2,
SNRΛ=2var1Λvar1Λ+var2Λ1/2.
λk1=M1k0,λk2=M1k1.
SNRλ=2M11-M10M10-M102+M11-M1121/2.
prg|H1=2πM detK-1/2×exp-12g-bTK-1g-b.
prg|H2=prg-s|H1.
λg=sTK-1g+12bTK-1b-12b+sTK-1b+s.
λ˜g=sTK-1g=K-1/2sTK-1/2g.
SNRλ=SNRλ˜=sTK-1s1/2.
SNRAUC=SNRG0=SNRλ,
M1β=exp12ββ-1sTK-1s.
SNRΛ=2expsTK-1s-1exp2sTK-1s+11/2.
prg|H1=m=1M1bmexp-gmbm,prg|H2=m=1M1bm+smexp-gmbm+sm.
λg=m=1Msmgmbmbm+sm+m=1Mlogbmbm+sm.
λ˜g=m=1Msmgmbmbm+sm.
M1β=m=1Mπmβ1-γmβ.
AUCλ=1-12 A1A2,
A1=m=1M ρm1+ρm,
A2=n=0M1+2ρnm=0mnM1+ρm+ρnρm-ρn-1.
SNRλ=2m=1M1ρmρm+1×m=1M1ρm+12+1ρm2-1/2.
SNRλ2 m=1M1ρm+121/22M.
SNRG0=2m=1Mlog2ρm+124ρmρm+11/2.
1α1-iγαM=1α+iγ1-iγα++iγ1-iγαM
AUCΛ=1+ρ1+2ρMk=0M-1M+k-1!k!M-1!ρ1+2ρk,
AUCλ12+12erf122M2ρ2+2ρ+11/2
AUCλ12+12erfM log2ρ+124ρρ+11/2
var1Λ=m=1Mπm22πm-1-1,
var2Λ=m=1Mπm33πm-2-m=1Mπm42πm-12.
prg|H1=m=1Mexpgm logbm-bmgm!,prg|H2=m=1Mexpgm logbm+sm-bm-smgm!.
λg=m=1M gm log κm-m=1M sm,
λ˜g=m=1M gm log κm.
M1β=exp-β m=1M smexpm=1M bmκmβ-1.
SNRG0=22m=1M122bm+sm-bmbm+sm1/21/2=2m=1Mbm+sm1/2-bm1/221/2,
SNRλ=2m=1M sm log κm×m=1M2bm+smlog2 κm1/2-1.
12πexp-M2b+s0expMbb+s1/2 cosα logb+sbdαα2+1/4.
fr=k=0 fkrk
fx+y=kl fklxkyl+xlyk+k=0 fkkxkyk,
fkl=k+llfk+l.
12π0 f2xycosα lnxydαα2+1/4=kl fkl minxkyl, xlyk+12k fkkxkyk.
AUCΛ=1-exp-M2b+s×k=0l=0k-1Mk+lbkb+slk!l!+12k=0M2kbkb+skk!k!.
AUCΛ12+12erfM21/2s2b+s1/2
AUCΛ12+12erf Mb+s-b
SNRG0SNRAUC,SNRG0SNRλ.
var1Λ=m=1Mexpsm2bm-1,
var2Λ=m=1Mexpsmκm2+κm-2-m=1Mexp2sm2bm.
prg|H1=m=1M12cmexp-1cm|gm-bm|,prg|H2=m=1M12cmexp-1cm|gm-bm-sm|.
λg=m=1M1cm|gm-bm|-|gm-bm-sm|.
M1β=m=1M121-12β-1+2β2β-1×exp2β-1θmexp-θmβ,
SNRG0=2m=1Mθm-2 log1+θm21/2.
λ˜g=λg+m=1M θm.
λ˜1=m=1M1-exp-θm,λ˜2=m=1M2θm-1+exp-θm,
var1λ˜=m=1M3-2 exp-θm-4θm exp-θm-exp-2θm=var2λ˜.
AUCΛ=1-12π122M exp-m=1M θm01α2Mm=1Msinαθm+2α cosαθm2dαα2+1/4.
AUCΛ=1-12exp-θ1+θ2.
AUCΛ12+12erf×θ-1+exp-θ3-2+4θexp-θ-exp-2θ1/2
AUCΛ12+12erfθ-2 log1+θ21/2.
var1Λ=m=1M13exp-2θm+2 expθm-1,
var2Λ=m=1M152 exp-3θm+3 exp2θm-m=1M19exp-2θm+2 expθm2.
prg|H1=m=1M12cmexp-1cm|gm-bm|,prg|H2=m=1M12smcmexp-1smcm|gm-bm|.
λg=m=1M1βmsm-1sm|gm-bm|-m=1Mlogsm,
M1β=m=1Msm-β1-τmβ.
prg|H1=2πM detK1-1/2 exp-12g-bTK1-1g-b,prg|H2=2πM detK2-1/2 exp-12g-bTK2-1g-b.
λg=12gTK1-1-K2-1g+12logdetK1K2-1.
M1β=detK11/2β-1detK21/2-βdet1-βK1-1+βK2-11/2.
SNRG0=22 logdetK1+K2-logdetK1-logdetK2-2M log 21/2.
z=2iα-12iα+1
AUCΛ=1-14πi×C1-zMdetK1K2-1-zIK2K1-1-zI1/2z.
M1β=a-β1+1/a-1βN.
exp-14 G0m=1M  prg˜m|H1prg˜m|H2dgm1/2.

Metrics