Abstract

When building an imaging system for detection tasks in medical imaging, we need to evaluate how well the system performs before we can optimize it. One way to do the evaluation is to calculate the performance of the Bayesian ideal observer. The ideal-observer performance is often computationally expensive, and it is very useful to have an approximation to it. We use a parameterized probability density function to represent the corresponding densities of data under the signal-absent and the signal-present hypotheses. We develop approximations to the ideal-observer detectability as a function of signal parameters involving the Fisher information matrix, which is normally used in parameter estimation problems. The accuracy of the approximation is illustrated in analytical examples and lumpy-background simulations. We are able to predict the slope of the detectability as a function of the signal parameter. This capability suggests that the Fisher information matrix itself evaluated at the null parameter value can be used as the figure of merit in imaging system evaluation. We are also able to provide a theoretical foundation for the connection between detection tasks and estimation tasks.

© 2006 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H. H. Barrett and E. Clarkson, "Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions," J. Opt. Soc. Am. A 15, 1520-1535 (1998).
    [CrossRef]
  2. H. H. Barret "Objective assessment of image quality: effects of quantum noise arid object variability," J. Opt. Soc. Am. A 7, 1266-1278 (1990).
    [CrossRef]
  3. H. H. Barrett and K. Myers, Foundations of Image Science (Wiley, 2004).
  4. K. M. Hanson, "Variations in task and the ideal observer," in Proc. SPIE 419, 60-67 (1983).
  5. S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
    [CrossRef] [PubMed]
  6. S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
    [CrossRef] [PubMed]
  7. H. H. Barrett, J. L. Denny, R. F. Wagner, and K. J. Myers, "Objective assessment of image quality. II. Fisher information, Fourier crosstalk, and figures of merit for task performance," J. Opt. Soc. Am. A 12, 834-852 (1995).
    [CrossRef]
  8. E. Clarkson and H. H. Barrett, "Approximations to ideal-observer performance on signal-detection tasks," Appl. Opt. 39, 1783-1793 (2000).
    [CrossRef]
  9. S. Kullback, Information Theory and Statistics (Dover, 1997).
  10. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley-Interscience, 1991).
    [CrossRef]
  11. F. Shen and E. Clarkson, "Using Fisher information to compute ideal-observer performance on detection tasks," in Proc. SPIE 5372, 22-30 (2004).
    [CrossRef]
  12. M. A. Kupinski, J. Hoppin, E. Clarkson, and H. Barrett, "Ideal observer computation in medical imaging with use of Markov chain Monte Carlo," J. Opt. Soc. Am. A 20, 430-438 (2003).
    [CrossRef]
  13. M. A. Kupinski, E. Clarkson, K. Gross, and J. W. Hoppin, "Optimizing imaging hardware for estimation tasks," in Proc. SPIE 5043, 309-313 (2003).

2005

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

2004

F. Shen and E. Clarkson, "Using Fisher information to compute ideal-observer performance on detection tasks," in Proc. SPIE 5372, 22-30 (2004).
[CrossRef]

2003

M. A. Kupinski, E. Clarkson, K. Gross, and J. W. Hoppin, "Optimizing imaging hardware for estimation tasks," in Proc. SPIE 5043, 309-313 (2003).

M. A. Kupinski, J. Hoppin, E. Clarkson, and H. Barrett, "Ideal observer computation in medical imaging with use of Markov chain Monte Carlo," J. Opt. Soc. Am. A 20, 430-438 (2003).
[CrossRef]

2000

1998

1995

H. H. Barrett, J. L. Denny, R. F. Wagner, and K. J. Myers, "Objective assessment of image quality. II. Fisher information, Fourier crosstalk, and figures of merit for task performance," J. Opt. Soc. Am. A 12, 834-852 (1995).
[CrossRef]

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

1990

1983

K. M. Hanson, "Variations in task and the ideal observer," in Proc. SPIE 419, 60-67 (1983).

Abbey, C. K.

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

Barret, H. H.

Barrett, H.

Barrett, H. H.

Clarkson, E.

Cover, T. M.

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley-Interscience, 1991).
[CrossRef]

de Vries, D. J.

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

Denny, J. L.

Gross, K.

M. A. Kupinski, E. Clarkson, K. Gross, and J. W. Hoppin, "Optimizing imaging hardware for estimation tasks," in Proc. SPIE 5043, 309-313 (2003).

Hanson, K. M.

K. M. Hanson, "Variations in task and the ideal observer," in Proc. SPIE 419, 60-67 (1983).

Hoppin, J.

Hoppin, J. W.

M. A. Kupinski, E. Clarkson, K. Gross, and J. W. Hoppin, "Optimizing imaging hardware for estimation tasks," in Proc. SPIE 5043, 309-313 (2003).

Kijewski, M. F.

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

Kullback, S.

S. Kullback, Information Theory and Statistics (Dover, 1997).

Kupinski, M. A.

M. A. Kupinski, J. Hoppin, E. Clarkson, and H. Barrett, "Ideal observer computation in medical imaging with use of Markov chain Monte Carlo," J. Opt. Soc. Am. A 20, 430-438 (2003).
[CrossRef]

M. A. Kupinski, E. Clarkson, K. Gross, and J. W. Hoppin, "Optimizing imaging hardware for estimation tasks," in Proc. SPIE 5043, 309-313 (2003).

Moore, S. C.

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

Müller, S. P.

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

Myers, K.

H. H. Barrett and K. Myers, Foundations of Image Science (Wiley, 2004).

Myers, K. J.

Nandram, B.

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

Rybicki, F. J.

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

Shen, F.

F. Shen and E. Clarkson, "Using Fisher information to compute ideal-observer performance on detection tasks," in Proc. SPIE 5372, 22-30 (2004).
[CrossRef]

Thomas, J. A.

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley-Interscience, 1991).
[CrossRef]

Wagner, R. F.

Appl. Opt.

J. Opt. Soc. Am. A

Med. Phys.

S. C. Moore, D. J. de Vries, B. Nandram, M. F. Kijewski, and S. P. Müller, "Collimator optimization for lesion detection incorporating prior information about lesion size," Med. Phys. 22, 703-713 (1995).
[CrossRef] [PubMed]

Phys. Med. Biol.

S. P. Müller, C. K. Abbey, F. J. Rybicki, S. C. Moore, and M. F. Kijewski, "Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio," Phys. Med. Biol. 50, 3697-3715 (2005).
[CrossRef] [PubMed]

Proc. SPIE

K. M. Hanson, "Variations in task and the ideal observer," in Proc. SPIE 419, 60-67 (1983).

F. Shen and E. Clarkson, "Using Fisher information to compute ideal-observer performance on detection tasks," in Proc. SPIE 5372, 22-30 (2004).
[CrossRef]

M. A. Kupinski, E. Clarkson, K. Gross, and J. W. Hoppin, "Optimizing imaging hardware for estimation tasks," in Proc. SPIE 5043, 309-313 (2003).

Other

H. H. Barrett and K. Myers, Foundations of Image Science (Wiley, 2004).

S. Kullback, Information Theory and Statistics (Dover, 1997).

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley-Interscience, 1991).
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Exact detectability and approximations to detectability for Poisson noise. The horizontal axis is the signal strength M s , and the vertical axis is the detectability. When the signal strength is 1 in (a), from top to bottom, we have the Fisher information approximation given by Eq. (3.13), the fourth-order G θ ( 0 ) approximation, the exact value of the detectability, and the third-order approximation using Eqs. (3.13, 3.14). When the signal strength is 1 in (b), from top to bottom, we have the exact value of the detectability, the second-order pr ( g ρ ) approximation given by Eq. (3.17), and the third-order pr ( g ρ ) approximation given by Eq. (3.18).

Fig. 2
Fig. 2

Lumpy model objects and their noisy image realizations. A signal-absent lumpy object and a signal-present lumpy object are shown in (a) and (b), respectively; their noisy realizations are shown in (c) and (d), respectively. The number and location of lumps in these objects are random.

Fig. 3
Fig. 3

Single point histogram of the lumpy objects.

Fig. 4
Fig. 4

Illustration of the second-order Fisher information approximation of the detectability for lumpy-background images. The parameter considered here is the amplitude of the signal. The simulated ideal-observer detectability is infinite when the signal amplitude equals 0.3 and therefore is not shown in the figure.

Fig. 5
Fig. 5

Illustration of the second-order Fisher information approximation of the detectability for lumpy-background images. The parameter considered here is the size of the signal.

Equations (53)

Equations on this page are rendered with MathJax. Learn more.

TPF ( x ) = pr ( t x H 1 ) = x pr ( t H 1 ) d t .
FPF ( x ) = pr ( t x H 0 ) = x pr ( t H 0 ) d t .
AUC = 0 1 TPF d ( FPF ) = d x TPF ( x ) d d x FPF ( x ) .
Λ ( g ) = pr ( g H 1 ) pr ( g H 0 ) .
AUC Λ 1 2 + 1 2 erf ( 1 2 2 G ( 0 ) ) ,
G ( β ) = ln Λ β + 1 2 0 ( β + 1 2 ) ( β 1 2 )
AUC Λ = 1 1 2 π 0 Λ 1 2 + i α 0 Λ 1 2 i α 0 d α α 2 + 1 4
AUC Λ = 1 2 + 1 2 erf ( 1 2 d A ) ,
d A 2 G ( 0 ) .
F i j ( θ 0 ) = 1 pr 2 ( g θ ) pr ( g θ ) θ i pr ( g θ ) θ j θ 0 .
f ( θ ) θ 0 = f ( θ ) pr ( g θ 0 ) d g .
Λ θ ( g ) = pr ( g θ ) pr ( g θ 0 ) ,
G θ ( β ) = ln Λ θ β + 1 2 θ 0 ( β + 1 2 ) ( β 1 2 )
AUC Λ ( θ ) = 1 1 2 π 0 Λ θ 1 2 + i α θ 0 Λ θ 1 2 i α θ 0 d α α 2 + 1 4 .
AUC Λ ( θ ) = 1 1 2 π 0 exp [ ( α 2 + 1 4 ) d A 2 ( θ ) ] d α α 2 + 1 4 .
0 { exp [ ( α 2 + 1 4 ) γ ( θ ) ] μ θ ( α ) } d α α 2 + 1 4 = 0 .
θ i γ ( θ ) = 0 θ i μ θ ( α ) d α α 2 + 1 4 0 exp [ ( α 2 + 1 4 ) γ ( θ ) ] d α .
θ i μ θ ( α ) = ( 1 2 + i α ) Λ θ 1 2 + i α 1 pr ( g θ 0 ) [ pr ( g θ ) θ i ] θ 0 Λ θ 1 2 i α θ 0 + c . c . ,
θ i μ θ ( α ) θ = θ 0 = ( 1 2 + i α ) 1 pr ( g θ 0 ) [ pr ( g θ ) θ i ] θ 0 θ = θ 0 + c . c . = ( 1 2 + i α ) θ i pr ( g θ ) pr ( g θ 0 ) θ 0 θ = θ 0 + c . c .
θ i γ ( θ ) θ = θ 0 = 0 .
2 θ i θ j exp [ ( α 2 + 1 4 ) γ ( θ ) ] θ = θ 0 = ( α 2 + 1 4 ) 2 θ i θ j γ ( θ ) θ = θ 0 ,
2 θ i θ j μ θ ( α ) θ = θ 0 = 2 ( α 2 + 1 4 ) F i j ( θ 0 ) ,
2 θ i θ j γ ( θ ) θ = θ 0 = 2 F i j ( θ 0 ) .
d A ( θ ) = ( θ θ 0 ) F ( θ 0 ) ( θ θ 0 ) + .
3 θ i θ j θ k γ ( θ ) θ = θ 0 = [ F j k ( θ ) θ i + F i k ( θ ) θ j + F i j ( θ ) θ k ] θ = θ 0 ,
G θ ( 0 ) = 4 ln ( Λ θ 1 2 θ 0 ) .
θ i G θ ( 0 ) θ = θ 0 = 2 Λ θ 1 2 Λ θ θ i θ 0 Λ θ 1 2 θ 0 θ = θ 0 = 2 1 pr ( g θ 0 ) [ pr ( g θ ) θ i ] θ 0 θ = θ 0 = 2 θ i pr ( g θ ) pr ( g θ 0 ) θ 0 θ = θ 0 = 0 .
pr ( g ρ ) = pr ρ ( g H 1 ) pr 1 ρ ( g H 0 ) Λ ρ 0 = Λ ρ ( g ) pr ( g H 0 ) Λ ρ 0 ,
F ( 0 ) = ρ [ ln pr ( g ρ ) ] ρ [ ln pr ( g ρ ) ] 0 ρ = 0 ,
ρ [ ln pr ( g ρ ) ] = ln Λ ( g ) 1 Λ ρ 0 ρ e ( ln λ ) ρ 0
= ln Λ ( g ) 1 Λ ρ 0 Λ ρ ln Λ 0
= ln Λ ( g ) Λ ρ Λ ρ 0 pr ( g H 0 ) ln Λ d g
= λ ( g ) λ ρ
= s ( g ) .
F ( 0 ) = s 2 0 = var 0 λ .
d A = var 0 λ + .
d A = var 0 λ + 1 2 s 3 0 + ,
pr ( g H 0 ) = m = 1 M exp [ g m log ( b m ) b m ] g m ! .
g ¯ m = n = 1 N h m ( r ) L ( r c n ) d r .
θ = signal lumps full width at half - magnitude background lumps full width at half - magnitude ,
2 θ i θ j μ θ ( α ) θ = θ 0 = ( α 2 + 1 4 ) Λ θ 3 2 + i α Λ θ θ i Λ θ θ j 0 Λ θ 1 2 i α 0 θ = θ 0 + ( 1 2 + i α ) Λ θ 1 2 + i α 2 Λ θ θ i θ j 0 Λ θ 1 2 i α 0 θ = θ 0 + ( α 2 + 1 4 ) Λ θ 1 2 + i α Λ θ θ i 0 Λ θ 1 2 i α Λ θ θ j 0 θ = θ 0 + c . c . = 2 ( α 2 + 1 4 ) Λ θ θ i Λ θ θ j 0 = 2 ( α 2 + 1 4 ) F i j ( θ 0 ) .
3 θ i θ j θ k μ θ ( α ) θ = θ 0 = ( α 2 + 1 4 ) Λ θ 3 2 + i α ( 2 Λ θ θ i θ k Λ θ θ j + Λ θ θ i 2 Λ θ θ j θ k + 2 Λ θ θ i θ j Λ θ θ k ) 0 Λ θ 1 2 i α 0 θ = θ 0 ( α 2 + 1 4 ) ( 3 2 + i α ) Λ θ 5 2 + i α Λ θ θ i Λ θ θ j Λ θ θ k 0 Λ θ 1 2 i α 0 θ = θ 0 + c . c . = ( α 2 + 1 4 ) 3 Λ θ θ i Λ θ θ j Λ θ θ k 2 ( Λ θ θ i 2 Λ θ θ j θ k + Λ θ θ j 2 Λ θ θ i θ k + Λ θ θ k 2 Λ θ θ i θ j ) 0 θ = θ 0 = ( α 2 + 1 4 ) [ θ i F j k ( θ ) + θ j F i k ( θ ) + θ k F i j ( θ ) ] θ = θ 0 .
3 γ ( θ ) θ i θ j θ k = [ θ i F j k ( θ ) + θ j F i k ( θ ) + θ k F i j ( θ ) ] θ = θ 0 .
d θ F ( θ 0 ) d θ = d θ θ ln pr ( g θ ) θ ln pr ( g θ ) θ 0 d θ .
J i j = τ i θ j .
J d θ = [ τ 1 θ 1 τ 1 θ 2 τ 1 θ n τ 2 θ 1 τ 2 θ 2 τ 2 θ n τ n θ 1 τ n θ 2 τ n θ n ] ( d θ 1 d θ 2 d θ n ) = ( i = 1 n τ 1 θ i d θ i i = 1 n τ 2 θ i d θ i i = 1 n τ n θ i d θ i ) = ( d τ 1 d τ 2 d τ n ) = d τ ,
J τ = [ τ 1 θ 1 τ 2 θ 1 τ n θ 1 τ 1 θ 2 τ 2 θ 2 τ n θ 2 τ 1 θ n τ 2 θ n τ n θ n ] ( τ 1 τ 2 τ n ) = ( i = 1 n τ i τ i θ 1 i = 1 n τ i τ i θ 2 i = 1 n τ i τ i θ n ) = ( θ 1 θ 2 θ n ) = θ .
d τ τ ln pr ( g τ ( θ ) ) = ( J d θ ) ( J ) 1 θ ln pr ( g θ ) = d θ θ ln pr ( g θ ) ,
d θ F ( θ 0 ) d θ = d τ F ( τ 0 ) d τ .
2 θ i θ j G θ ( 0 ) θ = θ 0 = ( Λ θ 3 2 Λ θ θ i Λ θ θ j 0 Λ θ 1 2 0 2 Λ θ 1 2 2 Λ θ θ i θ j 0 Λ θ 1 2 0 + Λ θ 1 2 Λ θ θ i 0 Λ θ 1 2 Λ θ θ j 0 Λ θ 1 2 0 2 ) θ = θ 0 = Λ θ θ i Λ θ θ j 0 θ = θ 0 = F i j ( θ 0 ) .
3 θ i θ j θ j G θ ( 0 ) θ = θ 0 = [ θ k ( Λ θ 3 2 Λ θ θ i Λ θ θ j 0 Λ θ 1 2 0 2 Λ θ 1 2 2 Λ θ θ i θ j 0 Λ θ 1 2 0 + Λ θ 1 2 Λ θ θ i 0 Λ θ 1 2 Λ θ θ j 0 Λ θ 1 2 0 2 ) ] θ = θ 0 = [ Λ θ 3 2 ( Λ θ θ i 2 Λ θ θ j θ k + Λ θ θ j 2 Λ θ θ i θ k + Λ θ θ k 2 Λ θ θ i θ j ) 0 Λ θ 1 2 0 3 2 Λ θ 5 2 Λ θ θ i Λ θ θ j Λ θ θ k 0 Λ θ 1 2 0 2 ] θ = θ 0 = Λ θ θ i 2 Λ θ θ j θ k + Λ θ θ j 2 Λ θ θ i θ k + Λ θ θ k 2 Λ θ θ i θ j 3 2 Λ θ θ i Λ θ θ j Λ θ θ k 0 θ = θ 0 = 1 2 [ θ i F j k ( θ ) + θ j F i k ( θ ) + θ k F i j ( θ ) ] θ = θ 0 .
3 γ ( ρ ) ρ 3 = 3 3 ρ F ( ρ ) = 6 [ ρ ln pr ( g ρ ) ] [ 2 ρ 2 ln pr ( g ρ ) ] 3 [ ρ ln pr ( g ρ ) ] 3 0 = 6 ( λ λ ρ ) ( λ 2 ρ λ 2 ρ ) 3 ( λ λ ρ ) 3 0 .
3 γ ( ρ ) ρ 3 = 3 ( λ λ 0 ) 3 0 = 3 s 3 0 .

Metrics