Abstract

A longstanding problem of estimation is extraction of the signal image from a given noisy image. Here, prior knowledge in the form of a second, template image is also assumed to be present. Examples of templates are a blurred version of the signal image, the reference image used in cross-entropy minimization, or the signal power spectrum. We propose and test a method of optimally combining the data and template images to form an improved output. The output is biased, respectively, toward the template or the data image by trading off two goals: (a) minimum output probability of being successfully distinguished from the template as predicted by standard maximum likelihood theory, and (b) maximum output probability of having formed the image data. For additive Gaussian noise the estimation approach is least-squares; for Poisson noise the approach is a compromise between maximum Shannon cross entropy and maximum Burg-type entropy; and for exponential noise the approach includes maximum Burg entropy.

© 1989 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. See, e.g., H. Stark, Ed. Image Recovery, Theory and Applications (Academic, Orlando, 1987).
  2. The classic treatment of Wiener-Kolmogoroff filtering is in H.W. Bode, C. E. Shannon, “A Simplified Derivation of Linear Least-Square Smoothing and Prediction Theory,” Proc. IRE 38, 417 (1950).
    [CrossRef]
  3. L. A. Wainstein, V. D. Zubakov, Extraction of Signals from Noise (Prentice-Hall, Englewood Cliffs, NJ, 1962).
  4. R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” J. Basic Eng., ASME Trans. 82D, 35 (1960).
    [CrossRef]
  5. G. U. Yule, “On the Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer’s Sun-Spot Numbers,” Philos. Trans. R. Soc. London Ser. A 226, 267 (1927). This is the original reference on autoregressive modeling. A more recent review is in A. S.Willsky, Digital Signal Processing and Control, and Estimation Theory (MIT Press, Cambridge, 1979).
    [CrossRef]
  6. A. C. Schell, “Enhancing the Angular Resolution of Incoherent Sources,” Radio Electron. Eng. 29, 21 (1965).
    [CrossRef]
  7. H. Wolter, “On Basic Analogies and Principal Differences Between Optical and Electronic Information,” Prog. Opt. 1, 155 (1961).
    [CrossRef]
  8. D. L. Phillips, “A Technique for the Numerical Solution of Certain Integral Equations of the First Kind,” J. ACM 9, 8–97 (1962).
    [CrossRef]
  9. P. A. Jansson, R. H. Hunt, E. K. Plyler, “Spectral Resolution Enhancement,” J. Opt. Soc. Am. 60, 596 (1970).
    [CrossRef]
  10. J. E. Shore, R. W. Johnson, “Properties of Cross-Entropy Minimization,” IEEE Trans. Inf. Theory IT-27, 472 (1981).
    [CrossRef]
  11. H. L. Van Trees, Detection, Estimation and Modulation Theory, Part 1 (Wiley, New York, 1968).
  12. B. R. Frieden, Probability, Statistical Optics and Data Testing (Springer-Verlag, New York, 1983).
    [CrossRef]
  13. J. L. Harris, “Resolving Power and Decision Theory,” J. Opt. Soc. Am. 54, 606 (1964).
    [CrossRef]
  14. J. W. Goodman, Statistical Optics (Wiley, New York, 1985).
  15. J. P. Burg, “Maximum Entropy Spectral Analysis,” paper presented at Thirty-Seventh Meeting of the Society of Exploration Geophysicists, Oklahoma City (1960);J. P. Burg, “Maximum Entropy Spectral Analysis,” Ph.D. Dissertation, Geophysics Department, Stanford U. (May1975).
  16. J. W. Goodman, “Some Effects of Target-Induced Scintillation on Optical Radar Performance,” Proc. IEEE 53, 1688 (1965).
    [CrossRef]
  17. V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
    [CrossRef]
  18. J. C. Dainty, “Stellar Speckle Interferometry” (Springer-Verlag, New York, 1984).
  19. J. W. Tukey, Exploratory Data Analysis (Addison-Wesley, Reading, MA, 1977).

1982 (1)

V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
[CrossRef]

1981 (1)

J. E. Shore, R. W. Johnson, “Properties of Cross-Entropy Minimization,” IEEE Trans. Inf. Theory IT-27, 472 (1981).
[CrossRef]

1970 (1)

1965 (2)

J. W. Goodman, “Some Effects of Target-Induced Scintillation on Optical Radar Performance,” Proc. IEEE 53, 1688 (1965).
[CrossRef]

A. C. Schell, “Enhancing the Angular Resolution of Incoherent Sources,” Radio Electron. Eng. 29, 21 (1965).
[CrossRef]

1964 (1)

1962 (1)

D. L. Phillips, “A Technique for the Numerical Solution of Certain Integral Equations of the First Kind,” J. ACM 9, 8–97 (1962).
[CrossRef]

1961 (1)

H. Wolter, “On Basic Analogies and Principal Differences Between Optical and Electronic Information,” Prog. Opt. 1, 155 (1961).
[CrossRef]

1960 (1)

R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” J. Basic Eng., ASME Trans. 82D, 35 (1960).
[CrossRef]

1950 (1)

The classic treatment of Wiener-Kolmogoroff filtering is in H.W. Bode, C. E. Shannon, “A Simplified Derivation of Linear Least-Square Smoothing and Prediction Theory,” Proc. IRE 38, 417 (1950).
[CrossRef]

1927 (1)

G. U. Yule, “On the Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer’s Sun-Spot Numbers,” Philos. Trans. R. Soc. London Ser. A 226, 267 (1927). This is the original reference on autoregressive modeling. A more recent review is in A. S.Willsky, Digital Signal Processing and Control, and Estimation Theory (MIT Press, Cambridge, 1979).
[CrossRef]

Bode, H.W.

The classic treatment of Wiener-Kolmogoroff filtering is in H.W. Bode, C. E. Shannon, “A Simplified Derivation of Linear Least-Square Smoothing and Prediction Theory,” Proc. IRE 38, 417 (1950).
[CrossRef]

Burg, J. P.

J. P. Burg, “Maximum Entropy Spectral Analysis,” paper presented at Thirty-Seventh Meeting of the Society of Exploration Geophysicists, Oklahoma City (1960);J. P. Burg, “Maximum Entropy Spectral Analysis,” Ph.D. Dissertation, Geophysics Department, Stanford U. (May1975).

Dainty, J. C.

J. C. Dainty, “Stellar Speckle Interferometry” (Springer-Verlag, New York, 1984).

Frieden, B. R.

B. R. Frieden, Probability, Statistical Optics and Data Testing (Springer-Verlag, New York, 1983).
[CrossRef]

Frost, V. S.

V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
[CrossRef]

Goodman, J. W.

J. W. Goodman, “Some Effects of Target-Induced Scintillation on Optical Radar Performance,” Proc. IEEE 53, 1688 (1965).
[CrossRef]

J. W. Goodman, Statistical Optics (Wiley, New York, 1985).

Harris, J. L.

Holtzman, J. C.

V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
[CrossRef]

Hunt, R. H.

Jansson, P. A.

Johnson, R. W.

J. E. Shore, R. W. Johnson, “Properties of Cross-Entropy Minimization,” IEEE Trans. Inf. Theory IT-27, 472 (1981).
[CrossRef]

Kalman, R. E.

R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” J. Basic Eng., ASME Trans. 82D, 35 (1960).
[CrossRef]

Phillips, D. L.

D. L. Phillips, “A Technique for the Numerical Solution of Certain Integral Equations of the First Kind,” J. ACM 9, 8–97 (1962).
[CrossRef]

Plyler, E. K.

Schell, A. C.

A. C. Schell, “Enhancing the Angular Resolution of Incoherent Sources,” Radio Electron. Eng. 29, 21 (1965).
[CrossRef]

Shanmugan, K. S.

V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
[CrossRef]

Shannon, C. E.

The classic treatment of Wiener-Kolmogoroff filtering is in H.W. Bode, C. E. Shannon, “A Simplified Derivation of Linear Least-Square Smoothing and Prediction Theory,” Proc. IRE 38, 417 (1950).
[CrossRef]

Shore, J. E.

J. E. Shore, R. W. Johnson, “Properties of Cross-Entropy Minimization,” IEEE Trans. Inf. Theory IT-27, 472 (1981).
[CrossRef]

Stiles, J. A.

V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
[CrossRef]

Tukey, J. W.

J. W. Tukey, Exploratory Data Analysis (Addison-Wesley, Reading, MA, 1977).

Van Trees, H. L.

H. L. Van Trees, Detection, Estimation and Modulation Theory, Part 1 (Wiley, New York, 1968).

Wainstein, L. A.

L. A. Wainstein, V. D. Zubakov, Extraction of Signals from Noise (Prentice-Hall, Englewood Cliffs, NJ, 1962).

Wolter, H.

H. Wolter, “On Basic Analogies and Principal Differences Between Optical and Electronic Information,” Prog. Opt. 1, 155 (1961).
[CrossRef]

Yule, G. U.

G. U. Yule, “On the Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer’s Sun-Spot Numbers,” Philos. Trans. R. Soc. London Ser. A 226, 267 (1927). This is the original reference on autoregressive modeling. A more recent review is in A. S.Willsky, Digital Signal Processing and Control, and Estimation Theory (MIT Press, Cambridge, 1979).
[CrossRef]

Zubakov, V. D.

L. A. Wainstein, V. D. Zubakov, Extraction of Signals from Noise (Prentice-Hall, Englewood Cliffs, NJ, 1962).

IEEE Trans. Inf. Theory (1)

J. E. Shore, R. W. Johnson, “Properties of Cross-Entropy Minimization,” IEEE Trans. Inf. Theory IT-27, 472 (1981).
[CrossRef]

IEEE Trans. Pattern Anal. Machine Intell. (1)

V. S. Frost, J. A. Stiles, K. S. Shanmugan, J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” in IEEE Trans. Pattern Anal. Machine Intell. PAMI-4, 157 (1982).
[CrossRef]

J. ACM (1)

D. L. Phillips, “A Technique for the Numerical Solution of Certain Integral Equations of the First Kind,” J. ACM 9, 8–97 (1962).
[CrossRef]

J. Basic Eng., ASME Trans. (1)

R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” J. Basic Eng., ASME Trans. 82D, 35 (1960).
[CrossRef]

J. Opt. Soc. Am. (2)

Philos. Trans. R. Soc. London Ser. A (1)

G. U. Yule, “On the Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer’s Sun-Spot Numbers,” Philos. Trans. R. Soc. London Ser. A 226, 267 (1927). This is the original reference on autoregressive modeling. A more recent review is in A. S.Willsky, Digital Signal Processing and Control, and Estimation Theory (MIT Press, Cambridge, 1979).
[CrossRef]

Proc. IEEE (1)

J. W. Goodman, “Some Effects of Target-Induced Scintillation on Optical Radar Performance,” Proc. IEEE 53, 1688 (1965).
[CrossRef]

Proc. IRE (1)

The classic treatment of Wiener-Kolmogoroff filtering is in H.W. Bode, C. E. Shannon, “A Simplified Derivation of Linear Least-Square Smoothing and Prediction Theory,” Proc. IRE 38, 417 (1950).
[CrossRef]

Prog. Opt. (1)

H. Wolter, “On Basic Analogies and Principal Differences Between Optical and Electronic Information,” Prog. Opt. 1, 155 (1961).
[CrossRef]

Radio Electron. Eng. (1)

A. C. Schell, “Enhancing the Angular Resolution of Incoherent Sources,” Radio Electron. Eng. 29, 21 (1965).
[CrossRef]

Other (8)

L. A. Wainstein, V. D. Zubakov, Extraction of Signals from Noise (Prentice-Hall, Englewood Cliffs, NJ, 1962).

H. L. Van Trees, Detection, Estimation and Modulation Theory, Part 1 (Wiley, New York, 1968).

B. R. Frieden, Probability, Statistical Optics and Data Testing (Springer-Verlag, New York, 1983).
[CrossRef]

J. W. Goodman, Statistical Optics (Wiley, New York, 1985).

J. P. Burg, “Maximum Entropy Spectral Analysis,” paper presented at Thirty-Seventh Meeting of the Society of Exploration Geophysicists, Oklahoma City (1960);J. P. Burg, “Maximum Entropy Spectral Analysis,” Ph.D. Dissertation, Geophysics Department, Stanford U. (May1975).

See, e.g., H. Stark, Ed. Image Recovery, Theory and Applications (Academic, Orlando, 1987).

J. C. Dainty, “Stellar Speckle Interferometry” (Springer-Verlag, New York, 1984).

J. W. Tukey, Exploratory Data Analysis (Addison-Wesley, Reading, MA, 1977).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Signal image (girl with hat) on left. Template image, strongly blurred, on right. The objective in the following demonstrations is to recover the signal image by using the information provided by the template as prior knowledge.

Fig. 2
Fig. 2

Gaussian noise case: (a) template image; (b) data image, SNR = 5.0; (c) MDT output.

Fig. 3
Fig. 3

Median window processed version of Fig. 2(b); to be compared with Fig. 2(c).

Fig. 4
Fig. 4

Poisson noise case: (a) template image; (b) data image, SNR = 2.0; (c) MDT output.

Fig. 5
Fig. 5

Exponential noise case: (a) template image; (b) data image, SNR = 1.0; (c) MDT output.

Tables (1)

Tables Icon

Table I Estimators and Solutions for Various Noise Types; Since Each Pixel is Processed the Same Way, the Pixel Subscript is Suppressed

Equations (51)

Equations on this page are rendered with MathJax. Learn more.

p ( R | i ) = maximum
p 0 ( i ) p ( R | i ) = maximum .
ln 1 p 0 ( i ) ln p ( R | i ) = minimum .
ln p ( R | i ) p ( R | t ) λ ln p ( R | i ) = minimum , R = i  .
p 0 ( i ) p 0 ( i | t ) = exp [ ln p ( R | i ) p ( R | t ) ] .
γ i t ( R ) p ( R | i ) p ( R | t )
ψ i t ln γ i t ,
P ( C ) ψ i t .
P ( C ) = ½ [ 1 + erf ( ψ i t / 2 ) ] .
P ( C ) = minimum .
ln p ( R | i ) p ( R | t ) = minimum ,
ln   p ( R | i ) = maximum ,
ln   p ( R | i ) = minimum .
i ^ { t as λ 0 , R as λ .
p ( R | i ) = A exp [ 1 2 σ 2 n ( R n i n ) 2 ]
n   ( i n t n ) 2 + λ   n   ( i n R n ) 2 = minimum .
i ^ = t + λ R 1 + λ .
p ( R | i ) = n exp ( i n ) i n R n R n ! .
p ( R | i ) p ( R | t ) = n exp ( t n i n ) ( i n / t n ) R n .
ln p ( R | i ) p ( R | t ) = n   ( t n i n ) + n   R n   ln ( i n / t n ) .
ln p ( R | i ) p ( R | t ) = n [ i n   ln ( i n / t n ) + ( t n i n ) ] .
n   [ i n   ln ( i n / t n ) + ( t n i n ) ] λ   n   ( R n   ln i n i n ) = minimum ,
ln ( i / t ) + λ ( 1 R / i ) = 0 T
d T / d i = i 1 + λ R i 2 .
Δ i = T ( d T / d i ) 1
p ( R | i ) = A   n i n k R n k 1   exp ( k R n / i n ) ,
ln p ( R | i ) p ( R | t ) =   k   n [ ln ( t n / i n ) + R n ( t n 1 i n 1 ) ] .
ln p ( R | i ) p ( R | t ) =   k n   [ ln ( t n / i n ) + i n / t n ] ,
( λ 1 ) n   ln i n + n   i n / t n + λ   R n / i n = minimum ,
( 1 λ ) i i 2 / t + λ R = 0
i ^ = ( 1 λ ) t 2 { 1 + [ 1 + 4 λ R t ( 1 λ ) 2 t 2 ] 1 / 2 } .
p ( R | i ) = n   R n 1   exp [ 1 2 σ 2 ( ln R n ln i n ) 2 ] ,
σ 2 ( ln R n ln i n ) 2 = ( ln R n ) 2 ( ln i n ) 2 .
ln p ( R | i ) p ( R | t ) = ( 2 σ 2 ) 1   n   [ ln 2 ( R n / t n ) ln 2 ( R n / i n ) ] .
ln p ( R | i ) p ( R | t ) = ( 2 σ 2 ) 1 n ln 2 ( i n / t n ) .
n   ln 2 ( i n / t n ) + λ n   ln 2 ( R n / i n ) = minimum ,
i ^ = ( R λ t ) 1 1 + λ .
ln p ( R | o ) p ( R | t )
  ln p ( R | i ) = d R p ( R )   ln p ( R | i ) ,
  ln p ( R | t ) = d R p ( R )   ln p ( R | t ) .
ψ   ln p ( R | i ) p ( R | t ) = d R p ( R ) ln p ( R | i ) p ( R | t ) ,
R = i = d RR p ( R ) .
ψ = d R p ( R | i ) ℓn p ( R | i ) p ( R | t ) .
d x p ( x )   ln p ( x ) q ( x ) = minimum ,
d x p ( x )   ln p ( x ) d x p ( x )   ln q ( x ) + λ ( d x p ( x ) 1 ) = minimum .
p = 0 ,
= p   ln p p   ln q + λ p .
p = q   exp ( 1 λ ) .
exp ( 1 λ ) = 1 ,
p ( x ) = q ( x ) .
i = t ,

Metrics