Abstract

In this paper, we present an algorithm for the restoration of images with an unknown, spatially-varying blur. Existing computational methods for image restoration require the assumption that the blur is known and/or spatially-invariant. Our algorithm uses a combination of techniques. First, we section the image, and then treat the sections as a sequence of frames whose unknown PSFs are correlated and approximately spatially-invariant. To estimate the PSFs in each section, phase diversity is used. With the PSF estimates in hand, we then use a technique by Nagy and O’Leary for the restoration of images with a known, spatially-varying blur to restore the image globally. Test results on star cluster data are presented.

© 2006 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Biretta, "WFPC and WFPC 2 instrumental characteristics," in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 224-235 (Space Telescope Science Institute, Baltimore, MD, 1994).
  2. H. J. Trussell and S. Fogel, "Identification and restoration of spatially variant motion blurs in sequential images," IEEE Trans. Image Proc. 1, 123-126 (1992).
    [CrossRef]
  3. R. G. Paxman, B. J. Thelen, and J. H. Seldin, "Phase-diversity correction of turbulence-induced space-variant blur," Opt. Lett. 19, 1231-1233 (1994).
    [CrossRef] [PubMed]
  4. M. C. Roggemann and B. Welsh, Imaging Through Turbulence (CRC Press, Boca Raton, FL, 1996).
  5. M. Faisal, A. D. Lanterman, D. L. Snyder, and R. L. White, "Implementation of a Modified Richardson-Lucy Method for Image Restoration on a Massively Parallel Computer to Compensate for Space-Variant Point Spread Function of a Charge-Coupled Device Camera," J. Opt. Soc. Am. A 12, 2593-2603 (1995).
    [CrossRef]
  6. A. F. Boden, D. C. Redding, R. J. Hanisch, and J. Mo, "Massively Parallel Spatially-Variant Maximum Likelihood Restoration of Hubble Space Telescope Imagery," J. Opt. Soc. Am. A 13, 1537-1545 (1996).
    [CrossRef]
  7. J. G. Nagy and D. P. O’Leary, "Restoring images degraded by spatially-variant blur," SIAM J. Sci. Comput. 19, 1063-1082 (1998).
    [CrossRef]
  8. R. A. Gonsalves, "Phase diversity in adaptive optics," Opt. Eng. 21, 829-832 (1982).
  9. C. R. Vogel, T. Chan, and R. J. Plemmons, "Fast algorithms for phase diversity-based blind deconvolution," in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).
  10. R. G. Paxman, T. Schulz, and J. Fienup, "Joint estimation of object and aberrations by using phase diversity," J. Opt. Soc. Am. A 9, 1072-1085 (1992).
    [CrossRef]
  11. L. Gilles, C. R. Vogel, and J. M. Bardsley, "Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics," Inverse Probl. 18, 237-252 (2002).
    [CrossRef]
  12. J. Nocedal and S. J. Wright, Numerical Optimization (Springer-Verlag, New York, 1999).
    [CrossRef]
  13. H.-M. Adorf, "Towards HST restoration with space-variant PSF, cosmic rays and other missing data," in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 72-78 (1994).
  14. H. J. Trussell and B. R. Hunt, "Image Restoration of Space-Variant Blurs by Sectional Methods," IEEE Trans. Acoust.Speech, Signal Processing 26, 608-609 (1978).
    [CrossRef]
  15. J. G. Nagy and D. P. O’Leary, "Fast iterative image restoration with a spatially varying PSF," in Advanced Signal Processing Algorithms, Architectures, and Implementations VII, F. T. Luk, ed., vol. 3162, pp. 388-399 (SPIE, 1997).
  16. H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).
  17. P. C. Hansen, Rank-deficient and discrete ill-posed problems (SIAM, Philadelphia, PA, 1997).
  18. C. R. Vogel, Computational Methods for Inverse Problems (SIAM, Philadelphia, PA, 2002).
    [CrossRef]

2002 (1)

L. Gilles, C. R. Vogel, and J. M. Bardsley, "Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics," Inverse Probl. 18, 237-252 (2002).
[CrossRef]

1998 (1)

J. G. Nagy and D. P. O’Leary, "Restoring images degraded by spatially-variant blur," SIAM J. Sci. Comput. 19, 1063-1082 (1998).
[CrossRef]

1996 (1)

1995 (1)

1994 (1)

1992 (2)

H. J. Trussell and S. Fogel, "Identification and restoration of spatially variant motion blurs in sequential images," IEEE Trans. Image Proc. 1, 123-126 (1992).
[CrossRef]

R. G. Paxman, T. Schulz, and J. Fienup, "Joint estimation of object and aberrations by using phase diversity," J. Opt. Soc. Am. A 9, 1072-1085 (1992).
[CrossRef]

1982 (1)

R. A. Gonsalves, "Phase diversity in adaptive optics," Opt. Eng. 21, 829-832 (1982).

1978 (1)

H. J. Trussell and B. R. Hunt, "Image Restoration of Space-Variant Blurs by Sectional Methods," IEEE Trans. Acoust.Speech, Signal Processing 26, 608-609 (1978).
[CrossRef]

Bardsley, J. M.

L. Gilles, C. R. Vogel, and J. M. Bardsley, "Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics," Inverse Probl. 18, 237-252 (2002).
[CrossRef]

Boden, A. F.

Faisal, M.

Fienup, J.

Fogel, S.

H. J. Trussell and S. Fogel, "Identification and restoration of spatially variant motion blurs in sequential images," IEEE Trans. Image Proc. 1, 123-126 (1992).
[CrossRef]

Gilles, L.

L. Gilles, C. R. Vogel, and J. M. Bardsley, "Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics," Inverse Probl. 18, 237-252 (2002).
[CrossRef]

Gonsalves, R. A.

R. A. Gonsalves, "Phase diversity in adaptive optics," Opt. Eng. 21, 829-832 (1982).

Hanisch, R. J.

Hunt, B. R.

H. J. Trussell and B. R. Hunt, "Image Restoration of Space-Variant Blurs by Sectional Methods," IEEE Trans. Acoust.Speech, Signal Processing 26, 608-609 (1978).
[CrossRef]

Lanterman, A. D.

Mo, J.

Nagy, J. G.

J. G. Nagy and D. P. O’Leary, "Restoring images degraded by spatially-variant blur," SIAM J. Sci. Comput. 19, 1063-1082 (1998).
[CrossRef]

O’Leary, D. P.

J. G. Nagy and D. P. O’Leary, "Restoring images degraded by spatially-variant blur," SIAM J. Sci. Comput. 19, 1063-1082 (1998).
[CrossRef]

Paxman, R. G.

Redding, D. C.

Schulz, T.

Seldin, J. H.

Snyder, D. L.

Thelen, B. J.

Trussell, H. J.

H. J. Trussell and S. Fogel, "Identification and restoration of spatially variant motion blurs in sequential images," IEEE Trans. Image Proc. 1, 123-126 (1992).
[CrossRef]

H. J. Trussell and B. R. Hunt, "Image Restoration of Space-Variant Blurs by Sectional Methods," IEEE Trans. Acoust.Speech, Signal Processing 26, 608-609 (1978).
[CrossRef]

Vogel, C. R.

L. Gilles, C. R. Vogel, and J. M. Bardsley, "Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics," Inverse Probl. 18, 237-252 (2002).
[CrossRef]

White, R. L.

IEEE Trans. Image Proc. (1)

H. J. Trussell and S. Fogel, "Identification and restoration of spatially variant motion blurs in sequential images," IEEE Trans. Image Proc. 1, 123-126 (1992).
[CrossRef]

Inverse Probl. (1)

L. Gilles, C. R. Vogel, and J. M. Bardsley, "Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics," Inverse Probl. 18, 237-252 (2002).
[CrossRef]

J. Opt. Soc. Am. A (3)

Opt. Eng. (1)

R. A. Gonsalves, "Phase diversity in adaptive optics," Opt. Eng. 21, 829-832 (1982).

Opt. Lett. (1)

SIAM J. Sci. Comput. (1)

J. G. Nagy and D. P. O’Leary, "Restoring images degraded by spatially-variant blur," SIAM J. Sci. Comput. 19, 1063-1082 (1998).
[CrossRef]

Speech, Signal Processing (1)

H. J. Trussell and B. R. Hunt, "Image Restoration of Space-Variant Blurs by Sectional Methods," IEEE Trans. Acoust.Speech, Signal Processing 26, 608-609 (1978).
[CrossRef]

Other (9)

J. G. Nagy and D. P. O’Leary, "Fast iterative image restoration with a spatially varying PSF," in Advanced Signal Processing Algorithms, Architectures, and Implementations VII, F. T. Luk, ed., vol. 3162, pp. 388-399 (SPIE, 1997).

H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).

P. C. Hansen, Rank-deficient and discrete ill-posed problems (SIAM, Philadelphia, PA, 1997).

C. R. Vogel, Computational Methods for Inverse Problems (SIAM, Philadelphia, PA, 2002).
[CrossRef]

J. Nocedal and S. J. Wright, Numerical Optimization (Springer-Verlag, New York, 1999).
[CrossRef]

H.-M. Adorf, "Towards HST restoration with space-variant PSF, cosmic rays and other missing data," in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 72-78 (1994).

M. C. Roggemann and B. Welsh, Imaging Through Turbulence (CRC Press, Boca Raton, FL, 1996).

C. R. Vogel, T. Chan, and R. J. Plemmons, "Fast algorithms for phase diversity-based blind deconvolution," in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).

J. Biretta, "WFPC and WFPC 2 instrumental characteristics," in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 224-235 (Space Telescope Science Institute, Baltimore, MD, 1994).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1.
Fig. 1.

Simulated image of a star field.

Fig. 2.
Fig. 2.

Pupil function, two sample phases and their corresponding PSFs.

Fig. 3.
Fig. 3.

Simulated space variant blur of star field image; a linear scale is used to display the image on the left, and a log scale is used to display the image on the right.

Fig. 4.
Fig. 4.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur. These results were computed using a noise free blurred image.

Fig. 5.
Fig. 5.

Reconstructions computed by the conjugate gradient algorithm, using accurate approximations of the PSF, and with a noise free blurred image. For comparison, we also show the true image and the true image convolved with the diffraction-limited PSF.

Fig. 6.
Fig. 6.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur. These results were computed using a noisy (Poisson and Gaussian) blurred image.

Fig. 7.
Fig. 7.

Reconstructions computed by the conjugate gradient algorithm, using accurate approximations of the PSF, with Poisson and Gaussian noise added to the blurred image. For comparison, we also show the true image and the true image convolved with the diffraction-limited PSF.

Fig. 8.
Fig. 8.

Example of adaptive refinement of region partitions. The left plot shows an initial partitioning of an image, the middle shows a refinement of the partitioning, and the right plot shows what might happen in an adaptive approach where only certain subregions are refined further.

Fig. 9.
Fig. 9.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur. The PSFs used to approximate the space variant blur were computed using phase diversity-based blind deconvolution. These results were computed using noise free blurred image data.

Fig. 10.
Fig. 10.

Reconstructions computed by the conjugate gradient algorithm, using PSFs computed from a phase diversity blind deconvolution algorithm. The blurred image data in this case is noise free.

Fig. 11.
Fig. 11.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur, and with a noisy (Poisson and Gaussian) blurred image.

Fig. 12.
Fig. 12.

Reconstructions computed by the conjugate gradient algorithm, using PSFs computed from a phase diversity-based blind deconvolution algorithm, and with a noisy (Poisson and Gaussian) blurred image.

Equations (35)

Equations on this page are rendered with MathJax. Learn more.

d = Sf + η ,
Sf u v = s * f u v := s u ξ v η f ξ η d ξ d η .
s ( u , v ; ξ , η ) = n = 1 P s n u ξ v η c n ξ η .
d = n = 1 P s n * f n + η ,
s [ ϕ ] = 1 ( p e ιϕ ) 2 ,
( h ) ( y ) = IR 2 h ( x ) e ι 2 π x y d x , y IR 2 .
d k = s [ ϕ + θ k ] * f + η k , k = 1 , , K ,
θ k x 1 x 2 = b k ( x 1 2 + x 2 2 ) ,
J data [ ϕ , f ] = 1 2 K k = 1 K s [ ϕ + θ k ] * f d k , t 2 .
J data [ ϕ , F ] = 1 2 K k = 1 K S k [ ϕ ] F D k 2 .
J full [ ϕ , f ] = J data [ ϕ , f ] + γ J object [ f ] + α J phase [ ϕ ] .
Φ ( ω ) = C 1 ( C 2 + ω 2 ) 11 6 ,
J phase [ ϕ ] = 1 2 Φ 1 { ϕ } , { ϕ } ,
J object [ f ] = 1 2 f 2 = 1 2 f 2 .
F = P [ ϕ ] * Q [ ϕ ] ,
P [ ϕ ] = k D k * S k [ ϕ ] , Q [ ϕ ] = γ + k S k [ ϕ ] 2 .
J ϕ = J reduced data [ ϕ ] + α J phase [ ϕ ] ,
J reduced data [ ϕ ] = k D k 2 P [ ϕ ] Q [ ϕ ] , P [ ϕ ] .
ν := 0 ;
ϕ 0 := initial guess ;
begin quasi-Newton iterations
g ν := grad J ( ϕ ν ) ; % compute gradient
B ν := SPD approximation to Hess J ( ϕ ν ) ;
d ν := B ν 1 g ν ; % compute quasi-Newton step
τ ν := argmin τ > 0 J ( ϕ ν + τ d ν ) ; % line search
ϕ ν + 1 := ϕ ν + τ ν d ν ; % update approximate solution
ν := ν + 1 ;
end quasi-Newton iterations
s v = def ϕ v + 1 ϕ v ,
y v = def grad J ( ϕ v + 1 ) grad J ( ϕ v ) .
B v + 1 1 = ( I y v s v T y v T s v ) B v 1 ( I s v y v T y v T s v ) + s v s v T y v T s v , v = 0,1 , .
M v ψ = 1 ( Φ 1 ( ψ ) ) ,
S = i = 1 p i = 1 p D ij s ij ,
d = i = 1 p i = 1 p D ij ( s ij * f ) + n ,
f true f k 2 f true 2

Metrics