Abstract

In this paper, we present an algorithm for the restoration of images with an unknown, spatially-varying blur. Existing computational methods for image restoration require the assumption that the blur is known and/or spatially-invariant. Our algorithm uses a combination of techniques. First, we section the image, and then treat the sections as a sequence of frames whose unknown PSFs are correlated and approximately spatially-invariant. To estimate the PSFs in each section, phase diversity is used. With the PSF estimates in hand, we then use a technique by Nagy and O’Leary for the restoration of images with a known, spatially-varying blur to restore the image globally. Test results on star cluster data are presented.

© 2006 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Satellite image restoration in the context of a spatially varying point spread function

Nasreddine Hajlaoui, Caroline Chaux, Guillaume Perrin, Frédéric Falzon, and Amel Benazza-Benyahia
J. Opt. Soc. Am. A 27(6) 1473-1481 (2010)

Multiframe blind deconvolution of heavily blurred astronomical images

Yulia V. Zhulina
Appl. Opt. 45(28) 7342-7352 (2006)

Restoring atmospheric-turbulence-degraded images

Md. Hasan Furhad, Murat Tahtali, and Andrew Lambert
Appl. Opt. 55(19) 5082-5090 (2016)

More Recommended Articles

References

  • View by:
  • Article Order
  • |
  • Year
  • |
  • Author
  • |
  • Publication
  1. J. Biretta, “WFPC and WFPC 2 instrumental characteristics,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 224–235 (Space Telescope Science Institute, Baltimore, MD, 1994).
  2. H. J. Trussell and S. Fogel, “Identification and restoration of spatially variant motion blurs in sequential images,” IEEE Trans. Image Proc. 1, 123–126 (1992).
    [Crossref]
  3. R. G. Paxman, B. J. Thelen, and J. H. Seldin, “Phase-diversity correction of turbulence-induced space-variant blur,” Opt. Lett. 19(16), 1231–1233 (1994).
    [Crossref] [PubMed]
  4. M. C. Roggemann and B. Welsh, Imaging Through Turbulence (CRC Press, Boca Raton, FL, 1996).
  5. M. Faisal, A. D. Lanterman, D. L. Snyder, and R. L. White, “Implementation of a Modified Richardson-Lucy Method for Image Restoration on a Massively Parallel Computer to Compensate for Space-Variant Point Spread Function of a Charge-Coupled Device Camera,” J. Opt. Soc. Am. A 12, 2593–2603 (1995).
    [Crossref]
  6. A. F. Boden, D. C. Redding, R. J. Hanisch, and J. Mo, “Massively Parallel Spatially-Variant Maximum Likelihood Restoration of Hubble Space Telescope Imagery,” J. Opt. Soc. Am. A 13, 1537–1545 (1996).
    [Crossref]
  7. J. G. Nagy and D. P. O’Leary, “Restoring images degraded by spatially-variant blur,” SIAM J. Sci. Comput. 19, 1063–1082 (1998).
    [Crossref]
  8. R. A. Gonsalves, “Phase diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).
  9. C. R. Vogel, T. Chan, and R. J. Plemmons, “Fast algorithms for phase diversity-based blind deconvolution,” in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).
  10. R. G. Paxman, T. Schulz, and J. Fienup, “Joint estimation of object and aberrations by using phase diversity,” J. Opt. Soc. Am. A 9, 1072–1085 (1992).
    [Crossref]
  11. L. Gilles, C. R. Vogel, and J. M. Bardsley, “Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics,” Inverse Probl. 18, 237–252 (2002).
    [Crossref]
  12. J. Nocedal and S. J. Wright, Numerical Optimization (Springer-Verlag, New York, 1999).
    [Crossref]
  13. H.-M. Adorf, “Towards HST restoration with space-variant PSF, cosmic rays and other missing data,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 72–78 (1994).
  14. D. A. Fish, J. Grochmalicki, and E. R. Pike, “Scanning singular-value-decomposition method for restoration of images with space-variant blur,” J. Opt. Soc. Am. A 13, 1–6 (1996).
    [Crossref]
  15. H. J. Trussell and B. R. Hunt, “Image Restoration of Space-Variant Blurs by Sectional Methods,” IEEE Trans. Acoust. Speech, Signal Processing 26, 608–609 (1978).
    [Crossref]
  16. J. G. Nagy and D. P. O’Leary, “Fast iterative image restoration with a spatially varying PSF,” in Advanced Signal Processing Algorithms, Architectures, and Implementations VII, F. T. Luk, ed., vol. 3162, pp. 388–399 (SPIE, 1997).
  17. H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).
  18. P. C. Hansen, Rank-deficient and discrete ill-posed problems (SIAM, Philadelphia, PA, 1997).
  19. C. R. Vogel, Computational Methods for Inverse Problems (SIAM, Philadelphia, PA, 2002).
    [Crossref]

2002 (1)

L. Gilles, C. R. Vogel, and J. M. Bardsley, “Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics,” Inverse Probl. 18, 237–252 (2002).
[Crossref]

1998 (1)

J. G. Nagy and D. P. O’Leary, “Restoring images degraded by spatially-variant blur,” SIAM J. Sci. Comput. 19, 1063–1082 (1998).
[Crossref]

1996 (2)

A. F. Boden, D. C. Redding, R. J. Hanisch, and J. Mo, “Massively Parallel Spatially-Variant Maximum Likelihood Restoration of Hubble Space Telescope Imagery,” J. Opt. Soc. Am. A 13, 1537–1545 (1996).
[Crossref]

D. A. Fish, J. Grochmalicki, and E. R. Pike, “Scanning singular-value-decomposition method for restoration of images with space-variant blur,” J. Opt. Soc. Am. A 13, 1–6 (1996).
[Crossref]

1995 (1)

1994 (2)

R. G. Paxman, B. J. Thelen, and J. H. Seldin, “Phase-diversity correction of turbulence-induced space-variant blur,” Opt. Lett. 19(16), 1231–1233 (1994).
[Crossref] [PubMed]

H.-M. Adorf, “Towards HST restoration with space-variant PSF, cosmic rays and other missing data,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 72–78 (1994).

1992 (2)

H. J. Trussell and S. Fogel, “Identification and restoration of spatially variant motion blurs in sequential images,” IEEE Trans. Image Proc. 1, 123–126 (1992).
[Crossref]

R. G. Paxman, T. Schulz, and J. Fienup, “Joint estimation of object and aberrations by using phase diversity,” J. Opt. Soc. Am. A 9, 1072–1085 (1992).
[Crossref]

1982 (1)

R. A. Gonsalves, “Phase diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).

1978 (1)

H. J. Trussell and B. R. Hunt, “Image Restoration of Space-Variant Blurs by Sectional Methods,” IEEE Trans. Acoust. Speech, Signal Processing 26, 608–609 (1978).
[Crossref]

Adorf, H.-M.

H.-M. Adorf, “Towards HST restoration with space-variant PSF, cosmic rays and other missing data,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 72–78 (1994).

Bardsley, J. M.

L. Gilles, C. R. Vogel, and J. M. Bardsley, “Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics,” Inverse Probl. 18, 237–252 (2002).
[Crossref]

Biretta, J.

J. Biretta, “WFPC and WFPC 2 instrumental characteristics,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 224–235 (Space Telescope Science Institute, Baltimore, MD, 1994).

Boden, A. F.

Chan, T.

C. R. Vogel, T. Chan, and R. J. Plemmons, “Fast algorithms for phase diversity-based blind deconvolution,” in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).

Engl, H. W.

H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).

Faisal, M.

Fienup, J.

Fish, D. A.

D. A. Fish, J. Grochmalicki, and E. R. Pike, “Scanning singular-value-decomposition method for restoration of images with space-variant blur,” J. Opt. Soc. Am. A 13, 1–6 (1996).
[Crossref]

Fogel, S.

H. J. Trussell and S. Fogel, “Identification and restoration of spatially variant motion blurs in sequential images,” IEEE Trans. Image Proc. 1, 123–126 (1992).
[Crossref]

Gilles, L.

L. Gilles, C. R. Vogel, and J. M. Bardsley, “Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics,” Inverse Probl. 18, 237–252 (2002).
[Crossref]

Gonsalves, R. A.

R. A. Gonsalves, “Phase diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).

Grochmalicki, J.

D. A. Fish, J. Grochmalicki, and E. R. Pike, “Scanning singular-value-decomposition method for restoration of images with space-variant blur,” J. Opt. Soc. Am. A 13, 1–6 (1996).
[Crossref]

Hanisch, R. J.

Hanke, M.

H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).

Hansen, P. C.

P. C. Hansen, Rank-deficient and discrete ill-posed problems (SIAM, Philadelphia, PA, 1997).

Hunt, B. R.

H. J. Trussell and B. R. Hunt, “Image Restoration of Space-Variant Blurs by Sectional Methods,” IEEE Trans. Acoust. Speech, Signal Processing 26, 608–609 (1978).
[Crossref]

Lanterman, A. D.

Mo, J.

Nagy, J. G.

J. G. Nagy and D. P. O’Leary, “Restoring images degraded by spatially-variant blur,” SIAM J. Sci. Comput. 19, 1063–1082 (1998).
[Crossref]

J. G. Nagy and D. P. O’Leary, “Fast iterative image restoration with a spatially varying PSF,” in Advanced Signal Processing Algorithms, Architectures, and Implementations VII, F. T. Luk, ed., vol. 3162, pp. 388–399 (SPIE, 1997).

Neubauer, A.

H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).

Nocedal, J.

J. Nocedal and S. J. Wright, Numerical Optimization (Springer-Verlag, New York, 1999).
[Crossref]

O’Leary, D. P.

J. G. Nagy and D. P. O’Leary, “Restoring images degraded by spatially-variant blur,” SIAM J. Sci. Comput. 19, 1063–1082 (1998).
[Crossref]

J. G. Nagy and D. P. O’Leary, “Fast iterative image restoration with a spatially varying PSF,” in Advanced Signal Processing Algorithms, Architectures, and Implementations VII, F. T. Luk, ed., vol. 3162, pp. 388–399 (SPIE, 1997).

Paxman, R. G.

Pike, E. R.

D. A. Fish, J. Grochmalicki, and E. R. Pike, “Scanning singular-value-decomposition method for restoration of images with space-variant blur,” J. Opt. Soc. Am. A 13, 1–6 (1996).
[Crossref]

Plemmons, R. J.

C. R. Vogel, T. Chan, and R. J. Plemmons, “Fast algorithms for phase diversity-based blind deconvolution,” in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).

Redding, D. C.

Roggemann, M. C.

M. C. Roggemann and B. Welsh, Imaging Through Turbulence (CRC Press, Boca Raton, FL, 1996).

Schulz, T.

Seldin, J. H.

Snyder, D. L.

Thelen, B. J.

Trussell, H. J.

H. J. Trussell and S. Fogel, “Identification and restoration of spatially variant motion blurs in sequential images,” IEEE Trans. Image Proc. 1, 123–126 (1992).
[Crossref]

H. J. Trussell and B. R. Hunt, “Image Restoration of Space-Variant Blurs by Sectional Methods,” IEEE Trans. Acoust. Speech, Signal Processing 26, 608–609 (1978).
[Crossref]

Vogel, C. R.

L. Gilles, C. R. Vogel, and J. M. Bardsley, “Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics,” Inverse Probl. 18, 237–252 (2002).
[Crossref]

C. R. Vogel, T. Chan, and R. J. Plemmons, “Fast algorithms for phase diversity-based blind deconvolution,” in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).

C. R. Vogel, Computational Methods for Inverse Problems (SIAM, Philadelphia, PA, 2002).
[Crossref]

Welsh, B.

M. C. Roggemann and B. Welsh, Imaging Through Turbulence (CRC Press, Boca Raton, FL, 1996).

White, R. L.

Wright, S. J.

J. Nocedal and S. J. Wright, Numerical Optimization (Springer-Verlag, New York, 1999).
[Crossref]

IEEE Trans. Acoust. Speech, Signal Processing (1)

H. J. Trussell and B. R. Hunt, “Image Restoration of Space-Variant Blurs by Sectional Methods,” IEEE Trans. Acoust. Speech, Signal Processing 26, 608–609 (1978).
[Crossref]

IEEE Trans. Image Proc. (1)

H. J. Trussell and S. Fogel, “Identification and restoration of spatially variant motion blurs in sequential images,” IEEE Trans. Image Proc. 1, 123–126 (1992).
[Crossref]

Inverse Probl. (1)

L. Gilles, C. R. Vogel, and J. M. Bardsley, “Computational Methods for a Large-Scale Inverse Problem Arising in Atmospheric Optics,” Inverse Probl. 18, 237–252 (2002).
[Crossref]

J. Opt. Soc. Am. A (4)

Opt. Eng. (1)

R. A. Gonsalves, “Phase diversity in adaptive optics,” Opt. Eng. 21, 829–832 (1982).

Opt. Lett. (1)

SIAM J. Sci. Comput. (1)

J. G. Nagy and D. P. O’Leary, “Restoring images degraded by spatially-variant blur,” SIAM J. Sci. Comput. 19, 1063–1082 (1998).
[Crossref]

Other (9)

M. C. Roggemann and B. Welsh, Imaging Through Turbulence (CRC Press, Boca Raton, FL, 1996).

C. R. Vogel, T. Chan, and R. J. Plemmons, “Fast algorithms for phase diversity-based blind deconvolution,” in Adaptive Optical System Technologies, vol. 3353 (SPIE, 1998).

J. Biretta, “WFPC and WFPC 2 instrumental characteristics,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 224–235 (Space Telescope Science Institute, Baltimore, MD, 1994).

J. Nocedal and S. J. Wright, Numerical Optimization (Springer-Verlag, New York, 1999).
[Crossref]

H.-M. Adorf, “Towards HST restoration with space-variant PSF, cosmic rays and other missing data,” in The Restoration of HST Images and Spectra II, R. J. Hanisch and R. L. White, eds., pp. 72–78 (1994).

J. G. Nagy and D. P. O’Leary, “Fast iterative image restoration with a spatially varying PSF,” in Advanced Signal Processing Algorithms, Architectures, and Implementations VII, F. T. Luk, ed., vol. 3162, pp. 388–399 (SPIE, 1997).

H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems (Kluwer Academic Publishers, Dordrecht, 2000).

P. C. Hansen, Rank-deficient and discrete ill-posed problems (SIAM, Philadelphia, PA, 1997).

C. R. Vogel, Computational Methods for Inverse Problems (SIAM, Philadelphia, PA, 2002).
[Crossref]

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1.
Fig. 1.

Simulated image of a star field.

Download Full Size | PPT Slide | PDF

Fig. 2.
Fig. 2.

Pupil function, two sample phases and their corresponding PSFs.

Download Full Size | PPT Slide | PDF

Fig. 3.
Fig. 3.

Simulated space variant blur of star field image; a linear scale is used to display the image on the left, and a log scale is used to display the image on the right.

Download Full Size | PPT Slide | PDF

Fig. 4.
Fig. 4.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur. These results were computed using a noise free blurred image.

Download Full Size | PPT Slide | PDF

Fig. 5.
Fig. 5.

Reconstructions computed by the conjugate gradient algorithm, using accurate approximations of the PSF, and with a noise free blurred image. For comparison, we also show the true image and the true image convolved with the diffraction-limited PSF.

Download Full Size | PPT Slide | PDF

Fig. 6.
Fig. 6.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur. These results were computed using a noisy (Poisson and Gaussian) blurred image.

Download Full Size | PPT Slide | PDF

Fig. 7.
Fig. 7.

Reconstructions computed by the conjugate gradient algorithm, using accurate approximations of the PSF, with Poisson and Gaussian noise added to the blurred image. For comparison, we also show the true image and the true image convolved with the diffraction-limited PSF.

Download Full Size | PPT Slide | PDF

Fig. 8.
Fig. 8.

Example of adaptive refinement of region partitions. The left plot shows an initial partitioning of an image, the middle shows a refinement of the partitioning, and the right plot shows what might happen in an adaptive approach where only certain subregions are refined further.

Download Full Size | PPT Slide | PDF

Fig. 9.
Fig. 9.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur. The PSFs used to approximate the space variant blur were computed using phase diversity-based blind deconvolution. These results were computed using noise free blurred image data.

Download Full Size | PPT Slide | PDF

Fig. 10.
Fig. 10.

Reconstructions computed by the conjugate gradient algorithm, using PSFs computed from a phase diversity blind deconvolution algorithm. The blurred image data in this case is noise free.

Download Full Size | PPT Slide | PDF

Fig. 11.
Fig. 11.

Relative errors at each iteration of the conjugate gradient method, using increasingly more accurate approximations of the spatially variant blur, and with a noisy (Poisson and Gaussian) blurred image.

Download Full Size | PPT Slide | PDF

Fig. 12.
Fig. 12.

Reconstructions computed by the conjugate gradient algorithm, using PSFs computed from a phase diversity-based blind deconvolution algorithm, and with a noisy (Poisson and Gaussian) blurred image.

Download Full Size | PPT Slide | PDF

Equations (35)

Equations on this page are rendered with MathJax. Learn more.

d = Sf + η ,
Sf u v = s * f u v := ∫ − ∞ ∞ ∫ − ∞ ∞ s u − ξ v − η f ξ η d ξ d η .
s ( u , v ; ξ , η ) = ∑ n = 1 P s n u − ξ v − η c n ξ η .
d = ∑ n = 1 P s n * f n + η ,
s [ ϕ ] = ∣ ℱ − 1 ( p e ιϕ ) ∣ 2 ,
( ℱ h ) ( y ) = ∫ ∫ IR 2 h ( x ) e − ι 2 π x ∙ y d x , y ∈ IR 2 .
d k = s [ ϕ + θ k ] * f + η k , k = 1 , … , K ,
θ k x 1 x 2 = b k ( x 1 2 + x 2 2 ) ,
J data [ ϕ , f ] = 1 2 K ∑ k = 1 K ∥ s [ ϕ + θ k ] * f − d k , t ∥ 2 .
J data [ ϕ , F ] = 1 2 K ∑ k = 1 K ∥ S k [ ϕ ] F − D k ∥ 2 .
J full [ ϕ , f ] = J data [ ϕ , f ] + γ J object [ f ] + α J phase [ ϕ ] .
Φ ( ω ) = C 1 ( C 2 + ∣ ω ∣ 2 ) 11 6 ,
J phase [ ϕ ] = 1 2 〈 Φ − 1 ℱ { ϕ } , ℱ { ϕ } 〉 ,
J object [ f ] = 1 2 ∥ f ∥ 2 = 1 2 ∥ f ∥ 2 .
F = P [ ϕ ] * Q [ ϕ ] ,
P [ ϕ ] = ∑ k D k * S k [ ϕ ] , Q [ ϕ ] = γ + ∑ k ∣ S k [ ϕ ] ∣ 2 .
J ϕ = J reduced data [ ϕ ] + α J phase [ ϕ ] ,
J reduced data [ ϕ ] = ∑ k ∥ D k ∥ 2 − 〈 P [ ϕ ] Q [ ϕ ] , P [ ϕ ] 〉 .
ν := 0 ;
ϕ 0 := initial guess ;
begin quasi-Newton iterations
g ν := grad J ( ϕ ν ) ; % compute gradient
B ν := SPD approximation to Hess J ( ϕ ν ) ;
d ν := − B ν − 1 g ν ; % compute quasi-Newton step
τ ν := argmin τ > 0 J ( ϕ ν + τ d ν ) ; % line search
ϕ ν + 1 := ϕ ν + τ ν d ν ; % update approximate solution
ν := ν + 1 ;
end quasi-Newton iterations
s v = def ϕ v + 1 − ϕ v ,
y v = def grad J ( ϕ v + 1 ) − grad J ( ϕ v ) .
B v + 1 − 1 = ( I − y v s v T y v T s v ) B v − 1 ( I − s v y v T y v T s v ) + s v s v T y v T s v , v = 0,1 , … .
M v ψ = ℱ − 1 ( Φ − 1 ℱ ( ψ ) ) ,
S = ∑ i = 1 p ∑ i = 1 p D ij s ij ,
d = ∑ i = 1 p ∑ i = 1 p D ij ( s ij * f ) + n ,
∥ f true − f k ∥ 2 ∥ f true ∥ 2

Metrics