Abstract

In this paper, we propose an algorithm for image restoration based on fusing nonstationary edge-preserving priors. We develop a Bayesian modeling followed by an evidence approximation inference approach for deriving the analytic foundations of the proposed restoration method. Through a series of approximations, the final implementation of the proposed image restoration algorithm is iterative and takes advantage of the Fourier domain. Simulation results over a variety of blurred and noisy standard test images indicate that the presented method comfortably surpasses the current state-of-the-art image restoration for compactly supported degradations. We finally present experimental results by digitally refocusing images captured with controlled defocus, successfully confirming the ability of the proposed restoration algorithm in recovering extra features and rich details, while still preserving edges.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Banham and A. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997).
    [CrossRef]
  2. H. C. Andrews and B. R. Hunt, Digital Image Restoration (Prentice-Hall, 1977).
  3. M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging (Taylor & Francis, 1998).
  4. A. Katsaggelos, S. Babacan, and C.-J. Tsai, “Iterative image restoration,” in The Essential Guide to Image Processing, A. Bovik, ed. (Elsevier, 2009), Chap. 15.
  5. R. Molina, “On the hierarchical Bayesian approach to image restoration: applications to astronomical images,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 1122–1128 (1994).
    [CrossRef]
  6. R. Molina, A. Katsaggelos, and J. Mateos, “Bayesian and regularization methods for hyperparameter estimation in image restoration,” IEEE Trans. Image Process. 8, 231–246 (1999).
    [CrossRef]
  7. D. Tzikas, A. Likas, and N. Galatsanos, “The variational approximation for Bayesian inference,” IEEE Signal Process. Mag. 25(6), 131–146 (2008).
    [CrossRef]
  8. S. Babacan, R. Molina, and A. Katsaggelos, “Parameter estimation in tv image restoration using variational distribution approximation,” IEEE Trans. Image Process. 17, 326–339 (2008).
    [CrossRef]
  9. R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
    [CrossRef]
  10. G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
    [CrossRef]
  11. G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
    [CrossRef]
  12. G. Chantas, N. Galatsanos, and A. Likas, “Bayesian restoration using a new nonstationary edge-preserving image prior,” IEEE Trans. Image Process. 15, 2987–2997 (2006).
    [CrossRef]
  13. S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vis. 82, 205–229 (2009).
    [CrossRef]
  14. S. Babacan, R. Molina, M. Do, and A. Katsaggelos, “Blind deconvolution with general sparse image priors,” in Proceedings of European Conference on Computer Vision (ECCV) (Springer, 2012), pp. 341–355.
  15. R. Molina, J. Mateos, and A. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. Image Process. 15, 3715–3727 (2006).
    [CrossRef]

2010 (1)

G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
[CrossRef]

2009 (1)

S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vis. 82, 205–229 (2009).
[CrossRef]

2008 (3)

G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
[CrossRef]

D. Tzikas, A. Likas, and N. Galatsanos, “The variational approximation for Bayesian inference,” IEEE Signal Process. Mag. 25(6), 131–146 (2008).
[CrossRef]

S. Babacan, R. Molina, and A. Katsaggelos, “Parameter estimation in tv image restoration using variational distribution approximation,” IEEE Trans. Image Process. 17, 326–339 (2008).
[CrossRef]

2006 (3)

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

G. Chantas, N. Galatsanos, and A. Likas, “Bayesian restoration using a new nonstationary edge-preserving image prior,” IEEE Trans. Image Process. 15, 2987–2997 (2006).
[CrossRef]

R. Molina, J. Mateos, and A. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. Image Process. 15, 3715–3727 (2006).
[CrossRef]

1999 (1)

R. Molina, A. Katsaggelos, and J. Mateos, “Bayesian and regularization methods for hyperparameter estimation in image restoration,” IEEE Trans. Image Process. 8, 231–246 (1999).
[CrossRef]

1997 (1)

M. Banham and A. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997).
[CrossRef]

1994 (1)

R. Molina, “On the hierarchical Bayesian approach to image restoration: applications to astronomical images,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 1122–1128 (1994).
[CrossRef]

Andrews, H. C.

H. C. Andrews and B. R. Hunt, Digital Image Restoration (Prentice-Hall, 1977).

Babacan, S.

S. Babacan, R. Molina, and A. Katsaggelos, “Parameter estimation in tv image restoration using variational distribution approximation,” IEEE Trans. Image Process. 17, 326–339 (2008).
[CrossRef]

A. Katsaggelos, S. Babacan, and C.-J. Tsai, “Iterative image restoration,” in The Essential Guide to Image Processing, A. Bovik, ed. (Elsevier, 2009), Chap. 15.

S. Babacan, R. Molina, M. Do, and A. Katsaggelos, “Blind deconvolution with general sparse image priors,” in Proceedings of European Conference on Computer Vision (ECCV) (Springer, 2012), pp. 341–355.

Banham, M.

M. Banham and A. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997).
[CrossRef]

Bertero, M.

M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging (Taylor & Francis, 1998).

Black, M. J.

S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vis. 82, 205–229 (2009).
[CrossRef]

Boccacci, P.

M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging (Taylor & Francis, 1998).

Chantas, G.

G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
[CrossRef]

G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
[CrossRef]

G. Chantas, N. Galatsanos, and A. Likas, “Bayesian restoration using a new nonstationary edge-preserving image prior,” IEEE Trans. Image Process. 15, 2987–2997 (2006).
[CrossRef]

Do, M.

S. Babacan, R. Molina, M. Do, and A. Katsaggelos, “Blind deconvolution with general sparse image priors,” in Proceedings of European Conference on Computer Vision (ECCV) (Springer, 2012), pp. 341–355.

Fergus, R.

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

Freeman, W. T.

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

Galatsanos, N.

G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
[CrossRef]

D. Tzikas, A. Likas, and N. Galatsanos, “The variational approximation for Bayesian inference,” IEEE Signal Process. Mag. 25(6), 131–146 (2008).
[CrossRef]

G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
[CrossRef]

G. Chantas, N. Galatsanos, and A. Likas, “Bayesian restoration using a new nonstationary edge-preserving image prior,” IEEE Trans. Image Process. 15, 2987–2997 (2006).
[CrossRef]

Hertzmann, A.

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

Hunt, B. R.

H. C. Andrews and B. R. Hunt, Digital Image Restoration (Prentice-Hall, 1977).

Katsaggelos, A.

G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
[CrossRef]

S. Babacan, R. Molina, and A. Katsaggelos, “Parameter estimation in tv image restoration using variational distribution approximation,” IEEE Trans. Image Process. 17, 326–339 (2008).
[CrossRef]

R. Molina, J. Mateos, and A. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. Image Process. 15, 3715–3727 (2006).
[CrossRef]

R. Molina, A. Katsaggelos, and J. Mateos, “Bayesian and regularization methods for hyperparameter estimation in image restoration,” IEEE Trans. Image Process. 8, 231–246 (1999).
[CrossRef]

M. Banham and A. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997).
[CrossRef]

A. Katsaggelos, S. Babacan, and C.-J. Tsai, “Iterative image restoration,” in The Essential Guide to Image Processing, A. Bovik, ed. (Elsevier, 2009), Chap. 15.

S. Babacan, R. Molina, M. Do, and A. Katsaggelos, “Blind deconvolution with general sparse image priors,” in Proceedings of European Conference on Computer Vision (ECCV) (Springer, 2012), pp. 341–355.

Likas, A.

D. Tzikas, A. Likas, and N. Galatsanos, “The variational approximation for Bayesian inference,” IEEE Signal Process. Mag. 25(6), 131–146 (2008).
[CrossRef]

G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
[CrossRef]

G. Chantas, N. Galatsanos, and A. Likas, “Bayesian restoration using a new nonstationary edge-preserving image prior,” IEEE Trans. Image Process. 15, 2987–2997 (2006).
[CrossRef]

Mateos, J.

R. Molina, J. Mateos, and A. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. Image Process. 15, 3715–3727 (2006).
[CrossRef]

R. Molina, A. Katsaggelos, and J. Mateos, “Bayesian and regularization methods for hyperparameter estimation in image restoration,” IEEE Trans. Image Process. 8, 231–246 (1999).
[CrossRef]

Molina, R.

G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
[CrossRef]

S. Babacan, R. Molina, and A. Katsaggelos, “Parameter estimation in tv image restoration using variational distribution approximation,” IEEE Trans. Image Process. 17, 326–339 (2008).
[CrossRef]

R. Molina, J. Mateos, and A. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. Image Process. 15, 3715–3727 (2006).
[CrossRef]

R. Molina, A. Katsaggelos, and J. Mateos, “Bayesian and regularization methods for hyperparameter estimation in image restoration,” IEEE Trans. Image Process. 8, 231–246 (1999).
[CrossRef]

R. Molina, “On the hierarchical Bayesian approach to image restoration: applications to astronomical images,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 1122–1128 (1994).
[CrossRef]

S. Babacan, R. Molina, M. Do, and A. Katsaggelos, “Blind deconvolution with general sparse image priors,” in Proceedings of European Conference on Computer Vision (ECCV) (Springer, 2012), pp. 341–355.

Roth, S.

S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vis. 82, 205–229 (2009).
[CrossRef]

Roweis, S. T.

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

Saunders, M.

G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
[CrossRef]

Singh, B.

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

Tsai, C.-J.

A. Katsaggelos, S. Babacan, and C.-J. Tsai, “Iterative image restoration,” in The Essential Guide to Image Processing, A. Bovik, ed. (Elsevier, 2009), Chap. 15.

Tzikas, D.

D. Tzikas, A. Likas, and N. Galatsanos, “The variational approximation for Bayesian inference,” IEEE Signal Process. Mag. 25(6), 131–146 (2008).
[CrossRef]

ACM Trans. Graph. (1)

R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25, 787–794 (2006).
[CrossRef]

IEEE Signal Process. Mag. (2)

M. Banham and A. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997).
[CrossRef]

D. Tzikas, A. Likas, and N. Galatsanos, “The variational approximation for Bayesian inference,” IEEE Signal Process. Mag. 25(6), 131–146 (2008).
[CrossRef]

IEEE Trans. Image Process. (6)

S. Babacan, R. Molina, and A. Katsaggelos, “Parameter estimation in tv image restoration using variational distribution approximation,” IEEE Trans. Image Process. 17, 326–339 (2008).
[CrossRef]

R. Molina, A. Katsaggelos, and J. Mateos, “Bayesian and regularization methods for hyperparameter estimation in image restoration,” IEEE Trans. Image Process. 8, 231–246 (1999).
[CrossRef]

G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, “Variational Bayesian image restoration based on a product of t-distributions image prior,” IEEE Trans. Image Process. 17, 1795–1805 (2008).
[CrossRef]

G. Chantas, N. Galatsanos, R. Molina, and A. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process. 19, 351–362 (2010).
[CrossRef]

G. Chantas, N. Galatsanos, and A. Likas, “Bayesian restoration using a new nonstationary edge-preserving image prior,” IEEE Trans. Image Process. 15, 2987–2997 (2006).
[CrossRef]

R. Molina, J. Mateos, and A. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. Image Process. 15, 3715–3727 (2006).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

R. Molina, “On the hierarchical Bayesian approach to image restoration: applications to astronomical images,” IEEE Trans. Pattern Anal. Mach. Intell. 16, 1122–1128 (1994).
[CrossRef]

Int. J. Comput. Vis. (1)

S. Roth and M. J. Black, “Fields of experts,” Int. J. Comput. Vis. 82, 205–229 (2009).
[CrossRef]

Other (4)

S. Babacan, R. Molina, M. Do, and A. Katsaggelos, “Blind deconvolution with general sparse image priors,” in Proceedings of European Conference on Computer Vision (ECCV) (Springer, 2012), pp. 341–355.

H. C. Andrews and B. R. Hunt, Digital Image Restoration (Prentice-Hall, 1977).

M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging (Taylor & Francis, 1998).

A. Katsaggelos, S. Babacan, and C.-J. Tsai, “Iterative image restoration,” in The Essential Guide to Image Processing, A. Bovik, ed. (Elsevier, 2009), Chap. 15.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1.

Restoration results for cameraman corrupted with a 9×9 uniform kernel and 40 dB BSNR. (a) Degraded; (b) original; (c) BTV, ISNR=8.60dB; (d) BST, ISNR=8.80dB; (e) NF2, ISNR=9.34dB; and (f) NF4, ISNR=9.75dB.

Fig. 2.
Fig. 2.

Zoomed-in restoration results for cameraman corrupted with a 9×9 uniform kernel and 40 dB BSNR. (a) Original; (b) BTV, ISNR=9.34dB; and (c) NF4, ISNR=9.75dB.

Fig. 3.
Fig. 3.

Digital refocusing image samples using a medium defocus. (a) Focused image, (b) defocused image (inset: measured PSF), (c) BTV restoration, and (d) NF4 restoration.

Fig. 4.
Fig. 4.

Digital refocusing image samples using a strong defocus. (a) Focused image, (b) defocused image (inset: measured PSF), (c) BTV restoration, and (d) NF4 restoration.

Fig. 5.
Fig. 5.

Digital refocusing experiment. (a) Focused image (inset: 4× measured PSF), (b) moderate defocus image (inset: 4× measured PSF), (c) medium defocus image (inset: 4× measured PSF), (d) strong defocus image (inset: 4× measured PSF), (e) restored image (b) using NF4, (f) restored image (c) using NF4, (g) restored image (d) using NF4.

Tables (1)

Tables Icon

Table 1. Comparative Restoration Performance Results in Terms of the ISNR (dB)

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

y=Hx+n.
p(y|x,β)βN/2exp{β2yHx2},
p(x|a1,,aL)|i=1LCitAiCi|1/2exp{12i=1LAi1/2Cix2},
p(y,x,β,a1,,aL)p(y|x,β)p(x|a1,,aL)|i=1LCitAiCi|1/2βN/2exp{12i=1LAi1/2Cix2}×exp{β2yHx2}.
p(y|β,a1,,aL)xp(y,x,β,a1,,aL)dx,
lnp(y|β,a1,,aL,)=12ln|i=1LCitAiCi|+N2lnβ12i=1LAi1/2Cix¯2β2yHx¯212ln|i=1LCitAiCi+βHtH|.
δlnp(y|β,a1,,aL)δaij=12(trace[(i=1LCitAiCi)1CitJjjCi]x¯tCitJjjCix¯trace[(i=1LCitAiCi+βHtH)1CitJjjCi]),
x¯=(i=1LCitAiCi+βHtH)1Hty,
trace[ΣP1CitJjjCi]=(νij)2+trace[ΣT1CitJjjCi].
aij(k+1)=trace[ΣP(k)1CitJjjCi](νij(k))2+trace[ΣT(k)1CitJjjCi]aij(k),
ΣP(k)1DIAG(ζP(k)),ΣT(k)1DIAG(ζT(k)).
CitJjjCiSijj,
trace[ΣP1CitJjjCi]trace[ΣP1Sijj].
Ai(k+1)[DIAG(μi(k)+FiΣT(k)11⃗)]1DIAG(FiΣP(k)1ai(k)),
f=[010111010].
c1=12[11],c2=12[11]t;
c3=12[1001],c4=12[0110].

Metrics