Abstract

We address the problem of inpainting noisy photographs. We present a recursive image recovery scheme based on the unscented Kalman filter (UKF) to simultaneously inpaint identified damaged portions in an image and suppress film-grain noise. Inpainting of the missing observations is guided by a mask-dependent reconstruction of the image edges. Prediction within the UKF is based on a discontinuity-adaptive Markov random field prior that attempts to preserve edges while achieving noise reduction in uniform regions. We demonstrate the capability of the proposed method with many examples.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.
  2. T. F. Chan and J. Shen, “Mathematical models of local non-texture inpainting,” SIAM J. Appl. Math. 62, 1019–1043 (2001).
  3. B. R. Hunt, “Bayesian methods in nonlinear digital image restoration,” IEEE Trans. Comput. 26, 219–229 (1977).
    [CrossRef]
  4. A. Stefano, B. Collis, and P. White, “Synthesising and reducing film grain,” J. Visual Commun. Image Represent 17, 163–182 (2006).
    [CrossRef]
  5. A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
    [CrossRef]
  6. J. Jia and C. Tang, “Image repairing: Robust image synthesis by adaptive ND tensor voting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1988), pp. 643–650.
  7. A. Telea, “An image inpainting technique based on the fast marching method,” J. Graph. Tools 9, 25–36 (2004).
  8. A. Rares, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Trans. Image Process. 14, 1454–1468 (2005).
    [CrossRef] [PubMed]
  9. N. Komodakis and G. Tziritas, “Image completion using global optimization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 442–452.
  10. J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
    [CrossRef] [PubMed]
  11. Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
    [CrossRef] [PubMed]
  12. K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process. 16, 1200–1212 (2007).
    [CrossRef]
  13. C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
    [CrossRef] [PubMed]
  14. F. Naderi and A. A. Sawchuk, “Estimation of images degraded by film-grain noise,” Appl. Opt. 20, 1228–1237 (1978).
    [CrossRef]
  15. A. M. Tekalp and G. Pavlovic, “Restoration in the presence of multiplicative noise with application to scanned photographic images,” IEEE Trans. Signal Process. 39, 2132–2136 (1991).
    [CrossRef]
  16. A. D. Stefano, P. R. White, and W. B. Collis, “Film grain reduction on colour images using undecimated wavelet transform,” Image Vision Comput. 22, 873–882 (2004).
    [CrossRef]
  17. S. Ibrahim Sadhar and A. N. Rajagopalan, “Image estimation in film-grain noise,” IEEE Signal Process. Lett. 12, 238–241 (2005).
    [CrossRef]
  18. G. R. K. S. Subrahmanyam, A. N. Rajagopalan, and R. Aravind, “Importance-sampling-based unscented Kalman filter for film-grain noise removal,” IEEE Multimedia 15, 52–63 (2008).
    [CrossRef]
  19. A. C. Kokaram and S. J. Godsill, “MCMC for joint noise reduction and missing data treatment in degraded video,” IEEE Trans. Signal Process. 50, 189–205 (2002).
    [CrossRef]
  20. A. C. Kokaram, “On missing data treatment for degraded video and film archives: A survey and a new Bayesian approach,” IEEE Trans. Image Process. 13, 397–415 (2004).
    [CrossRef] [PubMed]
  21. C. A. Z. Barcelos and M. A. Batista, “Image restoration using digital inpainting and noise removal,” Image Vision Comput. 25, 61–69 (2007).
    [CrossRef]
  22. P. Favaro and S. Soatto, “Seeing beyond occlusions (and other marvels of a finite lens aperture),” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003), pp. 579–586.
  23. S. W. Hasinoff and K. N. Kutulakos, “A layer-based restoration framework for variable-aperture photography,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–8.
  24. S. J. Julier and J. K. Uhlmann, “A new extension of the Kalman filter to nonlinear systems,” in Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Controls: Signal Processing, Sensor Fusion, and Target Recognition VI, Orlando, Florida (1997), Vol. 3068, pp. 182–183.
  25. R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).
  26. S. J. Julier and J. K. Uhlmann, “A general method for approximating nonlinear transformations of probability distributions,” Tech. Rep. RRG, Dept. of Engineering Science (University of Oxford, 1996).
  27. S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer, 1995).
  28. S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984).
    [CrossRef]
  29. D. J. C. Mackay, “Introduction to Monte Carlo methods,” in Learning in Graphical Models, NATO Science Series, M.I.Jordan, ed. (Kluwer, 1998), pp. 175–204.
  30. A. K. Jain, “Advances in mathematical models for image processing,” IEEE Photon. Technol. Lett. 69, 502–528 (1981).
  31. D. Corrigan, N. Harte, and A. Kokaram, “Automatic segmentation of torn frames using the graph cuts technique,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. I-557–I-560.

2008 (1)

G. R. K. S. Subrahmanyam, A. N. Rajagopalan, and R. Aravind, “Importance-sampling-based unscented Kalman filter for film-grain noise removal,” IEEE Multimedia 15, 52–63 (2008).
[CrossRef]

2007 (5)

K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process. 16, 1200–1212 (2007).
[CrossRef]

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

C. A. Z. Barcelos and M. A. Batista, “Image restoration using digital inpainting and noise removal,” Image Vision Comput. 25, 61–69 (2007).
[CrossRef]

S. W. Hasinoff and K. N. Kutulakos, “A layer-based restoration framework for variable-aperture photography,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–8.

D. Corrigan, N. Harte, and A. Kokaram, “Automatic segmentation of torn frames using the graph cuts technique,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. I-557–I-560.

2006 (4)

A. Stefano, B. Collis, and P. White, “Synthesising and reducing film grain,” J. Visual Commun. Image Represent 17, 163–182 (2006).
[CrossRef]

N. Komodakis and G. Tziritas, “Image completion using global optimization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 442–452.

J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
[CrossRef] [PubMed]

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

2005 (2)

A. Rares, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Trans. Image Process. 14, 1454–1468 (2005).
[CrossRef] [PubMed]

S. Ibrahim Sadhar and A. N. Rajagopalan, “Image estimation in film-grain noise,” IEEE Signal Process. Lett. 12, 238–241 (2005).
[CrossRef]

2004 (3)

A. D. Stefano, P. R. White, and W. B. Collis, “Film grain reduction on colour images using undecimated wavelet transform,” Image Vision Comput. 22, 873–882 (2004).
[CrossRef]

A. Telea, “An image inpainting technique based on the fast marching method,” J. Graph. Tools 9, 25–36 (2004).

A. C. Kokaram, “On missing data treatment for degraded video and film archives: A survey and a new Bayesian approach,” IEEE Trans. Image Process. 13, 397–415 (2004).
[CrossRef] [PubMed]

2003 (1)

P. Favaro and S. Soatto, “Seeing beyond occlusions (and other marvels of a finite lens aperture),” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003), pp. 579–586.

2002 (1)

A. C. Kokaram and S. J. Godsill, “MCMC for joint noise reduction and missing data treatment in degraded video,” IEEE Trans. Signal Process. 50, 189–205 (2002).
[CrossRef]

2001 (1)

T. F. Chan and J. Shen, “Mathematical models of local non-texture inpainting,” SIAM J. Appl. Math. 62, 1019–1043 (2001).

2000 (2)

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.

R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).

1998 (1)

D. J. C. Mackay, “Introduction to Monte Carlo methods,” in Learning in Graphical Models, NATO Science Series, M.I.Jordan, ed. (Kluwer, 1998), pp. 175–204.

1997 (1)

S. J. Julier and J. K. Uhlmann, “A new extension of the Kalman filter to nonlinear systems,” in Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Controls: Signal Processing, Sensor Fusion, and Target Recognition VI, Orlando, Florida (1997), Vol. 3068, pp. 182–183.

1996 (1)

S. J. Julier and J. K. Uhlmann, “A general method for approximating nonlinear transformations of probability distributions,” Tech. Rep. RRG, Dept. of Engineering Science (University of Oxford, 1996).

1995 (2)

S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer, 1995).

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
[CrossRef]

1991 (1)

A. M. Tekalp and G. Pavlovic, “Restoration in the presence of multiplicative noise with application to scanned photographic images,” IEEE Trans. Signal Process. 39, 2132–2136 (1991).
[CrossRef]

1988 (1)

J. Jia and C. Tang, “Image repairing: Robust image synthesis by adaptive ND tensor voting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1988), pp. 643–650.

1984 (1)

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984).
[CrossRef]

1981 (1)

A. K. Jain, “Advances in mathematical models for image processing,” IEEE Photon. Technol. Lett. 69, 502–528 (1981).

1978 (1)

F. Naderi and A. A. Sawchuk, “Estimation of images degraded by film-grain noise,” Appl. Opt. 20, 1228–1237 (1978).
[CrossRef]

1977 (1)

B. R. Hunt, “Bayesian methods in nonlinear digital image restoration,” IEEE Trans. Comput. 26, 219–229 (1977).
[CrossRef]

Aravind, R.

G. R. K. S. Subrahmanyam, A. N. Rajagopalan, and R. Aravind, “Importance-sampling-based unscented Kalman filter for film-grain noise removal,” IEEE Multimedia 15, 52–63 (2008).
[CrossRef]

Ballester, C.

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.

Barcelos, C. A. Z.

C. A. Z. Barcelos and M. A. Batista, “Image restoration using digital inpainting and noise removal,” Image Vision Comput. 25, 61–69 (2007).
[CrossRef]

Batista, M. A.

C. A. Z. Barcelos and M. A. Batista, “Image restoration using digital inpainting and noise removal,” Image Vision Comput. 25, 61–69 (2007).
[CrossRef]

Bertalmio, M.

K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process. 16, 1200–1212 (2007).
[CrossRef]

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.

Biemond, J.

A. Rares, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Trans. Image Process. 14, 1454–1468 (2005).
[CrossRef] [PubMed]

Caselles, V.

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.

Chan, T. F.

T. F. Chan and J. Shen, “Mathematical models of local non-texture inpainting,” SIAM J. Appl. Math. 62, 1019–1043 (2001).

Collis, B.

A. Stefano, B. Collis, and P. White, “Synthesising and reducing film grain,” J. Visual Commun. Image Represent 17, 163–182 (2006).
[CrossRef]

Collis, W. B.

A. D. Stefano, P. R. White, and W. B. Collis, “Film grain reduction on colour images using undecimated wavelet transform,” Image Vision Comput. 22, 873–882 (2004).
[CrossRef]

Corrigan, D.

D. Corrigan, N. Harte, and A. Kokaram, “Automatic segmentation of torn frames using the graph cuts technique,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. I-557–I-560.

de Freitas, J. F. G.

R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).

Doucet, D.

R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).

Favaro, P.

P. Favaro and S. Soatto, “Seeing beyond occlusions (and other marvels of a finite lens aperture),” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003), pp. 579–586.

Fitzgerald, W. J.

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
[CrossRef]

Garrido, L.

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

Ge, W.

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

Geman, D.

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984).
[CrossRef]

Geman, S.

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984).
[CrossRef]

Godsill, S. J.

A. C. Kokaram and S. J. Godsill, “MCMC for joint noise reduction and missing data treatment in degraded video,” IEEE Trans. Signal Process. 50, 189–205 (2002).
[CrossRef]

Harte, N.

D. Corrigan, N. Harte, and A. Kokaram, “Automatic segmentation of torn frames using the graph cuts technique,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. I-557–I-560.

Hasinoff, S. W.

S. W. Hasinoff and K. N. Kutulakos, “A layer-based restoration framework for variable-aperture photography,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Hunt, B. R.

B. R. Hunt, “Bayesian methods in nonlinear digital image restoration,” IEEE Trans. Comput. 26, 219–229 (1977).
[CrossRef]

Ibrahim Sadhar, S.

S. Ibrahim Sadhar and A. N. Rajagopalan, “Image estimation in film-grain noise,” IEEE Signal Process. Lett. 12, 238–241 (2005).
[CrossRef]

Jain, A. K.

A. K. Jain, “Advances in mathematical models for image processing,” IEEE Photon. Technol. Lett. 69, 502–528 (1981).

Jia, J.

J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
[CrossRef] [PubMed]

J. Jia and C. Tang, “Image repairing: Robust image synthesis by adaptive ND tensor voting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1988), pp. 643–650.

Julier, S. J.

S. J. Julier and J. K. Uhlmann, “A new extension of the Kalman filter to nonlinear systems,” in Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Controls: Signal Processing, Sensor Fusion, and Target Recognition VI, Orlando, Florida (1997), Vol. 3068, pp. 182–183.

S. J. Julier and J. K. Uhlmann, “A general method for approximating nonlinear transformations of probability distributions,” Tech. Rep. RRG, Dept. of Engineering Science (University of Oxford, 1996).

Kokaram, A.

D. Corrigan, N. Harte, and A. Kokaram, “Automatic segmentation of torn frames using the graph cuts technique,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. I-557–I-560.

Kokaram, A. C.

A. C. Kokaram, “On missing data treatment for degraded video and film archives: A survey and a new Bayesian approach,” IEEE Trans. Image Process. 13, 397–415 (2004).
[CrossRef] [PubMed]

A. C. Kokaram and S. J. Godsill, “MCMC for joint noise reduction and missing data treatment in degraded video,” IEEE Trans. Signal Process. 50, 189–205 (2002).
[CrossRef]

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
[CrossRef]

Komodakis, N.

N. Komodakis and G. Tziritas, “Image completion using global optimization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 442–452.

Kutulakos, K. N.

S. W. Hasinoff and K. N. Kutulakos, “A layer-based restoration framework for variable-aperture photography,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Li, S. Z.

S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer, 1995).

Mackay, D. J. C.

D. J. C. Mackay, “Introduction to Monte Carlo methods,” in Learning in Graphical Models, NATO Science Series, M.I.Jordan, ed. (Kluwer, 1998), pp. 175–204.

Marques, A.

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

Matsushita, Y.

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

Morris, R. D.

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
[CrossRef]

Naderi, F.

F. Naderi and A. A. Sawchuk, “Estimation of images degraded by film-grain noise,” Appl. Opt. 20, 1228–1237 (1978).
[CrossRef]

Ofek, E.

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

Patwardhan, K. A.

K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process. 16, 1200–1212 (2007).
[CrossRef]

Pavlovic, G.

A. M. Tekalp and G. Pavlovic, “Restoration in the presence of multiplicative noise with application to scanned photographic images,” IEEE Trans. Signal Process. 39, 2132–2136 (1991).
[CrossRef]

Rajagopalan, A. N.

G. R. K. S. Subrahmanyam, A. N. Rajagopalan, and R. Aravind, “Importance-sampling-based unscented Kalman filter for film-grain noise removal,” IEEE Multimedia 15, 52–63 (2008).
[CrossRef]

S. Ibrahim Sadhar and A. N. Rajagopalan, “Image estimation in film-grain noise,” IEEE Signal Process. Lett. 12, 238–241 (2005).
[CrossRef]

Ranchin, F.

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

Rares, A.

A. Rares, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Trans. Image Process. 14, 1454–1468 (2005).
[CrossRef] [PubMed]

Rayner, P. J. W.

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
[CrossRef]

Reinders, M. J. T.

A. Rares, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Trans. Image Process. 14, 1454–1468 (2005).
[CrossRef] [PubMed]

Sapiro, G.

K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process. 16, 1200–1212 (2007).
[CrossRef]

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.

Sawchuk, A. A.

F. Naderi and A. A. Sawchuk, “Estimation of images degraded by film-grain noise,” Appl. Opt. 20, 1228–1237 (1978).
[CrossRef]

Shen, J.

T. F. Chan and J. Shen, “Mathematical models of local non-texture inpainting,” SIAM J. Appl. Math. 62, 1019–1043 (2001).

Shum, H.

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

Soatto, S.

P. Favaro and S. Soatto, “Seeing beyond occlusions (and other marvels of a finite lens aperture),” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003), pp. 579–586.

Stefano, A.

A. Stefano, B. Collis, and P. White, “Synthesising and reducing film grain,” J. Visual Commun. Image Represent 17, 163–182 (2006).
[CrossRef]

Stefano, A. D.

A. D. Stefano, P. R. White, and W. B. Collis, “Film grain reduction on colour images using undecimated wavelet transform,” Image Vision Comput. 22, 873–882 (2004).
[CrossRef]

Subrahmanyam, G. R. K. S.

G. R. K. S. Subrahmanyam, A. N. Rajagopalan, and R. Aravind, “Importance-sampling-based unscented Kalman filter for film-grain noise removal,” IEEE Multimedia 15, 52–63 (2008).
[CrossRef]

Tai, Y.

J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
[CrossRef] [PubMed]

Tang, C.

J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
[CrossRef] [PubMed]

J. Jia and C. Tang, “Image repairing: Robust image synthesis by adaptive ND tensor voting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1988), pp. 643–650.

Tang, X.

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

Tekalp, A. M.

A. M. Tekalp and G. Pavlovic, “Restoration in the presence of multiplicative noise with application to scanned photographic images,” IEEE Trans. Signal Process. 39, 2132–2136 (1991).
[CrossRef]

Telea, A.

A. Telea, “An image inpainting technique based on the fast marching method,” J. Graph. Tools 9, 25–36 (2004).

Tziritas, G.

N. Komodakis and G. Tziritas, “Image completion using global optimization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 442–452.

Uhlmann, J. K.

S. J. Julier and J. K. Uhlmann, “A new extension of the Kalman filter to nonlinear systems,” in Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Controls: Signal Processing, Sensor Fusion, and Target Recognition VI, Orlando, Florida (1997), Vol. 3068, pp. 182–183.

S. J. Julier and J. K. Uhlmann, “A general method for approximating nonlinear transformations of probability distributions,” Tech. Rep. RRG, Dept. of Engineering Science (University of Oxford, 1996).

van der Merwe, R.

R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).

Wan, E. A.

R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).

White, P.

A. Stefano, B. Collis, and P. White, “Synthesising and reducing film grain,” J. Visual Commun. Image Represent 17, 163–182 (2006).
[CrossRef]

White, P. R.

A. D. Stefano, P. R. White, and W. B. Collis, “Film grain reduction on colour images using undecimated wavelet transform,” Image Vision Comput. 22, 873–882 (2004).
[CrossRef]

Wu, T.

J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
[CrossRef] [PubMed]

Appl. Opt. (1)

F. Naderi and A. A. Sawchuk, “Estimation of images degraded by film-grain noise,” Appl. Opt. 20, 1228–1237 (1978).
[CrossRef]

IEEE Multimedia (1)

G. R. K. S. Subrahmanyam, A. N. Rajagopalan, and R. Aravind, “Importance-sampling-based unscented Kalman filter for film-grain noise removal,” IEEE Multimedia 15, 52–63 (2008).
[CrossRef]

IEEE Photon. Technol. Lett. (1)

A. K. Jain, “Advances in mathematical models for image processing,” IEEE Photon. Technol. Lett. 69, 502–528 (1981).

IEEE Signal Process. Lett. (1)

S. Ibrahim Sadhar and A. N. Rajagopalan, “Image estimation in film-grain noise,” IEEE Signal Process. Lett. 12, 238–241 (2005).
[CrossRef]

IEEE Trans. Comput. (1)

B. R. Hunt, “Bayesian methods in nonlinear digital image restoration,” IEEE Trans. Comput. 26, 219–229 (1977).
[CrossRef]

IEEE Trans. Image Process. (5)

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, “Interpolation of missing data in image sequences,” IEEE Trans. Image Process. 11, 1509–1519 (1995).
[CrossRef]

A. Rares, M. J. T. Reinders, and J. Biemond, “Edge-based image restoration,” IEEE Trans. Image Process. 14, 1454–1468 (2005).
[CrossRef] [PubMed]

K. A. Patwardhan, G. Sapiro, and M. Bertalmio, “Video inpainting under constrained camera motion,” IEEE Trans. Image Process. 16, 1200–1212 (2007).
[CrossRef]

C. Ballester, M. Bertalmio, V. Caselles, L. Garrido, A. Marques, and F. Ranchin, “An inpainting-based deinterlacing method,” IEEE Trans. Image Process. 16, 2476–2491 (2007).
[CrossRef] [PubMed]

A. C. Kokaram, “On missing data treatment for degraded video and film archives: A survey and a new Bayesian approach,” IEEE Trans. Image Process. 13, 397–415 (2004).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (3)

S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,” IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984).
[CrossRef]

J. Jia, Y. Tai, T. Wu, and C. Tang, “Video repairing under variable illumination using cyclic motions,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 832–839 (2006).
[CrossRef] [PubMed]

Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150–1164 (2006).
[CrossRef] [PubMed]

IEEE Trans. Signal Process. (2)

A. M. Tekalp and G. Pavlovic, “Restoration in the presence of multiplicative noise with application to scanned photographic images,” IEEE Trans. Signal Process. 39, 2132–2136 (1991).
[CrossRef]

A. C. Kokaram and S. J. Godsill, “MCMC for joint noise reduction and missing data treatment in degraded video,” IEEE Trans. Signal Process. 50, 189–205 (2002).
[CrossRef]

Image Vision Comput. (2)

C. A. Z. Barcelos and M. A. Batista, “Image restoration using digital inpainting and noise removal,” Image Vision Comput. 25, 61–69 (2007).
[CrossRef]

A. D. Stefano, P. R. White, and W. B. Collis, “Film grain reduction on colour images using undecimated wavelet transform,” Image Vision Comput. 22, 873–882 (2004).
[CrossRef]

J. Graph. Tools (1)

A. Telea, “An image inpainting technique based on the fast marching method,” J. Graph. Tools 9, 25–36 (2004).

J. Visual Commun. Image Represent (1)

A. Stefano, B. Collis, and P. White, “Synthesising and reducing film grain,” J. Visual Commun. Image Represent 17, 163–182 (2006).
[CrossRef]

SIAM J. Appl. Math. (1)

T. F. Chan and J. Shen, “Mathematical models of local non-texture inpainting,” SIAM J. Appl. Math. 62, 1019–1043 (2001).

Other (11)

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press, 2000), pp. 417–424.

N. Komodakis and G. Tziritas, “Image completion using global optimization,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 442–452.

J. Jia and C. Tang, “Image repairing: Robust image synthesis by adaptive ND tensor voting,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1988), pp. 643–650.

P. Favaro and S. Soatto, “Seeing beyond occlusions (and other marvels of a finite lens aperture),” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003), pp. 579–586.

S. W. Hasinoff and K. N. Kutulakos, “A layer-based restoration framework for variable-aperture photography,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2007), pp. 1–8.

S. J. Julier and J. K. Uhlmann, “A new extension of the Kalman filter to nonlinear systems,” in Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Controls: Signal Processing, Sensor Fusion, and Target Recognition VI, Orlando, Florida (1997), Vol. 3068, pp. 182–183.

R. van der Merwe, J. F. G. de Freitas, D. Doucet, and E. A. Wan, “The unscented particle filter,” Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department (2000).

S. J. Julier and J. K. Uhlmann, “A general method for approximating nonlinear transformations of probability distributions,” Tech. Rep. RRG, Dept. of Engineering Science (University of Oxford, 1996).

S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer, 1995).

D. J. C. Mackay, “Introduction to Monte Carlo methods,” in Learning in Graphical Models, NATO Science Series, M.I.Jordan, ed. (Kluwer, 1998), pp. 175–204.

D. Corrigan, N. Harte, and A. Kokaram, “Automatic segmentation of torn frames using the graph cuts technique,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. I-557–I-560.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Edge reconstruction: (a) Degraded image showing the dimensions of the maximum matching area. (b) Edge map of the degraded image. (c) Located end points of (b). (d) Reconstructed edge image.

Fig. 2
Fig. 2

Proposed inpainting algorithm.

Fig. 3
Fig. 3

Proposed UKF framework for inpainting in the presence of noise.

Fig. 4
Fig. 4

Peppers: (a) Original. (b) Degraded and superimposed with text. (c) Image recovered by the proposed filter ( ISNR = 16.16 dB ) .

Fig. 5
Fig. 5

(a) Frame from a movie. (b) Edge image of (a). (c) Reconstructed edge image when input is (b). (d) Image recovered by the proposed filter. [Courtesy of the Criterion Collection]

Fig. 6
Fig. 6

(a) Image of a child with text, scratch, and film-grain noise. (b) Inpainted and filtered output of the proposed method.

Fig. 7
Fig. 7

(a) Degraded frame of a real sequence with film tear [20] (©2004 IEEE. Published with permission.). (b) Output of the Bayesian method [20] (©2004 IEEE. Published with permission.). (c) Results obtained using our algorithm.

Fig. 8
Fig. 8

(a) Another real film tear [31] (©2007 IEEE. Published with permission.). (b) MCMC output [31] (©2007 IEEE. Published with permission.). (c) Output of our method. (d), (e) Zoomed-in portions from (b) and (c), respectively.

Fig. 9
Fig. 9

(a) Degraded image [21] (©2007 Elsevier. Published with permission.). (b) MCF output [21] (©2007 Elsevier. Published with permission.). (c) Our output. (d), (e) Zoomed-in regions from (b) and (c), respectively.

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

r d ( m , n ) = α log 10 ( s ( m , n ) ) + β + v ( m , n ) ,
r e ( m , n ) = s ( m , n ) ( 10 v ( m , n ) α ) .
r e ( m , n ) = 10 ( r d ( m , n ) β α )
P ( s ( m , n ) s ̂ ) = exp ( γ log ( 1 + η 2 ( s ( m , n ) , s ̂ ) γ ) ) ,
g ( z ) p ( z ) d z = g ( z ) ( p ( z ) q ( z ) ) q ( z ) d z = E q ( z ) [ g ( z ) ( p ( z ) q ( z ) ) ] = 1 L l = 1 L g ( z ( l ) ) ( p ( z ( l ) ) q ( z ( l ) ) )
η 2 ( s ( m , n ) , s ̂ ) = 1 P ( m , n 1 ) ( i , j ) R ( s ( m , n ) s ̂ ( m i , n j ) ) 2 i 2 + j 2 ,
( i , j ) R = { ( i , j ) | ( 1 i M 1 , M 1 j M 1 ) ( i = 0 , 1 j M 1 ) }
x ¯ k k 1 a = [ x ¯ k k 1 T 0 ] T = [ μ ̂ p 0 ] T ,
P k k 1 a = [ P k k 1 0 0 σ v 2 ] = [ σ ̂ p 2 0 0 σ v 2 ] .
X k k 1 a = [ x ¯ k k 1 a x ¯ k k 1 a ± ( n a + λ ) P k k 1 a ] .
Y ( m , n ) ( m , n 1 ) = h ( X ( m , n ) ( m , n 1 ) x , X ( m , n 1 ) v ) = X ( m , n ) ( m , n 1 ) x . 10 . ( X ( m , n 1 ) v α ) ,
y ¯ ( m , n ) ( m , n 1 ) = i = 0 2 n a w i ( m ) Y i , ( m , n ) ( m , n 1 ) .
P y y = i = 0 2 n a w i ( c ) [ Y i , ( m , n ) ( m , n 1 ) y ¯ ( m , n ) ( m , n 1 ) ] [ Y i , ( m , n ) ( m , n 1 ) y ¯ ( m , n ) ( m , n 1 ) ] T .
P x y = i = 0 2 n a w i ( c ) [ X i , ( m , n ) ( m , n 1 ) x ¯ ( m , n ) ( m , n 1 ) ] [ Y i , ( m , n ) ( m , n 1 ) y ¯ ( m , n ) ( m , n 1 ) ] T .
S = S { I ( i , j ) : ( i , j ) W , ( i , j ) Ω }

Metrics