Abstract

Superresolution is the process of combining information from multiple subpixel-shifted low-resolution images to form a high-resolution image. It works quite well under ideal conditions but deteriorates rapidly with inaccuracies in motion estimates. We model the original high-resolution image as a Markov random field (MRF) with a discontinuity adaptive regularizer. Given the low-resolution observations, an estimate of the superresolved image is obtained by using the iterated conditional modes (ICM) algorithm, which maximizes the local posterior conditional probability sequentially. The proposed method not only preserves edges but also lends robustness to errors in the estimates of motion and blur parameters. We derive theoretically the neighborhood structure for the posterior distribution in the presence of warping, blurring, and downsampling operations and use this to effectively reduce the overall computations. Results are given on synthetic as well as real data to validate our method.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. S. Borman and R. L. Stevenson, "Super-resolution from image sequences--a review," in Proceedings of 1998 Midwest Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, 1998), pp. 374-378.
  2. S. Chaudhuri, Super-Resolution Imaging (Kluwer, 2001).
  3. M. G. Kang and S. Chaudhuri, "Super-resolution image reconstruction," IEEE Signal Process. Mag. 20, 21-36 (2003).
    [CrossRef]
  4. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
    [CrossRef]
  5. M. Elad and A. Feuer, "Restoration of a single super-resolution image from several blurred, noisy and under-sampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
    [CrossRef] [PubMed]
  6. D. Rajan and S. Chaudhuri, "Generation of super-resolution images from blurred observations using Markov random fields," in Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1837-1840.
  7. A. N. Rajagopalan and V. P. Kiran, "Motion-free super-resolution and the role of relative blur," J. Opt. Soc. Am. A 20, 2022-2032 (2003).
    [CrossRef]
  8. N. K. Bose, M. K. Ng, and A. C. Yau, "A fast algorithm for image super-resolution from blurred observations," EURASIP J. Appl. Signal Process 2006, 35726 (2006).
  9. S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
    [CrossRef]
  10. Q. Wang, X. Tang, and H. Shum, "Patch-based blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2005), pp. 709-716.
  11. T. S. Huang and R. Y. Tsai, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.S.Huang, ed. (JAI Press, 1984), Vol. 1, pp. 317-339.
  12. S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy under-sampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
    [CrossRef]
  13. H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
    [CrossRef]
  14. D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displacements," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 1988), pp. 742-746.
  15. S. Peleg and M. Irani, "Improving resolution by image registration," CVGIP: Graph. Models Image Process. 53, 231-239 (1991).
    [CrossRef]
  16. R. L. Stevenson and R. R. Schultz, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
    [CrossRef] [PubMed]
  17. R. C. Hardie, K. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images," IEEE Trans. Image Process. 6, 1621-1632 (1997).
    [CrossRef] [PubMed]
  18. A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 2001), pp. 645-650.
  19. E. S. Lee and M. G. Kang, "Regularized adaptive high-resolution image reconstruction considering inaccurate sub-pixel registration," IEEE Trans. Image Process. 12, 826-837 (2003).
    [CrossRef]
  20. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
    [CrossRef] [PubMed]
  21. S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer-Verlag, 1995).
  22. J. Besag, "On the statistical analysis of dirty pictures," J. R. Stat. Soc. Ser. B (Methodol.) 48, 259-302 (1986).
  23. S. Chaudhuri and A. N. Rajagopalan, Depth from Defocus: A Real Aperture Imaging Approach (Springer-Verlag, 1999).
    [CrossRef]
  24. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images," IEEE Trans. Image Process. 6, 721-741 (1984).
  25. A. Zomet and S. Peleg, "Super-resolution from multiple images having arbitrary mutual motion," in Super-Resolution Imaging, S.Chaudhuri, ed. (Kluwer, 2001), pp. 195-209.

2006 (1)

N. K. Bose, M. K. Ng, and A. C. Yau, "A fast algorithm for image super-resolution from blurred observations," EURASIP J. Appl. Signal Process 2006, 35726 (2006).

2005 (1)

Q. Wang, X. Tang, and H. Shum, "Patch-based blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2005), pp. 709-716.

2004 (2)

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

2003 (3)

E. S. Lee and M. G. Kang, "Regularized adaptive high-resolution image reconstruction considering inaccurate sub-pixel registration," IEEE Trans. Image Process. 12, 826-837 (2003).
[CrossRef]

M. G. Kang and S. Chaudhuri, "Super-resolution image reconstruction," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

A. N. Rajagopalan and V. P. Kiran, "Motion-free super-resolution and the role of relative blur," J. Opt. Soc. Am. A 20, 2022-2032 (2003).
[CrossRef]

2002 (1)

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

2001 (4)

D. Rajan and S. Chaudhuri, "Generation of super-resolution images from blurred observations using Markov random fields," in Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1837-1840.

S. Chaudhuri, Super-Resolution Imaging (Kluwer, 2001).

A. Zomet and S. Peleg, "Super-resolution from multiple images having arbitrary mutual motion," in Super-Resolution Imaging, S.Chaudhuri, ed. (Kluwer, 2001), pp. 195-209.

A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 2001), pp. 645-650.

1999 (1)

S. Chaudhuri and A. N. Rajagopalan, Depth from Defocus: A Real Aperture Imaging Approach (Springer-Verlag, 1999).
[CrossRef]

1998 (1)

S. Borman and R. L. Stevenson, "Super-resolution from image sequences--a review," in Proceedings of 1998 Midwest Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, 1998), pp. 374-378.

1997 (2)

M. Elad and A. Feuer, "Restoration of a single super-resolution image from several blurred, noisy and under-sampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

R. C. Hardie, K. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images," IEEE Trans. Image Process. 6, 1621-1632 (1997).
[CrossRef] [PubMed]

1996 (1)

R. L. Stevenson and R. R. Schultz, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

1995 (1)

S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer-Verlag, 1995).

1992 (1)

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

1991 (1)

S. Peleg and M. Irani, "Improving resolution by image registration," CVGIP: Graph. Models Image Process. 53, 231-239 (1991).
[CrossRef]

1990 (1)

S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy under-sampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

1988 (1)

D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displacements," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 1988), pp. 742-746.

1986 (1)

J. Besag, "On the statistical analysis of dirty pictures," J. R. Stat. Soc. Ser. B (Methodol.) 48, 259-302 (1986).

1984 (2)

S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images," IEEE Trans. Image Process. 6, 721-741 (1984).

T. S. Huang and R. Y. Tsai, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.S.Huang, ed. (JAI Press, 1984), Vol. 1, pp. 317-339.

Armstrong, E. E.

R. C. Hardie, K. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images," IEEE Trans. Image Process. 6, 1621-1632 (1997).
[CrossRef] [PubMed]

Baker, S.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

Barnard, K.

R. C. Hardie, K. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images," IEEE Trans. Image Process. 6, 1621-1632 (1997).
[CrossRef] [PubMed]

Besag, J.

J. Besag, "On the statistical analysis of dirty pictures," J. R. Stat. Soc. Ser. B (Methodol.) 48, 259-302 (1986).

Borman, S.

S. Borman and R. L. Stevenson, "Super-resolution from image sequences--a review," in Proceedings of 1998 Midwest Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, 1998), pp. 374-378.

Bose, N. K.

N. K. Bose, M. K. Ng, and A. C. Yau, "A fast algorithm for image super-resolution from blurred observations," EURASIP J. Appl. Signal Process 2006, 35726 (2006).

S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy under-sampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

Brada, R.

D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displacements," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 1988), pp. 742-746.

Chaudhuri, S.

M. G. Kang and S. Chaudhuri, "Super-resolution image reconstruction," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

S. Chaudhuri, Super-Resolution Imaging (Kluwer, 2001).

D. Rajan and S. Chaudhuri, "Generation of super-resolution images from blurred observations using Markov random fields," in Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1837-1840.

S. Chaudhuri and A. N. Rajagopalan, Depth from Defocus: A Real Aperture Imaging Approach (Springer-Verlag, 1999).
[CrossRef]

Elad, M.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

M. Elad and A. Feuer, "Restoration of a single super-resolution image from several blurred, noisy and under-sampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Farsiu, S.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

Feuer, A.

M. Elad and A. Feuer, "Restoration of a single super-resolution image from several blurred, noisy and under-sampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Geman, D.

S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images," IEEE Trans. Image Process. 6, 721-741 (1984).

Geman, S.

S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images," IEEE Trans. Image Process. 6, 721-741 (1984).

Gross, D.

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

Hardie, R. C.

R. C. Hardie, K. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images," IEEE Trans. Image Process. 6, 1621-1632 (1997).
[CrossRef] [PubMed]

Huang, T. S.

T. S. Huang and R. Y. Tsai, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.S.Huang, ed. (JAI Press, 1984), Vol. 1, pp. 317-339.

Irani, M.

S. Peleg and M. Irani, "Improving resolution by image registration," CVGIP: Graph. Models Image Process. 53, 231-239 (1991).
[CrossRef]

Kanade, T.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

Kang, M. G.

M. G. Kang and S. Chaudhuri, "Super-resolution image reconstruction," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

E. S. Lee and M. G. Kang, "Regularized adaptive high-resolution image reconstruction considering inaccurate sub-pixel registration," IEEE Trans. Image Process. 12, 826-837 (2003).
[CrossRef]

Keren, D.

D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displacements," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 1988), pp. 742-746.

Kim, S. P.

S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy under-sampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

Kiran, V. P.

Lee, E. S.

E. S. Lee and M. G. Kang, "Regularized adaptive high-resolution image reconstruction considering inaccurate sub-pixel registration," IEEE Trans. Image Process. 12, 826-837 (2003).
[CrossRef]

Li, S. Z.

S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer-Verlag, 1995).

Milanfar, P.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Ng, M. K.

N. K. Bose, M. K. Ng, and A. C. Yau, "A fast algorithm for image super-resolution from blurred observations," EURASIP J. Appl. Signal Process 2006, 35726 (2006).

Peleg, S.

A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 2001), pp. 645-650.

A. Zomet and S. Peleg, "Super-resolution from multiple images having arbitrary mutual motion," in Super-Resolution Imaging, S.Chaudhuri, ed. (Kluwer, 2001), pp. 195-209.

S. Peleg and M. Irani, "Improving resolution by image registration," CVGIP: Graph. Models Image Process. 53, 231-239 (1991).
[CrossRef]

D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displacements," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 1988), pp. 742-746.

Rajagopalan, A. N.

A. N. Rajagopalan and V. P. Kiran, "Motion-free super-resolution and the role of relative blur," J. Opt. Soc. Am. A 20, 2022-2032 (2003).
[CrossRef]

S. Chaudhuri and A. N. Rajagopalan, Depth from Defocus: A Real Aperture Imaging Approach (Springer-Verlag, 1999).
[CrossRef]

Rajan, D.

D. Rajan and S. Chaudhuri, "Generation of super-resolution images from blurred observations using Markov random fields," in Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1837-1840.

Rav-Acha, A.

A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 2001), pp. 645-650.

Robinson, D.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

Robinson, M. D.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

Schultz, R. R.

R. L. Stevenson and R. R. Schultz, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

Shum, H.

Q. Wang, X. Tang, and H. Shum, "Patch-based blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2005), pp. 709-716.

Stevenson, R. L.

S. Borman and R. L. Stevenson, "Super-resolution from image sequences--a review," in Proceedings of 1998 Midwest Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, 1998), pp. 374-378.

R. L. Stevenson and R. R. Schultz, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

Tang, X.

Q. Wang, X. Tang, and H. Shum, "Patch-based blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2005), pp. 709-716.

Tsai, R. Y.

T. S. Huang and R. Y. Tsai, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.S.Huang, ed. (JAI Press, 1984), Vol. 1, pp. 317-339.

Ur, H.

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

Valenzuela, H. M.

S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy under-sampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

Wang, Q.

Q. Wang, X. Tang, and H. Shum, "Patch-based blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2005), pp. 709-716.

Yau, A. C.

N. K. Bose, M. K. Ng, and A. C. Yau, "A fast algorithm for image super-resolution from blurred observations," EURASIP J. Appl. Signal Process 2006, 35726 (2006).

Zomet, A.

A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 2001), pp. 645-650.

A. Zomet and S. Peleg, "Super-resolution from multiple images having arbitrary mutual motion," in Super-Resolution Imaging, S.Chaudhuri, ed. (Kluwer, 2001), pp. 195-209.

CVGIP: Graph. Models Image Process. (2)

S. Peleg and M. Irani, "Improving resolution by image registration," CVGIP: Graph. Models Image Process. 53, 231-239 (1991).
[CrossRef]

H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process. 54, 181-186 (1992).
[CrossRef]

EURASIP J. Appl. Signal Process (1)

N. K. Bose, M. K. Ng, and A. C. Yau, "A fast algorithm for image super-resolution from blurred observations," EURASIP J. Appl. Signal Process 2006, 35726 (2006).

IEEE Signal Process. Mag. (1)

M. G. Kang and S. Chaudhuri, "Super-resolution image reconstruction," IEEE Signal Process. Mag. 20, 21-36 (2003).
[CrossRef]

IEEE Trans. Acoust., Speech, Signal Process. (1)

S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy under-sampled multiframes," IEEE Trans. Acoust., Speech, Signal Process. 38, 1013-1027 (1990).
[CrossRef]

IEEE Trans. Image Process. (6)

M. Elad and A. Feuer, "Restoration of a single super-resolution image from several blurred, noisy and under-sampled measured images," IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

R. L. Stevenson and R. R. Schultz, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

R. C. Hardie, K. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images," IEEE Trans. Image Process. 6, 1621-1632 (1997).
[CrossRef] [PubMed]

E. S. Lee and M. G. Kang, "Regularized adaptive high-resolution image reconstruction considering inaccurate sub-pixel registration," IEEE Trans. Image Process. 12, 826-837 (2003).
[CrossRef]

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process. 13, 1327-1344 (2004).
[CrossRef] [PubMed]

S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images," IEEE Trans. Image Process. 6, 721-741 (1984).

IEEE Trans. Pattern Anal. Mach. Intell. (1)

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
[CrossRef]

Int. J. Imaging Syst. Technol. (1)

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and challenges in super-resolution," Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
[CrossRef]

J. Opt. Soc. Am. A (1)

J. R. Stat. Soc. Ser. B (Methodol.) (1)

J. Besag, "On the statistical analysis of dirty pictures," J. R. Stat. Soc. Ser. B (Methodol.) 48, 259-302 (1986).

Other (10)

S. Chaudhuri and A. N. Rajagopalan, Depth from Defocus: A Real Aperture Imaging Approach (Springer-Verlag, 1999).
[CrossRef]

A. Zomet and S. Peleg, "Super-resolution from multiple images having arbitrary mutual motion," in Super-Resolution Imaging, S.Chaudhuri, ed. (Kluwer, 2001), pp. 195-209.

S. Z. Li, Markov Random Field Modeling in Computer Vision (Springer-Verlag, 1995).

D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displacements," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 1988), pp. 742-746.

D. Rajan and S. Chaudhuri, "Generation of super-resolution images from blurred observations using Markov random fields," in Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1837-1840.

A. Zomet, A. Rav-Acha, and S. Peleg, "Robust super-resolution," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, 2001), pp. 645-650.

S. Borman and R. L. Stevenson, "Super-resolution from image sequences--a review," in Proceedings of 1998 Midwest Symposium on Circuits and Systems (Institute of Electrical and Electronics Engineers, 1998), pp. 374-378.

S. Chaudhuri, Super-Resolution Imaging (Kluwer, 2001).

Q. Wang, X. Tang, and H. Shum, "Patch-based blind super-resolution," in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2005), pp. 709-716.

T. S. Huang and R. Y. Tsai, "Multiframe image restoration and registration," in Advances in Computer Vision and Image Processing, T.S.Huang, ed. (JAI Press, 1984), Vol. 1, pp. 317-339.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1

Discontinuity adaptive function.

Fig. 2
Fig. 2

Pictorial representation of the neighborhood for different operations. (a) Site ( k , l ) in LR image of size 4 × 4   pixels . Neighborhood structure after (b) downsampling ( ζ k , l samp ) , (c) defocus blurring ( ζ k , l cam ) , (d) warping ( ζ k , l warp ) , and (e) motion blurring ( ζ k , l mot ) .

Fig. 3
Fig. 3

Illustration of the set { A A r } for the case of q = 2 , σ b = 0.33 , and L = 3 . (a) Pixel ( i , j ) in HR image of size 8 × 8   pixels . Neighborhood structure after (b) motion blurring, (c) warping, (d) defocus blurring, and (e) downsampling.

Fig. 4
Fig. 4

Comparison among LS, RLS, and DAMRF. (a) Original, (b) LR observation, and (c) bilinear interpolation result. Superresolved image corresponding to (d) LS, (e) RLS, and (f) DAMRF.

Fig. 5
Fig. 5

(a) Original image. (b) One of the LR observations. Results corresponding to (c) bilinear interpolation, (d) GMRF, (e) HMRF, and (f) DAMRF. Error image corresponding to (g) GMRF, (h) HMRF, and (i) DAMRF.

Fig. 6
Fig. 6

(a) Original image. (b) Output with motion estimation error. (c) Output with error in blur estimation.

Fig. 7
Fig. 7

Results for the mobile calendar sequence. (a) LR observation and (b) DAMRF output.

Fig. 8
Fig. 8

Results for a real traffic video. (a) Cropped license plate and (b) bilinear interpolation result. Superresolved image using (c) LS, (d) GMRF, (e) HMRF, and (f) DAMRF.

Tables (2)

Tables Icon

Table 1 Comparison of Computational Requirements

Tables Icon

Table 2 Comparison among LS, RLS, and DAMRF Methods

Equations (28)

Equations on this page are rendered with MathJax. Learn more.

y r ̱ = D H r cam W r H r mot x ̱ + n r ̱ , 1 r m ,
( y 1 y 2 y m ) = ( D H 1 cam W 1 H 1 mot D H 2 cam W 2 H 2 mot D H m cam W m H m mot ) x ̱ + ( n 1 n 2 n m ) .
x ̱ ̂ = arg max x ̱ { P ( x ̱ y 1 ̱ , , y m ̱ ) } .
x ̱ ̂ = arg max x ̱ { P ( y 1 ̱ , , y m ̱ x ̱ ) P ( x ̱ ) P ( y 1 ̱ , , y m ̱ ) } .
x ̱ ̂ = arg max x ̱ { P ( y 1 ̱ , , y m ̱ x ̱ ) P ( x ̱ ) } .
x ̱ ̂ = arg max x ̱ { log [ P ( y 1 ̱ , , y m ̱ x ̱ ) ] + log P ( x ̱ ) } .
P [ F ̱ = f ̱ ] = ( 1 Z ) exp { U ( f ̱ ) } ,
U ( f ̱ ) = c C V c ( f ̱ ) .
P [ X ̱ = x ̱ ] = ( 1 Z ) exp { U ( x ̱ ) } ,
U ( x ̱ ) = c C V c ( x ̱ ) .
c C V c ( x ̱ ) = c C g ( d c x ̱ ) ,
g ( n ) = n 2 .
g ( n ) = { n 2 n T 2 T ( n T ) + T 2 n > T } ,
lim n g ( n ) = lim n 2 n h ( n ) = C ,
g ( n ) = γ γ e n 2 γ .
c C V c ( x ̱ ) = i = 1 N 1 j = 1 N 2 4 γ γ exp [ ( x ( i , j ) x ( i , j 1 ) ) 2 γ ] γ exp [ ( x ( i , j ) x ( i , j + 1 ) ) 2 γ ] γ exp [ ( x ( i , j ) x ( i 1 , j ) ) 2 γ ] γ exp [ ( x ( i , j ) x ( i + 1 , j ) ) 2 γ ] ,
P ( X ̱ = x ̱ ) = ( 1 Z ) exp { c C V c ( x ̱ ) } ,
P [ Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ X ̱ = x ̱ ] = 1 ( 2 π σ 2 ) m ( M 1 M 2 2 ) exp { r = 1 m y r ̱ D H r cam W r H r mot x ̱ 2 2 σ 2 } ,
P [ X ̱ = x ̱ Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ ] = K exp { U p ( x ̱ ) } ,
U p ( x ̱ ) = r = 1 m y r ̱ D H r cam W r H r mot x ̱ 2 2 σ 2 + λ i = 1 N 1 j = 1 N 2 4 γ γ exp [ ( x ( i , j ) x ( i , j 1 ) ) 2 γ ] γ exp [ ( x ( i , j ) x ( i , j + 1 ) ) 2 γ ] γ exp [ ( x ( i , j ) x ( i 1 , j ) ) 2 γ ] γ exp [ ( x ( i , j ) x ( i + 1 , j ) ) 2 γ ] ,
η i , j p = η i , j x r = 1 n { ( k , l ) { A A r } ζ k , l y r } ,
P [ X i , j = x i , j ; 1 ( i , j ) N 1 N 2 Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ ] = P [ X i , j = x i , j X k , l = x k , l ; 1 ( k , l ) N 1 N 2 , ( k , l ) ( i , j ) ; Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ ] P [ X k , l = x k , l ; 1 ( k , l ) N 1 N 2 , ( k , l ) ( i , j ) Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ ] .
P [ X i , j = x i , j X k , l = x k , l ; 1 ( k , l ) N 1 N 2 , ( k , l ) ( i , j ) ; Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ ] = exp { U p ( x ̱ ) } x i , j = all possible levels exp { U p ( x ̱ ) } .
Ψ ̱ r = 1 2 σ ( y r ̱ D H r cam W r H r mot x ̱ ) , 1 r m .
U p ( x ̱ ) = r = 1 m 1 ( k , l ) M 1 M 2 ψ r ̱ k , l 2 + c C V c ( x ̱ ) .
U p ( x ̱ ) = r = 1 m { A A r } ψ r ̱ k , l 2 + c C , ( i , j ) c V c ( x ̱ ) + r = 1 m { A r } ψ r ̱ k , l 2 + c C , ( i , j ) c V c ( x ̱ ) .
P [ X i , j = x i , j X k , l = x k , l ; 1 ( k , l ) N 1 N 2 , ( k , l ) ( i , j ) ; Y 1 ̱ = y 1 ̱ , , Y m ̱ = y m ̱ ] = exp { r = 1 m { A A r } ψ r ̱ k , l 2 c C , ( i , j ) c V c ( x ̱ ) } x i , j exp { r = 1 m { A A r } ψ r ̱ k , l 2 c C , ( i , j ) c V x ( x ̱ ) } ,
η i , j p = η i , j x r = 1 m { ( k , l ) { A A r } ζ k , l y r } .

Metrics