Abstract

This paper presents a digital zooming method using a super-resolution (SR) algorithm based on the local self-similarity between the wide- and tele-view images acquired by an asymmetric dual camera system. The proposed SR algorithm consists of four steps: (i) registration of an optically zoomed image to the wide-view image, (ii) restoration of the central region of the zoomed wide-view image, (iii) restoration of the boundary region of the zoomed wide-view image, and (iv) fusion of the results from steps (ii) and (iii). Since an asymmetric dual camera system acquires different-resolution images on the same scene due to the different optical specifications, the proposed method can restore the low-resolution wide-view image using the ideal high-frequency component estimated from the optically zoomed image. Experimental results demonstrate that the proposed method can provide significantly improved high-resolution wide-view images compared to existing single-image-based SR methods.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Single image super-resolution using locally adaptive multiple linear regression

Soohwan Yu, Wonseok Kang, Seungyong Ko, and Joonki Paik
J. Opt. Soc. Am. A 32(12) 2264-2275 (2015)

Simultaneous digital super-resolution and nonuniformity correction for infrared imaging systems

Pablo Meza, Guillermo Machuca, Sergio Torres, Cesar San Martin, and Esteban Vera
Appl. Opt. 54(21) 6508-6515 (2015)

Controlled angular and radial scanning for super resolution concentric circular imaging

Xian Du and Brian Anthony
Opt. Express 24(20) 22581-22595 (2016)

References

  • View by:
  • |
  • |
  • |

  1. S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
    [Crossref]
  2. K. Nasrollahi and T. B. Moeslund, “Super-resolution: a comprehensive survey,” Mach. Vis. Appl. 25, 1423–1468 (2014).
    [Crossref]
  3. R. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vis. Image Process. 1, 317–339 (1984).
  4. S. Jeong, I. Yoon, and J. Paik, “Multi-frame example-based super-resolution using locally directional self-similarity,” IEEE Trans. Consum. Electron. 61, 353–358 (2015).
    [Crossref]
  5. C. Liu and D. Sun, “A Bayesian approach to adaptive video super resolution,” in IEEE Computer Vision and Pattern Recognition (IEEE, 2011), pp. 209–216.
  6. W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appl. 22, 56–65 (2002).
    [Crossref]
  7. N. Suetake, M. Sakano, and E. Uchino, “Image super-resolution based on local self-similarity,” Opt. Rev. 15, 26–30 (2008).
    [Crossref]
  8. D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision (IEEE, 2009), pp. 349–356.
  9. J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
    [Crossref]
  10. G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graph. 30, 1–11 (2011).
    [Crossref]
  11. R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1920–1927.
  12. J. Sun, N. N. Zheng, H. Tao, and H.-Y. Shum, “Image hallucination with primal sketch priors,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003) Vol. 2, II–729–II–736.
  13. R. Fattal, “Image upsampling via imposed edge statistics,” ACM Trans. Graph. 26, 95 (2007).
    [Crossref]
  14. J. Sun, Z. Xu, and H.-Y. Shum, “Image super-resolution using gradient profile prior,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.
  15. M. Zontak and M. Irani, “Internal statistics of a single natural image,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 977–984.
  16. F. Chen, L. Zhang, and H. Yu, “External patch prior guided internal clustering for image denoising,” in IEEE International Conference on Computer Vision (IEEE, 2015), pp. 603–611.
  17. D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
    [Crossref]
  18. J. Park, D. Kim, D. Kim, and J. Paik, “Non-dyadic fisheye lens correction model for image enhancement,” J. Opt. Soc. Am. A 32, 2148–2155 (2015).
    [Crossref]
  19. E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
    [Crossref]
  20. A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: survey and experiments,” IEEE Trans. Image Process. 20, 2475–2489 (2011).
    [Crossref]
  21. R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Trans. Acoust. Speech Signal Process. 29, 1153–1160 (1981).
    [Crossref]
  22. H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: speeded up robust features,” in European Conference on Computer Vision (Springer, 2006), pp. 404–417.
  23. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
    [Crossref]
  24. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).
  25. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
    [Crossref]
  26. C. Yang, J. Huang, and M. Yang, “Exploiting self-similarities for single frame super-resolution,” in Asian Conference on Computer Vision (Springer, 2010), pp. 497–510.
  27. S. Yu, W. Kang, S. Ko, and J. Paik, “Single image super-resolution using locally adaptive multiple linear regression,” J. Opt. Soc. Am. A 32, 2264–2275 (2015).
    [Crossref]
  28. C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
    [Crossref]

2016 (1)

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
[Crossref]

2015 (3)

2014 (2)

K. Nasrollahi and T. B. Moeslund, “Super-resolution: a comprehensive survey,” Mach. Vis. Appl. 25, 1423–1468 (2014).
[Crossref]

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

2011 (2)

A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: survey and experiments,” IEEE Trans. Image Process. 20, 2475–2489 (2011).
[Crossref]

G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graph. 30, 1–11 (2011).
[Crossref]

2010 (1)

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[Crossref]

2008 (1)

N. Suetake, M. Sakano, and E. Uchino, “Image super-resolution based on local self-similarity,” Opt. Rev. 15, 26–30 (2008).
[Crossref]

2007 (1)

R. Fattal, “Image upsampling via imposed edge statistics,” ACM Trans. Graph. 26, 95 (2007).
[Crossref]

2004 (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
[Crossref]

2003 (1)

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

2002 (1)

W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appl. 22, 56–65 (2002).
[Crossref]

2001 (1)

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
[Crossref]

1984 (1)

R. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vis. Image Process. 1, 317–339 (1984).

1981 (2)

R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Trans. Acoust. Speech Signal Process. 29, 1153–1160 (1981).
[Crossref]

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Adhikhmin, M.

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
[Crossref]

Bagon, S.

D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision (IEEE, 2009), pp. 349–356.

Bay, H.

H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: speeded up robust features,” in European Conference on Computer Vision (Springer, 2006), pp. 404–417.

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Bovik, A. C.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
[Crossref]

Chen, F.

F. Chen, L. Zhang, and H. Yu, “External patch prior guided internal clustering for image denoising,” in IEEE International Conference on Computer Vision (IEEE, 2015), pp. 603–611.

De Smet, V.

R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1920–1927.

Dong, C.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
[Crossref]

Fattal, R.

G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graph. 30, 1–11 (2011).
[Crossref]

R. Fattal, “Image upsampling via imposed edge statistics,” ACM Trans. Graph. 26, 95 (2007).
[Crossref]

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Freedman, G.

G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graph. 30, 1–11 (2011).
[Crossref]

Freeman, W. T.

W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appl. 22, 56–65 (2002).
[Crossref]

Gevers, T.

A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: survey and experiments,” IEEE Trans. Image Process. 20, 2475–2489 (2011).
[Crossref]

Gijsenij, A.

A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: survey and experiments,” IEEE Trans. Image Process. 20, 2475–2489 (2011).
[Crossref]

Glasner, D.

D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision (IEEE, 2009), pp. 349–356.

Gonzalez, R. C.

R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).

Gooch, B.

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
[Crossref]

He, K.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
[Crossref]

Huang, J.

C. Yang, J. Huang, and M. Yang, “Exploiting self-similarities for single frame super-resolution,” in Asian Conference on Computer Vision (Springer, 2010), pp. 497–510.

Huang, T. S.

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[Crossref]

R. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vis. Image Process. 1, 317–339 (1984).

Irani, M.

M. Zontak and M. Irani, “Internal statistics of a single natural image,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 977–984.

D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision (IEEE, 2009), pp. 349–356.

Jeong, S.

S. Jeong, I. Yoon, and J. Paik, “Multi-frame example-based super-resolution using locally directional self-similarity,” IEEE Trans. Consum. Electron. 61, 353–358 (2015).
[Crossref]

Jones, T. R.

W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appl. 22, 56–65 (2002).
[Crossref]

Jung, J.

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

Kang, M. G.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

Kang, W.

Keys, R.

R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Trans. Acoust. Speech Signal Process. 29, 1153–1160 (1981).
[Crossref]

Kim, D.

J. Park, D. Kim, D. Kim, and J. Paik, “Non-dyadic fisheye lens correction model for image enhancement,” J. Opt. Soc. Am. A 32, 2148–2155 (2015).
[Crossref]

J. Park, D. Kim, D. Kim, and J. Paik, “Non-dyadic fisheye lens correction model for image enhancement,” J. Opt. Soc. Am. A 32, 2148–2155 (2015).
[Crossref]

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

Kim, T.

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

Ko, S.

Liu, C.

C. Liu and D. Sun, “A Bayesian approach to adaptive video super resolution,” in IEEE Computer Vision and Pattern Recognition (IEEE, 2011), pp. 209–216.

Loy, C. C.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
[Crossref]

Ma, Y.

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[Crossref]

Moeslund, T. B.

K. Nasrollahi and T. B. Moeslund, “Super-resolution: a comprehensive survey,” Mach. Vis. Appl. 25, 1423–1468 (2014).
[Crossref]

Nasrollahi, K.

K. Nasrollahi and T. B. Moeslund, “Super-resolution: a comprehensive survey,” Mach. Vis. Appl. 25, 1423–1468 (2014).
[Crossref]

Paik, J.

S. Jeong, I. Yoon, and J. Paik, “Multi-frame example-based super-resolution using locally directional self-similarity,” IEEE Trans. Consum. Electron. 61, 353–358 (2015).
[Crossref]

S. Yu, W. Kang, S. Ko, and J. Paik, “Single image super-resolution using locally adaptive multiple linear regression,” J. Opt. Soc. Am. A 32, 2264–2275 (2015).
[Crossref]

J. Park, D. Kim, D. Kim, and J. Paik, “Non-dyadic fisheye lens correction model for image enhancement,” J. Opt. Soc. Am. A 32, 2148–2155 (2015).
[Crossref]

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

Park, J.

J. Park, D. Kim, D. Kim, and J. Paik, “Non-dyadic fisheye lens correction model for image enhancement,” J. Opt. Soc. Am. A 32, 2148–2155 (2015).
[Crossref]

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

Park, M. K.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

Park, S. C.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

Pasztor, E. C.

W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appl. 22, 56–65 (2002).
[Crossref]

Reinhard, E.

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
[Crossref]

Sakano, M.

N. Suetake, M. Sakano, and E. Uchino, “Image super-resolution based on local self-similarity,” Opt. Rev. 15, 26–30 (2008).
[Crossref]

Sheikh, H. R.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
[Crossref]

Shirley, P.

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
[Crossref]

Shum, H.-Y.

J. Sun, Z. Xu, and H.-Y. Shum, “Image super-resolution using gradient profile prior,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.

J. Sun, N. N. Zheng, H. Tao, and H.-Y. Shum, “Image hallucination with primal sketch priors,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003) Vol. 2, II–729–II–736.

Simoncelli, E. P.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
[Crossref]

Suetake, N.

N. Suetake, M. Sakano, and E. Uchino, “Image super-resolution based on local self-similarity,” Opt. Rev. 15, 26–30 (2008).
[Crossref]

Sun, D.

C. Liu and D. Sun, “A Bayesian approach to adaptive video super resolution,” in IEEE Computer Vision and Pattern Recognition (IEEE, 2011), pp. 209–216.

Sun, J.

J. Sun, Z. Xu, and H.-Y. Shum, “Image super-resolution using gradient profile prior,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.

J. Sun, N. N. Zheng, H. Tao, and H.-Y. Shum, “Image hallucination with primal sketch priors,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003) Vol. 2, II–729–II–736.

Tang, X.

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
[Crossref]

Tao, H.

J. Sun, N. N. Zheng, H. Tao, and H.-Y. Shum, “Image hallucination with primal sketch priors,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003) Vol. 2, II–729–II–736.

Timofte, R.

R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1920–1927.

Tsai, R.

R. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vis. Image Process. 1, 317–339 (1984).

Tuytelaars, T.

H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: speeded up robust features,” in European Conference on Computer Vision (Springer, 2006), pp. 404–417.

Uchino, E.

N. Suetake, M. Sakano, and E. Uchino, “Image super-resolution based on local self-similarity,” Opt. Rev. 15, 26–30 (2008).
[Crossref]

Van De Weijer, J.

A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: survey and experiments,” IEEE Trans. Image Process. 20, 2475–2489 (2011).
[Crossref]

Van Gool, L.

H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: speeded up robust features,” in European Conference on Computer Vision (Springer, 2006), pp. 404–417.

R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1920–1927.

Wang, Z.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
[Crossref]

Woods, R. E.

R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).

Wright, J.

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[Crossref]

Xu, Z.

J. Sun, Z. Xu, and H.-Y. Shum, “Image super-resolution using gradient profile prior,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.

Yang, C.

C. Yang, J. Huang, and M. Yang, “Exploiting self-similarities for single frame super-resolution,” in Asian Conference on Computer Vision (Springer, 2010), pp. 497–510.

Yang, J.

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[Crossref]

Yang, M.

C. Yang, J. Huang, and M. Yang, “Exploiting self-similarities for single frame super-resolution,” in Asian Conference on Computer Vision (Springer, 2010), pp. 497–510.

Yoon, I.

S. Jeong, I. Yoon, and J. Paik, “Multi-frame example-based super-resolution using locally directional self-similarity,” IEEE Trans. Consum. Electron. 61, 353–358 (2015).
[Crossref]

Yu, H.

F. Chen, L. Zhang, and H. Yu, “External patch prior guided internal clustering for image denoising,” in IEEE International Conference on Computer Vision (IEEE, 2015), pp. 603–611.

Yu, S.

Zhang, L.

F. Chen, L. Zhang, and H. Yu, “External patch prior guided internal clustering for image denoising,” in IEEE International Conference on Computer Vision (IEEE, 2015), pp. 603–611.

Zheng, N. N.

J. Sun, N. N. Zheng, H. Tao, and H.-Y. Shum, “Image hallucination with primal sketch priors,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003) Vol. 2, II–729–II–736.

Zontak, M.

M. Zontak and M. Irani, “Internal statistics of a single natural image,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 977–984.

ACM Trans. Graph. (2)

G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Trans. Graph. 30, 1–11 (2011).
[Crossref]

R. Fattal, “Image upsampling via imposed edge statistics,” ACM Trans. Graph. 26, 95 (2007).
[Crossref]

Adv. Comput. Vis. Image Process. (1)

R. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Adv. Comput. Vis. Image Process. 1, 317–339 (1984).

Commun. ACM (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

IEEE Comput. Graph. Appl. (2)

W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appl. 22, 56–65 (2002).
[Crossref]

E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Comput. Graph. Appl. 21, 34–41 (2001).
[Crossref]

IEEE Signal Process. Mag. (1)

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

IEEE Trans. Acoust. Speech Signal Process. (1)

R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Trans. Acoust. Speech Signal Process. 29, 1153–1160 (1981).
[Crossref]

IEEE Trans. Consum. Electron. (2)

S. Jeong, I. Yoon, and J. Paik, “Multi-frame example-based super-resolution using locally directional self-similarity,” IEEE Trans. Consum. Electron. 61, 353–358 (2015).
[Crossref]

D. Kim, J. Park, J. Jung, T. Kim, and J. Paik, “Lens distortion correction and enhancement based on local self-similarity for high-quality consumer imaging systems,” IEEE Trans. Consum. Electron. 60, 18–22 (2014).
[Crossref]

IEEE Trans. Image Process. (3)

J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010).
[Crossref]

A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: survey and experiments,” IEEE Trans. Image Process. 20, 2475–2489 (2011).
[Crossref]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13, 600–612 (2004).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).
[Crossref]

J. Opt. Soc. Am. A (2)

Mach. Vis. Appl. (1)

K. Nasrollahi and T. B. Moeslund, “Super-resolution: a comprehensive survey,” Mach. Vis. Appl. 25, 1423–1468 (2014).
[Crossref]

Opt. Rev. (1)

N. Suetake, M. Sakano, and E. Uchino, “Image super-resolution based on local self-similarity,” Opt. Rev. 15, 26–30 (2008).
[Crossref]

Other (10)

D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision (IEEE, 2009), pp. 349–356.

R. Timofte, V. De Smet, and L. Van Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1920–1927.

J. Sun, N. N. Zheng, H. Tao, and H.-Y. Shum, “Image hallucination with primal sketch priors,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2003) Vol. 2, II–729–II–736.

C. Liu and D. Sun, “A Bayesian approach to adaptive video super resolution,” in IEEE Computer Vision and Pattern Recognition (IEEE, 2011), pp. 209–216.

J. Sun, Z. Xu, and H.-Y. Shum, “Image super-resolution using gradient profile prior,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.

M. Zontak and M. Irani, “Internal statistics of a single natural image,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 977–984.

F. Chen, L. Zhang, and H. Yu, “External patch prior guided internal clustering for image denoising,” in IEEE International Conference on Computer Vision (IEEE, 2015), pp. 603–611.

C. Yang, J. Huang, and M. Yang, “Exploiting self-similarities for single frame super-resolution,” in Asian Conference on Computer Vision (Springer, 2010), pp. 497–510.

H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: speeded up robust features,” in European Conference on Computer Vision (Springer, 2006), pp. 404–417.

R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (17)

Fig. 1.
Fig. 1.

Image acquisition model of an asymmetric dual camera system.

Fig. 2.
Fig. 2.

Block diagram of the proposed SR method using the different-resolution images.

Fig. 3.
Fig. 3.

Registration process of the input wide- and tele-view images to guarantee the local self-similarity between two input images.

Fig. 4.
Fig. 4.

Comparison of the resolution of the interpolated wide- and registered tele-view images and the selected similar patches: (a), (b) cropped, magnified regions of the interpolated wide- and registered tele-view images, (c) selected similar patches from the input wide-view image, and (d) selected similar patches from the registered tele-view image.

Fig. 5.
Fig. 5.

Reduced search range to remove boundary artifacts: (a) search range of gTR, (b) blocking artifact in the boundary region, (c) search range in the center of gTR, and (d) restored result the without blocking artifact.

Fig. 6.
Fig. 6.

Comparison of a simple fusion and the proposed method: (a) zoomed wide-view image, (b) result of simple fusion of the zoomed wide- and registered tele-view images, and (c) fused image using the proposed method.

Fig. 7.
Fig. 7.

Test images for the objective image quality assessments.

Fig. 8.
Fig. 8.

Simulation of different-view and ideally zoomed images for a magnification ratio of 1.5 in an asymmetric dual camera system.

Fig. 9.
Fig. 9.

Comparison of the digital zoomed results by a magnification factor of 1.5: (a) simulated wide-view image, (b) simulated tele-view image, (c) simulated ideal image, (d) cubic-spline interpolation [21], (e) Yang’s method [26], (f) Yang’s method [9], (g) Timofte’s method [11], (h) Yu’s method [27], (i) Dong’s method [28], and (j) the proposed method (σB=0.6).

Fig. 10.
Fig. 10.

Comparison of the digital zoomed results by a magnification factor of 1.5: (a) simulated wide-view image, (b) simulated tele-view image, (c) simulated ideal image, (d) cubic-spline interpolation [21], (e) Yang’s method [26], (f) Yang’s method [9], (g) Timofte’s method [11], (h) Yu’s method [27], (i) Dong’s method [28], and (j) the proposed method (σB=0.6).

Fig. 11.
Fig. 11.

Comparison of the digital zoomed results by a magnification factor of 2.0: (a) simulated wide-view image, (b) simulated tele-view image, (c) simulated ideal image, (d) cubic-spline interpolation [21], (e) Yang’s method [26], (f) Yang’s method [9], (g) Timofte’s method [11], (h) Yu’s method [27], (i) Dong’s method [28], and (j) the proposed method (σB=0.9).

Fig. 12.
Fig. 12.

Comparison of the digital zoomed results by a magnification factor of 2.0: (a) simulated wide-view image, (b) simulated tele-view image, (c) simulated ideal image, (d) cubic-spline interpolation [21], (e) Yang’s method [26], (f) Yang’s method [9], (g) Timofte’s method [11], (h) Yu’s method [27], (i) Dong’s method [28], and (j) the proposed method (σB=0.9).

Fig. 13.
Fig. 13.

Comparative results using three different standard variations: (a) σB=0.3 (PSNR: 30.17 dB; SSIM: 0.9165), (b) σB=0.6 (PSNR: 33.06 dB; SSIM: 0.9525), and (c) σB=0.9 (PSNR: 30.93 dB; SSIM: 0.9411).

Fig. 14.
Fig. 14.

Comparison of the objective image quality assessments using various values of σB and patch sizes for magnification factors of 1.5 and 2.0: (a) the variation of the PSNR values using various values of σB, and (b) the variation of the PSNR values using various patch sizes.

Fig. 15.
Fig. 15.

Comparison of the digital zoomed results by a magnification factor of 1.5 using the real different-resolution images: (a) input wide-view image, (b) input tele-view image, (c) cubic-spline interpolation [21], (d) Yang’s method [26], (e) Yang’s method [9], (f) Timofte’s method [11], (g) Yu’s method [27], (h) Dong’s method [28], and (i) the proposed method (σB=0.8).

Fig. 16.
Fig. 16.

Comparison of the digital zoomed results by a magnification factor of 1.5 using the real different-resolution images: (a) input wide-view image, (b) input tele-view image, (c) cubic-spline interpolation [21], (d) Yang’s method [26], (e) Yang’s method [9], (f) Timofte’s method [11], (g) Yu’s method [27], (h) Dong’s method [28], and (i) the proposed method (σB=0.8).

Fig. 17.
Fig. 17.

Simulated input noisy images and zoomed result for a magnification factor of 1.5: (a) simulated noisy wide-view image, (b) simulated noisy tele-view image, and (c) the resulting image.

Tables (3)

Tables Icon

Table 1. Comparative Evaluation of the Objective Quality Assessments Using PSNR and SSIM [25] for the Magnification Ratio 1.5

Tables Icon

Table 2. Comparative Evaluation of the Objective Quality Assessments Using PSNR and SSIM [25] for the Magnification Ratio 2.0

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

gW(x)=HL(HS(f(x)))+ηW(x),
gT(x)=HI(HC(f(x)))+ηT(x),
gTR(xj)=G1gT(xi),
gT(xj)=GgWI(xi),
x^=argminx^ΩipC(gWD,xi)pC(gTR,xi+x^)22,
gTH(x)=gTR(x)gTL(x),
pC(f^WC,xi)=pC(gWI,xi)+pC(gTH,xi+x^).
gWH(x)=gW(x)gWL(x),
pB(f^WB,xi)=pB(gWI,xi)+pB(gWH,x˜i+x^),

Metrics