Abstract

This Letter presents a computational model for saliency detection in natural images. While existing approaches usually make use of low-level or high-level visual features for establishing the saliency models, our method relies on midlevel visual cues, i.e., the superpixel representation of the image. In the proposed approach, the given image is first partitioned into superpixels. A fully connected superpixel graph is then constructed, and the random walk on the graph is adopted to measure saliency. In addition, a scheme based on multiple segmentations is used for multiscale processing. Our model has the advantage of generating high-resolution saliency maps with well-defined object borders. Experimental results on publicly available datasets demonstrate the proposed model can outperform the compared state-of-the-art saliency models.

© 2012 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Harel, C. Koch, and P. Perona, in Proceedings of the Advances in Neural Information Processing Systems (MIT, 2006), p. 545.
  2. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.
  3. W. Kim and C. Kim, Opt. Lett. 37, 1550 (2012).
    [CrossRef]
  4. S. Goferman, L. Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.
  5. M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.
  6. T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.
  7. C. Gao, N. Sang, and R. Huang, Opt. Lett. 37, 76 (2012).
    [CrossRef]
  8. P. Felzenszwalb and D. Huttenlocher, Int. J. Comput. Vis. 59, 167 (2004).
    [CrossRef]
  9. Y. Xie, H. Lu, and M.-H. Yang, “Bayesian saliency via low and mid-level cues,” IEEE Trans. Image Process. (to be published).
    [CrossRef]
  10. D. Aldous and J. Fill, http://stat-www.berkeley.edu/users/aldous/RWG/book.html , 1995.
  11. T. Malisiewicz and A. Efros, in Proceedings of the British Machine Vision Conference (BVMA, 2007), pp. 55.1–55.10.
  12. C. Christoudias, B. Georgescu, and P. Meer, in Proceeding of the IEEE 16th International Conference on Pattern Recognition (IEEE, 2002), pp. 150–155.

2012 (2)

2004 (1)

P. Felzenszwalb and D. Huttenlocher, Int. J. Comput. Vis. 59, 167 (2004).
[CrossRef]

Achanta, R.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Cheng, M.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Christoudias, C.

C. Christoudias, B. Georgescu, and P. Meer, in Proceeding of the IEEE 16th International Conference on Pattern Recognition (IEEE, 2002), pp. 150–155.

Efros, A.

T. Malisiewicz and A. Efros, in Proceedings of the British Machine Vision Conference (BVMA, 2007), pp. 55.1–55.10.

Estrada, F.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Felzenszwalb, P.

P. Felzenszwalb and D. Huttenlocher, Int. J. Comput. Vis. 59, 167 (2004).
[CrossRef]

Gao, C.

Georgescu, B.

C. Christoudias, B. Georgescu, and P. Meer, in Proceeding of the IEEE 16th International Conference on Pattern Recognition (IEEE, 2002), pp. 150–155.

Goferman, S.

S. Goferman, L. Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

Harel, J.

J. Harel, C. Koch, and P. Perona, in Proceedings of the Advances in Neural Information Processing Systems (MIT, 2006), p. 545.

Hemami, S.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Hu, S.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Huang, R.

Huang, X.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Huttenlocher, D.

P. Felzenszwalb and D. Huttenlocher, Int. J. Comput. Vis. 59, 167 (2004).
[CrossRef]

Kim, C.

Kim, W.

Koch, C.

J. Harel, C. Koch, and P. Perona, in Proceedings of the Advances in Neural Information Processing Systems (MIT, 2006), p. 545.

Liu, T.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.

Lu, H.

Y. Xie, H. Lu, and M.-H. Yang, “Bayesian saliency via low and mid-level cues,” IEEE Trans. Image Process. (to be published).
[CrossRef]

Malisiewicz, T.

T. Malisiewicz and A. Efros, in Proceedings of the British Machine Vision Conference (BVMA, 2007), pp. 55.1–55.10.

Manor, L.

S. Goferman, L. Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

Meer, P.

C. Christoudias, B. Georgescu, and P. Meer, in Proceeding of the IEEE 16th International Conference on Pattern Recognition (IEEE, 2002), pp. 150–155.

Mitra, N.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Perona, P.

J. Harel, C. Koch, and P. Perona, in Proceedings of the Advances in Neural Information Processing Systems (MIT, 2006), p. 545.

Sang, N.

Shum, H.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.

Sun, J.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.

Susstrunk, S.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Tal, A.

S. Goferman, L. Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

Tang, X.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.

Xie, Y.

Y. Xie, H. Lu, and M.-H. Yang, “Bayesian saliency via low and mid-level cues,” IEEE Trans. Image Process. (to be published).
[CrossRef]

Yang, M.-H.

Y. Xie, H. Lu, and M.-H. Yang, “Bayesian saliency via low and mid-level cues,” IEEE Trans. Image Process. (to be published).
[CrossRef]

Zhang, G.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Zheng, N.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.

Int. J. Comput. Vis. (1)

P. Felzenszwalb and D. Huttenlocher, Int. J. Comput. Vis. 59, 167 (2004).
[CrossRef]

Opt. Lett. (2)

Other (9)

S. Goferman, L. Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–4.

Y. Xie, H. Lu, and M.-H. Yang, “Bayesian saliency via low and mid-level cues,” IEEE Trans. Image Process. (to be published).
[CrossRef]

D. Aldous and J. Fill, http://stat-www.berkeley.edu/users/aldous/RWG/book.html , 1995.

T. Malisiewicz and A. Efros, in Proceedings of the British Machine Vision Conference (BVMA, 2007), pp. 55.1–55.10.

C. Christoudias, B. Georgescu, and P. Meer, in Proceeding of the IEEE 16th International Conference on Pattern Recognition (IEEE, 2002), pp. 150–155.

J. Harel, C. Koch, and P. Perona, in Proceedings of the Advances in Neural Information Processing Systems (MIT, 2006), p. 545.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1.
Fig. 1.

Illustration of the drawbacks of existing saliency models. From left to right are (a) original images, saliency maps obtained by, (b) GBVS [1], (c) FT [2], (d) CA [4], and (e) the proposed model, respectively.

Fig. 2.
Fig. 2.

Framework of the saliency model.

Fig. 3.
Fig. 3.

Qualitative results on the Achanta dataset. From left to right are original images, ground truth, the saliency maps obtained by CA [4], GBVS [1], FT [2], RC [5], and our method, respectively.

Fig. 4.
Fig. 4.

Quantitative results on the Achanta dataset. (a) Comparison to the state-of-the-art methods. (b) Comparison with different segmentation parameters. (c) The mean precision, recall, and F-measure obtained by the adaptive thresholding segmentation suggested in [2].

Tables (1)

Tables Icon

Table 1. Comparison of the Average Running Time (in Seconds)a

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

wij=ws(Ri,Rj)·da(Ri,Rj),
da(Ri,Rj)=hiTΣhj,
ws(Ri,Rj)=exp{ds2(Ri,Rj)/σs2},
pij=wijk=1nwik.
πj=i=1nπipij.

Metrics