Abstract

We propose an efficient regional histogram (RH)-based computation model for saliency detection in natural images. First, the global histogram is constructed by performing an adaptive color quantization on the original image. Then multiple RHs are built on the basis of the region segmentation result, and the color–spatial similarity between each pixel and each RH is calculated accordingly. Two efficient measures, distinctiveness and compactness of each RH, are evaluated based on the color difference with the global histogram and the color distribution over the whole image, respectively. Finally, the pixel-level saliency map is generated by integrating the color–spatial similarity measures with the distinctiveness and compactness measures. Experimental results on a dataset containing 1000 test images with ground truths demonstrate that the proposed saliency model consistently outperforms state-of-the-art saliency models.

© 2013 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254 (1998).
    [CrossRef]
  2. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.
  3. S. Goferman, L. Zelnik-Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.
  4. W. Kim and C. Kim, Opt. Lett. 37, 1550 (2012).
    [CrossRef]
  5. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
    [CrossRef]
  6. J. Harel, C. Koch, and P. Perona, in Proceedings of NIPS (MIT, 2006) pp. 545–552.
  7. T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.
  8. X. Hou and L. Zhang, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270292.
  9. M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.
  10. Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
    [CrossRef]
  11. F. Perazzil, P. Krähenbül, Y. Pritch, and A. Hornung, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740.
  12. D. Comaniciu and P. Meer, IEEE Trans. Pattern Anal. Mach. Intell. 24, 603 (2002).
    [CrossRef]

2012

W. Kim and C. Kim, Opt. Lett. 37, 1550 (2012).
[CrossRef]

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

2008

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

2002

D. Comaniciu and P. Meer, IEEE Trans. Pattern Anal. Mach. Intell. 24, 603 (2002).
[CrossRef]

1998

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254 (1998).
[CrossRef]

Achanta, R.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Cheng, M.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Comaniciu, D.

D. Comaniciu and P. Meer, IEEE Trans. Pattern Anal. Mach. Intell. 24, 603 (2002).
[CrossRef]

Cottrell, G. W.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

Estrada, F.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Goferman, S.

S. Goferman, L. Zelnik-Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

Harel, J.

J. Harel, C. Koch, and P. Perona, in Proceedings of NIPS (MIT, 2006) pp. 545–552.

Hemami, S.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Hornung, A.

F. Perazzil, P. Krähenbül, Y. Pritch, and A. Hornung, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740.

Hou, X.

X. Hou and L. Zhang, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270292.

Hu, S.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Huang, X.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Itti, L.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254 (1998).
[CrossRef]

Kim, C.

Kim, W.

Koch, C.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254 (1998).
[CrossRef]

J. Harel, C. Koch, and P. Perona, in Proceedings of NIPS (MIT, 2006) pp. 545–552.

Krähenbül, P.

F. Perazzil, P. Krähenbül, Y. Pritch, and A. Hornung, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740.

Liu, T.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.

Liu, Z.

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

Marks, T. K.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

Meer, P.

D. Comaniciu and P. Meer, IEEE Trans. Pattern Anal. Mach. Intell. 24, 603 (2002).
[CrossRef]

Mitra, N.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Ngan, K. N.

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

Niebur, E.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254 (1998).
[CrossRef]

Perazzil, F.

F. Perazzil, P. Krähenbül, Y. Pritch, and A. Hornung, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740.

Perona, P.

J. Harel, C. Koch, and P. Perona, in Proceedings of NIPS (MIT, 2006) pp. 545–552.

Pritch, Y.

F. Perazzil, P. Krähenbül, Y. Pritch, and A. Hornung, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740.

Shan, H.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

Shen, L.

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

Shi, R.

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

Shum, H. Y.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.

Sun, J.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.

Susstrunk, S.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

Tal, A.

S. Goferman, L. Zelnik-Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

Tang, X.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.

Tong, M. H.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

Xue, Y.

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

Zelnik-Manor, L.

S. Goferman, L. Zelnik-Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

Zhang, G.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

Zhang, L.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

X. Hou and L. Zhang, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270292.

Zhang, Z.

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

Zheng, N.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.

IEEE Trans. Multimedia

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, IEEE Trans. Multimedia 14, 1275 (2012).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254 (1998).
[CrossRef]

D. Comaniciu and P. Meer, IEEE Trans. Pattern Anal. Mach. Intell. 24, 603 (2002).
[CrossRef]

J. Vis.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, J. Vis. 8(7), 32 (2008).
[CrossRef]

Opt. Lett.

Other

J. Harel, C. Koch, and P. Perona, in Proceedings of NIPS (MIT, 2006) pp. 545–552.

T. Liu, J. Sun, N. Zheng, X. Tang, and H. Y. Shum, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270072.

X. Hou and L. Zhang, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), paper 4270292.

M. Cheng, G. Zhang, N. Mitra, X. Huang, and S. Hu, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604.

S. Goferman, L. Zelnik-Manor, and A. Tal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 2376–2383.

F. Perazzil, P. Krähenbül, Y. Pritch, and A. Hornung, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1.
Fig. 1.

(a) Original image, (b) pixel-level distinctiveness map, (c) pixel-level compactness map, and (d) pixel-level saliency map.

Fig. 2.
Fig. 2.

(a) Illustration of global histogram and multiple RHs, (b) distinctiveness measures, and (c) compactness measures of RHs.

Fig. 3.
Fig. 3.

Some examples of saliency detection. From left to right: original images, ground truths, and saliency maps generated using CA, FT, RC, KD, SF, and RH, respectively.

Fig. 4.
Fig. 4.

Precision–recall curves of different saliency models.

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

C(p,Hi)=Hi(bp)·DH(p,Hi),
DH(p,Hi)=1xpμid,
RD(Hi)=j=1m[Hi(j)k=1mcjck·H0(k)],
RS(Hi)=pIC(p,Hi)·DO(p)pIC(p,Hi),
RC(Hi)=1RS(Hi).
DM(p)=i=1nC(p,Hi)·RD(Hi),
CM(p)=i=1nC(p,Hi)·RC(Hi).
SM(p)=DM(p)·CM(p).

Metrics