Abstract

We present a new approach for visual saliency detection from various natural images. It is inspired by our careful observation that the human visual system (HVS) responds sensitively and quickly to high textural contrast, derived from the discriminative directional pattern from its surroundings as well as the noticeable luminance difference, for understanding a given scene. By formulating such textural contrast within a multiscale framework, we construct a more reliable saliency map even without color information when compared to most previous approaches still suffering from the complex and cluttered background. The proposed method has been extensively tested on a wide range of natural images, and experimental results show that the proposed scheme is effective in detecting visual saliency compared to various state-of-the-art methods.

© 2012 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Machine Intell. 20, 1254 (1998).
    [CrossRef]
  2. Y. F. Ma and H. J. Zhang, in Proceedings of ACM International Conference on Multimedia, 374 (2003).
  3. J. Harel, C. Koch, and P. Perona, in Proceedings of Advances in Neural Informational Processing Systems, 545 (2007).
  4. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).
  5. S. Goferman, L. Z. Manor, and A. Tal, Proc. IEEE, 2376 (2010).
  6. Y. Xu, Y. Zhao, C. Jin, J. Qu, L. Liu, and X. Sun, Opt. Lett. 35, 475 (2010).
    [CrossRef]
  7. W. Kim, C. Jung, and C. Kim, IEEE Trans. Circuits Syst. Video Technol. 21, 446 (2011).
    [CrossRef]
  8. D. Gao, V. Mahadevan, and N. Vasconcelos, J. Vision 8, 13 (2008).
    [CrossRef]
  9. X. Hou and L. Zhang, Proc. IEEE, 1 (2007).
  10. C. Guo and L. Zhang, IEEE Trans. Image Process. 15, 185 (2010).
  11. T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).
  12. M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
    [CrossRef]

2011 (1)

W. Kim, C. Jung, and C. Kim, IEEE Trans. Circuits Syst. Video Technol. 21, 446 (2011).
[CrossRef]

2010 (4)

C. Guo and L. Zhang, IEEE Trans. Image Process. 15, 185 (2010).

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

Y. Xu, Y. Zhao, C. Jin, J. Qu, L. Liu, and X. Sun, Opt. Lett. 35, 475 (2010).
[CrossRef]

S. Goferman, L. Z. Manor, and A. Tal, Proc. IEEE, 2376 (2010).

2009 (1)

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).

2008 (1)

D. Gao, V. Mahadevan, and N. Vasconcelos, J. Vision 8, 13 (2008).
[CrossRef]

2007 (2)

X. Hou and L. Zhang, Proc. IEEE, 1 (2007).

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

1998 (1)

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Machine Intell. 20, 1254 (1998).
[CrossRef]

Achanta, R.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).

Estrada, F.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).

Everingham, M.

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

Gao, D.

D. Gao, V. Mahadevan, and N. Vasconcelos, J. Vision 8, 13 (2008).
[CrossRef]

Goferman, S.

S. Goferman, L. Z. Manor, and A. Tal, Proc. IEEE, 2376 (2010).

Gool, L. V.

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

Guo, C.

C. Guo and L. Zhang, IEEE Trans. Image Process. 15, 185 (2010).

Harel, J.

J. Harel, C. Koch, and P. Perona, in Proceedings of Advances in Neural Informational Processing Systems, 545 (2007).

Hemami, S.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).

Hou, X.

X. Hou and L. Zhang, Proc. IEEE, 1 (2007).

Itti, L.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Machine Intell. 20, 1254 (1998).
[CrossRef]

Jin, C.

Jung, C.

W. Kim, C. Jung, and C. Kim, IEEE Trans. Circuits Syst. Video Technol. 21, 446 (2011).
[CrossRef]

Kim, C.

W. Kim, C. Jung, and C. Kim, IEEE Trans. Circuits Syst. Video Technol. 21, 446 (2011).
[CrossRef]

Kim, W.

W. Kim, C. Jung, and C. Kim, IEEE Trans. Circuits Syst. Video Technol. 21, 446 (2011).
[CrossRef]

Koch, C.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Machine Intell. 20, 1254 (1998).
[CrossRef]

J. Harel, C. Koch, and P. Perona, in Proceedings of Advances in Neural Informational Processing Systems, 545 (2007).

Liu, L.

Liu, T.

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

Ma, Y. F.

Y. F. Ma and H. J. Zhang, in Proceedings of ACM International Conference on Multimedia, 374 (2003).

Mahadevan, V.

D. Gao, V. Mahadevan, and N. Vasconcelos, J. Vision 8, 13 (2008).
[CrossRef]

Manor, L. Z.

S. Goferman, L. Z. Manor, and A. Tal, Proc. IEEE, 2376 (2010).

Niebur, E.

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Machine Intell. 20, 1254 (1998).
[CrossRef]

Perona, P.

J. Harel, C. Koch, and P. Perona, in Proceedings of Advances in Neural Informational Processing Systems, 545 (2007).

Qu, J.

Shum, H. -Y.

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

Sun, J.

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

Sun, X.

Susstrunk, S.

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).

Tal, A.

S. Goferman, L. Z. Manor, and A. Tal, Proc. IEEE, 2376 (2010).

Tang, X.

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

Vasconcelos, N.

D. Gao, V. Mahadevan, and N. Vasconcelos, J. Vision 8, 13 (2008).
[CrossRef]

Williams, C.

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

Winn, J.

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

Xu, Y.

Zhang, H. J.

Y. F. Ma and H. J. Zhang, in Proceedings of ACM International Conference on Multimedia, 374 (2003).

Zhang, L.

C. Guo and L. Zhang, IEEE Trans. Image Process. 15, 185 (2010).

X. Hou and L. Zhang, Proc. IEEE, 1 (2007).

Zhao, Y.

Zheng, N. -N.

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

Zisserman, A.

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

IEEE Trans. Circuits Syst. Video Technol. (1)

W. Kim, C. Jung, and C. Kim, IEEE Trans. Circuits Syst. Video Technol. 21, 446 (2011).
[CrossRef]

IEEE Trans. Image Process. (1)

C. Guo and L. Zhang, IEEE Trans. Image Process. 15, 185 (2010).

IEEE Trans. Pattern Anal. Machine Intell. (1)

L. Itti, C. Koch, and E. Niebur, IEEE Trans. Pattern Anal. Machine Intell. 20, 1254 (1998).
[CrossRef]

Int. J. Comput. Vis. (1)

M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, Int. J. Comput. Vis., 88, 303 (2010).
[CrossRef]

J. Vision (1)

D. Gao, V. Mahadevan, and N. Vasconcelos, J. Vision 8, 13 (2008).
[CrossRef]

Opt. Lett. (1)

Proc. IEEE (4)

X. Hou and L. Zhang, Proc. IEEE, 1 (2007).

T. Liu, J. Sun, N. -N. Zheng, X. Tang, and H. -Y. Shum, Proc. IEEE, 1 (2007).

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, Proc. IEEE, 1597 (2009).

S. Goferman, L. Z. Manor, and A. Tal, Proc. IEEE, 2376 (2010).

Other (2)

Y. F. Ma and H. J. Zhang, in Proceedings of ACM International Conference on Multimedia, 374 (2003).

J. Harel, C. Koch, and P. Perona, in Proceedings of Advances in Neural Informational Processing Systems, 545 (2007).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1.

(a) Original image, (b) color contrast by frequency-tuned model [4], (c) first-order model, (d) second-order model, and (e) fourth-order model.

Fig. 2.
Fig. 2.

Gradients obtained from selected image patches are illustrated. Note that λ1 and λ2 represent the energy along the dominant orientation of the gradient field and its perpendicular direction, respectively.

Fig. 3.
Fig. 3.

Scale-invariant saliency map. Note that the saliency map computed at each scale is resized to the size of the original image.

Fig. 4.
Fig. 4.

Some examples of visual saliency detection. (a) Original image, (b) ground truth, (c) saliency tool box (STB) [1], (d) local contrast method (LC) [2], (e) spectral residual (SR) [9], (f) graph-based visual saliency (GB) [3], (g) frequency-tuned method (FT) [4], (h) context-aware visual saliency (CA) [5], (i) spatial-frequency distribution (SFD), (j) difference of ordinal signatures (DOS) [7], (k) Proposed method (TC).

Fig. 5.
Fig. 5.

ROC curve for the quantitative analysis.

Tables (1)

Tables Icon

Table 1. Performance Comparison of the Processing Time (Most Images Have Resolution 400×300 on Our Database)

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

C(n)(i)=|Iμ1NjBiI(j)|n,
T(i)=[jBiIx2(j)jBiIx(j)Iy(j)jBiIx(j)Iy(j)jBiIy2(j)],
ϕ=(λ1λ2)2.
D(i)=jWi|ϕ(j)ϕ(i)|,
S(i)=C(2)(i)×D(i),
S˜(i)=1MrRSr(i),

Metrics