Abstract

In this paper, we analyzed eye fixation data obtained from 15 observers and 1003 images. When studying the eigen-decomposition of the correlation matrix constructed based on the fixation data of one observer viewing all images, it was observed that 23% of the data can be accounted for by one eigenvector. This finding implies a repeated viewing pattern that is independent of image content. Examination of this pattern revealed that it was highly correlated with the center region of the image. The presence of a repeated viewing pattern raised the following question: can we use the statistical information contained in the first eigenvector to filter out the fixations that were part of the pattern from those that are image feature dependent? To answer this question we designed a robust AUC metric that uses statistical analysis to better judge the goodness of the different saliency algorithms.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
    [CrossRef]
  2. C. Koch and S. Ullman, “Shifts in selective visual attention: towards the underlying neural circuitry,” Human Neurobiol. 4, 219–227 (1985).
  3. L. Itti and C. Koch, “A saliency-based search mechanism for overt and covert shifts of visual attention,” Vis. Res. 40, 1489–1506 (2000).
    [CrossRef]
  4. L. Itti and C. Koch, “Computational modelling of visual attention,” Nat. Rev. Neurosci. 2, 194–203 (2001).
    [CrossRef]
  5. D. Walther and C. Koch, “Modeling attention to salient proto-objects,” Neural Networks 19, 1395–1407 (2006).
    [CrossRef]
  6. M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Advances in Neural Information Processing Systems (NIPS) (2007), Vol. 20, pp. 241–248.
  7. J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Advances in Neural Information Processing Systems (NIPS) (2006).
  8. J. M. Henderson, J. R. Brockmole, M. S. Castelhano, and M. Mack, “Visual saliency does not account for eye movements during visual search in real-world scenes,” in Eye Movements: A Window on Mind and Brain (Elsevier, 2007), pp. 537–562.
  9. U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
    [CrossRef]
  10. D. Walther, “Interactions of visual attention and object recognition: computational modeling, algorithms, and psychophysics,” Ph.D. thesis (California Institute of Technology, 2006).
  11. J. M. Henderson, “Human gaze control during real-world scene perception,” Trends Cogn. Sci. 7, 498–504 (2003).
    [CrossRef]
  12. A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson, “Top-down control of visual attention in object detection,” in Proceedings of International Conference on Image Processing (IEEE, 2003), pp. 253–256.
  13. D. Parkhurst, K. Law, and E. Niebur, “Modeling the role of salience in the allocation of overt visual attention,” Vis. Res. 42, 107–123 (2002).
    [CrossRef]
  14. B. W. Tatler, “The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions,” J. Vis. 7(14):4 (2007).
    [CrossRef]
  15. B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: effects of scale and time,” Vis. Res. 45, 643–659 (2005).
    [CrossRef]
  16. T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in International Conference on Computer Vision (ICCV) (IEEE, 2009).
  17. R. Rosenholtz, “A simple saliency model predicts a number of motion popout phenomena,” Vis. Res. 39, 3157–3163 (1999).
    [CrossRef]
  18. G. Golub and W. Kahan, “Calculating the singular values and pseudo-inverse of a matrix,” J. Soc. Ind. Appl. Math. Ser. B 2, 205–224 (1965).
    [CrossRef]
  19. D. Kalman, “A singularly valuable decomposition: the SVD of a matrix,” College Math J. 27, 2–23 (1996).
    [CrossRef]
  20. A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 185–207 (2013).
    [CrossRef]
  21. T. Fawcett, “ROC graphs with instance-varying costs,” Pattern Recogn. Lett. 27, 882–891 (2006).
    [CrossRef]
  22. O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
    [CrossRef]
  23. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
    [CrossRef]
  24. M. Cerf, E. P. Frady, and C. Koch, “Faces and text attract gaze independent of the task: experimental data and computer model,” J. Vis. 9(12):10 (2009).
    [CrossRef]
  25. N. D. B. Bruce and J. K. Tsotsos, “Saliency based on information maximization,” in Advances in Neural Information Processing Systems (NIPS) (2005), pp. 155–162.
  26. A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
    [CrossRef]
  27. E. Erdem and A. Erdem, “Visual saliency estimation by nonlinearly integrating features using region covariances,” J. Vis. 13(4):11 (2013).
    [CrossRef]
  28. X. Hou and L. Zhang, “Saliency detection: a spectral residual approach,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8.
  29. B. Schauerte and R. Stiefelhagen, “Predicting human gaze using quaternion dct image signature saliency and face detection,” in Proceedings of the IEEE Workshop on the Applications of Computer Vision (WACV), Breckenridge, Colorado, January9–11, 2012 (IEEE, 2012).
  30. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
    [CrossRef]
  31. A. Alsam, P. Sharma, and A. Wrlsen, “Asymmetry as a measure of visual saliency,” in 18th Scandinavian Conference on Image Analysis (SCIA), Espoo, Finland (2013).
  32. A. Borji and L. Itti, “Exploiting local and global patch rarities for saliency detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island (2012).
  33. A. Borji, D. N. Sihite, and L. Itti, “Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study,” IEEE Trans. Image Process. 22, 55–69 (2013).
    [CrossRef]

2013 (3)

A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 185–207 (2013).
[CrossRef]

E. Erdem and A. Erdem, “Visual saliency estimation by nonlinearly integrating features using region covariances,” J. Vis. 13(4):11 (2013).
[CrossRef]

A. Borji, D. N. Sihite, and L. Itti, “Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study,” IEEE Trans. Image Process. 22, 55–69 (2013).
[CrossRef]

2012 (1)

A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
[CrossRef]

2009 (1)

M. Cerf, E. P. Frady, and C. Koch, “Faces and text attract gaze independent of the task: experimental data and computer model,” J. Vis. 9(12):10 (2009).
[CrossRef]

2008 (3)

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
[CrossRef]

U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
[CrossRef]

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

2007 (1)

B. W. Tatler, “The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions,” J. Vis. 7(14):4 (2007).
[CrossRef]

2006 (3)

D. Walther and C. Koch, “Modeling attention to salient proto-objects,” Neural Networks 19, 1395–1407 (2006).
[CrossRef]

T. Fawcett, “ROC graphs with instance-varying costs,” Pattern Recogn. Lett. 27, 882–891 (2006).
[CrossRef]

O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
[CrossRef]

2005 (1)

B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: effects of scale and time,” Vis. Res. 45, 643–659 (2005).
[CrossRef]

2003 (1)

J. M. Henderson, “Human gaze control during real-world scene perception,” Trends Cogn. Sci. 7, 498–504 (2003).
[CrossRef]

2002 (1)

D. Parkhurst, K. Law, and E. Niebur, “Modeling the role of salience in the allocation of overt visual attention,” Vis. Res. 42, 107–123 (2002).
[CrossRef]

2001 (1)

L. Itti and C. Koch, “Computational modelling of visual attention,” Nat. Rev. Neurosci. 2, 194–203 (2001).
[CrossRef]

2000 (1)

L. Itti and C. Koch, “A saliency-based search mechanism for overt and covert shifts of visual attention,” Vis. Res. 40, 1489–1506 (2000).
[CrossRef]

1999 (1)

R. Rosenholtz, “A simple saliency model predicts a number of motion popout phenomena,” Vis. Res. 39, 3157–3163 (1999).
[CrossRef]

1998 (1)

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

1996 (1)

D. Kalman, “A singularly valuable decomposition: the SVD of a matrix,” College Math J. 27, 2–23 (1996).
[CrossRef]

1985 (1)

C. Koch and S. Ullman, “Shifts in selective visual attention: towards the underlying neural circuitry,” Human Neurobiol. 4, 219–227 (1985).

1965 (1)

G. Golub and W. Kahan, “Calculating the singular values and pseudo-inverse of a matrix,” J. Soc. Ind. Appl. Math. Ser. B 2, 205–224 (1965).
[CrossRef]

Alsam, A.

A. Alsam, P. Sharma, and A. Wrlsen, “Asymmetry as a measure of visual saliency,” in 18th Scandinavian Conference on Image Analysis (SCIA), Espoo, Finland (2013).

Baddeley, R. J.

B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: effects of scale and time,” Vis. Res. 45, 643–659 (2005).
[CrossRef]

Barba, D.

O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
[CrossRef]

Borji, A.

A. Borji, D. N. Sihite, and L. Itti, “Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study,” IEEE Trans. Image Process. 22, 55–69 (2013).
[CrossRef]

A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 185–207 (2013).
[CrossRef]

A. Borji and L. Itti, “Exploiting local and global patch rarities for saliency detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island (2012).

Bovik, A. C.

U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
[CrossRef]

Brockmole, J. R.

J. M. Henderson, J. R. Brockmole, M. S. Castelhano, and M. Mack, “Visual saliency does not account for eye movements during visual search in real-world scenes,” in Eye Movements: A Window on Mind and Brain (Elsevier, 2007), pp. 537–562.

Bruce, N. D. B.

N. D. B. Bruce and J. K. Tsotsos, “Saliency based on information maximization,” in Advances in Neural Information Processing Systems (NIPS) (2005), pp. 155–162.

Castelhano, M. S.

J. M. Henderson, J. R. Brockmole, M. S. Castelhano, and M. Mack, “Visual saliency does not account for eye movements during visual search in real-world scenes,” in Eye Movements: A Window on Mind and Brain (Elsevier, 2007), pp. 537–562.

A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson, “Top-down control of visual attention in object detection,” in Proceedings of International Conference on Image Processing (IEEE, 2003), pp. 253–256.

Cerf, M.

M. Cerf, E. P. Frady, and C. Koch, “Faces and text attract gaze independent of the task: experimental data and computer model,” J. Vis. 9(12):10 (2009).
[CrossRef]

M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Advances in Neural Information Processing Systems (NIPS) (2007), Vol. 20, pp. 241–248.

Cormack, L. K.

U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
[CrossRef]

Cottrell, G. W.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

Dosil, R.

A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
[CrossRef]

Durand, F.

T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in International Conference on Computer Vision (ICCV) (IEEE, 2009).

Ehinger, K.

T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in International Conference on Computer Vision (ICCV) (IEEE, 2009).

Einhauser, W.

M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Advances in Neural Information Processing Systems (NIPS) (2007), Vol. 20, pp. 241–248.

Erdem, A.

E. Erdem and A. Erdem, “Visual saliency estimation by nonlinearly integrating features using region covariances,” J. Vis. 13(4):11 (2013).
[CrossRef]

Erdem, E.

E. Erdem and A. Erdem, “Visual saliency estimation by nonlinearly integrating features using region covariances,” J. Vis. 13(4):11 (2013).
[CrossRef]

Fawcett, T.

T. Fawcett, “ROC graphs with instance-varying costs,” Pattern Recogn. Lett. 27, 882–891 (2006).
[CrossRef]

Fdez-Vidal, X. R.

A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
[CrossRef]

Frady, E. P.

M. Cerf, E. P. Frady, and C. Koch, “Faces and text attract gaze independent of the task: experimental data and computer model,” J. Vis. 9(12):10 (2009).
[CrossRef]

Freeman, W. T.

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
[CrossRef]

Garcia-Diaz, A.

A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
[CrossRef]

Gilchrist, I. D.

B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: effects of scale and time,” Vis. Res. 45, 643–659 (2005).
[CrossRef]

Golub, G.

G. Golub and W. Kahan, “Calculating the singular values and pseudo-inverse of a matrix,” J. Soc. Ind. Appl. Math. Ser. B 2, 205–224 (1965).
[CrossRef]

Harel, J.

J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Advances in Neural Information Processing Systems (NIPS) (2006).

M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Advances in Neural Information Processing Systems (NIPS) (2007), Vol. 20, pp. 241–248.

Henderson, J. M.

J. M. Henderson, “Human gaze control during real-world scene perception,” Trends Cogn. Sci. 7, 498–504 (2003).
[CrossRef]

J. M. Henderson, J. R. Brockmole, M. S. Castelhano, and M. Mack, “Visual saliency does not account for eye movements during visual search in real-world scenes,” in Eye Movements: A Window on Mind and Brain (Elsevier, 2007), pp. 537–562.

A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson, “Top-down control of visual attention in object detection,” in Proceedings of International Conference on Image Processing (IEEE, 2003), pp. 253–256.

Hou, X.

X. Hou and L. Zhang, “Saliency detection: a spectral residual approach,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8.

Itti, L.

A. Borji, D. N. Sihite, and L. Itti, “Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study,” IEEE Trans. Image Process. 22, 55–69 (2013).
[CrossRef]

A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 185–207 (2013).
[CrossRef]

L. Itti and C. Koch, “Computational modelling of visual attention,” Nat. Rev. Neurosci. 2, 194–203 (2001).
[CrossRef]

L. Itti and C. Koch, “A saliency-based search mechanism for overt and covert shifts of visual attention,” Vis. Res. 40, 1489–1506 (2000).
[CrossRef]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

A. Borji and L. Itti, “Exploiting local and global patch rarities for saliency detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island (2012).

Judd, T.

T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in International Conference on Computer Vision (ICCV) (IEEE, 2009).

Kahan, W.

G. Golub and W. Kahan, “Calculating the singular values and pseudo-inverse of a matrix,” J. Soc. Ind. Appl. Math. Ser. B 2, 205–224 (1965).
[CrossRef]

Kalman, D.

D. Kalman, “A singularly valuable decomposition: the SVD of a matrix,” College Math J. 27, 2–23 (1996).
[CrossRef]

Koch, C.

M. Cerf, E. P. Frady, and C. Koch, “Faces and text attract gaze independent of the task: experimental data and computer model,” J. Vis. 9(12):10 (2009).
[CrossRef]

D. Walther and C. Koch, “Modeling attention to salient proto-objects,” Neural Networks 19, 1395–1407 (2006).
[CrossRef]

L. Itti and C. Koch, “Computational modelling of visual attention,” Nat. Rev. Neurosci. 2, 194–203 (2001).
[CrossRef]

L. Itti and C. Koch, “A saliency-based search mechanism for overt and covert shifts of visual attention,” Vis. Res. 40, 1489–1506 (2000).
[CrossRef]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

C. Koch and S. Ullman, “Shifts in selective visual attention: towards the underlying neural circuitry,” Human Neurobiol. 4, 219–227 (1985).

M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Advances in Neural Information Processing Systems (NIPS) (2007), Vol. 20, pp. 241–248.

J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Advances in Neural Information Processing Systems (NIPS) (2006).

Law, K.

D. Parkhurst, K. Law, and E. Niebur, “Modeling the role of salience in the allocation of overt visual attention,” Vis. Res. 42, 107–123 (2002).
[CrossRef]

Le Callet, P.

O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
[CrossRef]

Le Meur, O.

O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
[CrossRef]

Mack, M.

J. M. Henderson, J. R. Brockmole, M. S. Castelhano, and M. Mack, “Visual saliency does not account for eye movements during visual search in real-world scenes,” in Eye Movements: A Window on Mind and Brain (Elsevier, 2007), pp. 537–562.

Marks, T. K.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

Murphy, K. P.

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
[CrossRef]

Niebur, E.

D. Parkhurst, K. Law, and E. Niebur, “Modeling the role of salience in the allocation of overt visual attention,” Vis. Res. 42, 107–123 (2002).
[CrossRef]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

Oliva, A.

A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson, “Top-down control of visual attention in object detection,” in Proceedings of International Conference on Image Processing (IEEE, 2003), pp. 253–256.

Pardo, X. M.

A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
[CrossRef]

Parkhurst, D.

D. Parkhurst, K. Law, and E. Niebur, “Modeling the role of salience in the allocation of overt visual attention,” Vis. Res. 42, 107–123 (2002).
[CrossRef]

Perona, P.

J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Advances in Neural Information Processing Systems (NIPS) (2006).

Rajashekar, U.

U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
[CrossRef]

Rosenholtz, R.

R. Rosenholtz, “A simple saliency model predicts a number of motion popout phenomena,” Vis. Res. 39, 3157–3163 (1999).
[CrossRef]

Russell, B. C.

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
[CrossRef]

Schauerte, B.

B. Schauerte and R. Stiefelhagen, “Predicting human gaze using quaternion dct image signature saliency and face detection,” in Proceedings of the IEEE Workshop on the Applications of Computer Vision (WACV), Breckenridge, Colorado, January9–11, 2012 (IEEE, 2012).

Shan, H.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

Sharma, P.

A. Alsam, P. Sharma, and A. Wrlsen, “Asymmetry as a measure of visual saliency,” in 18th Scandinavian Conference on Image Analysis (SCIA), Espoo, Finland (2013).

Sihite, D. N.

A. Borji, D. N. Sihite, and L. Itti, “Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study,” IEEE Trans. Image Process. 22, 55–69 (2013).
[CrossRef]

Stiefelhagen, R.

B. Schauerte and R. Stiefelhagen, “Predicting human gaze using quaternion dct image signature saliency and face detection,” in Proceedings of the IEEE Workshop on the Applications of Computer Vision (WACV), Breckenridge, Colorado, January9–11, 2012 (IEEE, 2012).

Tatler, B. W.

B. W. Tatler, “The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions,” J. Vis. 7(14):4 (2007).
[CrossRef]

B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: effects of scale and time,” Vis. Res. 45, 643–659 (2005).
[CrossRef]

Thoreau, D.

O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
[CrossRef]

Tong, M. H.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

Torralba, A.

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
[CrossRef]

T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in International Conference on Computer Vision (ICCV) (IEEE, 2009).

A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson, “Top-down control of visual attention in object detection,” in Proceedings of International Conference on Image Processing (IEEE, 2003), pp. 253–256.

Tsotsos, J. K.

N. D. B. Bruce and J. K. Tsotsos, “Saliency based on information maximization,” in Advances in Neural Information Processing Systems (NIPS) (2005), pp. 155–162.

Ullman, S.

C. Koch and S. Ullman, “Shifts in selective visual attention: towards the underlying neural circuitry,” Human Neurobiol. 4, 219–227 (1985).

van der Linde, I.

U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
[CrossRef]

Walther, D.

D. Walther and C. Koch, “Modeling attention to salient proto-objects,” Neural Networks 19, 1395–1407 (2006).
[CrossRef]

D. Walther, “Interactions of visual attention and object recognition: computational modeling, algorithms, and psychophysics,” Ph.D. thesis (California Institute of Technology, 2006).

Wrlsen, A.

A. Alsam, P. Sharma, and A. Wrlsen, “Asymmetry as a measure of visual saliency,” in 18th Scandinavian Conference on Image Analysis (SCIA), Espoo, Finland (2013).

Zhang, L.

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

X. Hou and L. Zhang, “Saliency detection: a spectral residual approach,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8.

College Math J. (1)

D. Kalman, “A singularly valuable decomposition: the SVD of a matrix,” College Math J. 27, 2–23 (1996).
[CrossRef]

Human Neurobiol. (1)

C. Koch and S. Ullman, “Shifts in selective visual attention: towards the underlying neural circuitry,” Human Neurobiol. 4, 219–227 (1985).

IEEE Trans. Image Process. (2)

U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, “Gaffe: a gaze-attentive fixation finding engine,” IEEE Trans. Image Process. 17, 564—573 (2008).
[CrossRef]

A. Borji, D. N. Sihite, and L. Itti, “Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study,” IEEE Trans. Image Process. 22, 55–69 (2013).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (3)

A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 185–207 (2013).
[CrossRef]

O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, “A coherent computational approach to model bottom-up visual attention,” IEEE Trans. Pattern Anal. Mach. Intell. 28, 802–817 (2006).
[CrossRef]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998).
[CrossRef]

Image Vis. Comput. (1)

A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, “Saliency from hierarchical adaptation through decorrelation and variance normalization,” Image Vis. Comput. 30, 51–64 (2012).
[CrossRef]

Int. J. Comput. Vis. (1)

B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, “LabelMe: a database and web-based tool for image annotation,” Int. J. Comput. Vis. 77, 157–173 (2008).
[CrossRef]

J. Soc. Ind. Appl. Math. Ser. B (1)

G. Golub and W. Kahan, “Calculating the singular values and pseudo-inverse of a matrix,” J. Soc. Ind. Appl. Math. Ser. B 2, 205–224 (1965).
[CrossRef]

J. Vis. (4)

B. W. Tatler, “The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions,” J. Vis. 7(14):4 (2007).
[CrossRef]

M. Cerf, E. P. Frady, and C. Koch, “Faces and text attract gaze independent of the task: experimental data and computer model,” J. Vis. 9(12):10 (2009).
[CrossRef]

E. Erdem and A. Erdem, “Visual saliency estimation by nonlinearly integrating features using region covariances,” J. Vis. 13(4):11 (2013).
[CrossRef]

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “Sun: a Bayesian framework for saliency using natural statistics,” J. Vis. 8(7):32 (2008).
[CrossRef]

Nat. Rev. Neurosci. (1)

L. Itti and C. Koch, “Computational modelling of visual attention,” Nat. Rev. Neurosci. 2, 194–203 (2001).
[CrossRef]

Neural Networks (1)

D. Walther and C. Koch, “Modeling attention to salient proto-objects,” Neural Networks 19, 1395–1407 (2006).
[CrossRef]

Pattern Recogn. Lett. (1)

T. Fawcett, “ROC graphs with instance-varying costs,” Pattern Recogn. Lett. 27, 882–891 (2006).
[CrossRef]

Trends Cogn. Sci. (1)

J. M. Henderson, “Human gaze control during real-world scene perception,” Trends Cogn. Sci. 7, 498–504 (2003).
[CrossRef]

Vis. Res. (4)

D. Parkhurst, K. Law, and E. Niebur, “Modeling the role of salience in the allocation of overt visual attention,” Vis. Res. 42, 107–123 (2002).
[CrossRef]

B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: effects of scale and time,” Vis. Res. 45, 643–659 (2005).
[CrossRef]

R. Rosenholtz, “A simple saliency model predicts a number of motion popout phenomena,” Vis. Res. 39, 3157–3163 (1999).
[CrossRef]

L. Itti and C. Koch, “A saliency-based search mechanism for overt and covert shifts of visual attention,” Vis. Res. 40, 1489–1506 (2000).
[CrossRef]

Other (11)

D. Walther, “Interactions of visual attention and object recognition: computational modeling, algorithms, and psychophysics,” Ph.D. thesis (California Institute of Technology, 2006).

M. Cerf, J. Harel, W. Einhauser, and C. Koch, “Predicting human gaze using low-level saliency combined with face detection,” in Advances in Neural Information Processing Systems (NIPS) (2007), Vol. 20, pp. 241–248.

J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Advances in Neural Information Processing Systems (NIPS) (2006).

J. M. Henderson, J. R. Brockmole, M. S. Castelhano, and M. Mack, “Visual saliency does not account for eye movements during visual search in real-world scenes,” in Eye Movements: A Window on Mind and Brain (Elsevier, 2007), pp. 537–562.

T. Judd, K. Ehinger, F. Durand, and A. Torralba, “Learning to predict where humans look,” in International Conference on Computer Vision (ICCV) (IEEE, 2009).

A. Oliva, A. Torralba, M. S. Castelhano, and J. M. Henderson, “Top-down control of visual attention in object detection,” in Proceedings of International Conference on Image Processing (IEEE, 2003), pp. 253–256.

A. Alsam, P. Sharma, and A. Wrlsen, “Asymmetry as a measure of visual saliency,” in 18th Scandinavian Conference on Image Analysis (SCIA), Espoo, Finland (2013).

A. Borji and L. Itti, “Exploiting local and global patch rarities for saliency detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island (2012).

X. Hou and L. Zhang, “Saliency detection: a spectral residual approach,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8.

B. Schauerte and R. Stiefelhagen, “Predicting human gaze using quaternion dct image signature saliency and face detection,” in Proceedings of the IEEE Workshop on the Applications of Computer Vision (WACV), Breckenridge, Colorado, January9–11, 2012 (IEEE, 2012).

N. D. B. Bruce and J. K. Tsotsos, “Saliency based on information maximization,” in Advances in Neural Information Processing Systems (NIPS) (2005), pp. 155–162.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1.
Fig. 1.

Histogram obtained from the fixation data. (a) Distribution of fixations and (b) histogram of fixations.

Fig. 2.
Fig. 2.

Distribution of eigenvalues for observer No. 1 and different images. First dimension represents 25% of the data.

Fig. 3.
Fig. 3.

Distribution of eigenvalues for observer No. 2 and different images. First dimension represents 17% of the data.

Fig. 4.
Fig. 4.

Distribution of eigenvalues for observer No. 3 and different images. First dimension represents 20% of the data.

Fig. 5.
Fig. 5.

Distribution of eigenvalues for observer No. 4 and different images. First dimension represents 31% of the data.

Fig. 6.
Fig. 6.

Distribution of eigenvalues for an average observer. 23% of the data is captured by the first dimension.

Fig. 7.
Fig. 7.

Eigenvector for an average observer. It shows a concentration of fixations in the center region of the image. (a) Eigenvector for an average observer. (b) Probability histogram for the shared eigenvector.

Fig. 8.
Fig. 8.

Histogram of first eigenvalues for all the images where the mean, minimum, and maximum values of the distribution are 0.50, 0.25, and 0.92 respectively. It represents that the degree of agreement between observers varies from image to image.

Fig. 9.
Fig. 9.

Distribution of eigenvalues for an average image. 50% of the data is captured by the first dimension. 90% of the data is captured by the first five dimensions.

Fig. 10.
Fig. 10.

Category I: images together with their first vectors. Observers show good agreement on people, faces, and text. Img 1–16 used from the database by Judd et al. [16], and the LabelMe dataset by Russell et al. [23].

Fig. 11.
Fig. 11.

Category II: images together with their first vectors. Observers show poor agreement for complex images such as landscapes, buildings, and street views. Img 1–8 used from the database by Judd et al. [16], and the LabelMe dataset by Russell et al. [23].

Fig. 12.
Fig. 12.

Ranking of visual saliency models using the ordinary AUC metric.

Fig. 13.
Fig. 13.

Ranking of visual saliency models using the shuffled AUC metric.

Fig. 14.
Fig. 14.

Ranking of visual saliency models using the proposed AUC metric.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

A=[V1V2V3Vn],
C=ATA,
C=USVT,

Metrics