Abstract

We present an instance-based attention model to predict where humans could look first when searching for an object instance, and we show its application in image synthesis. The proposed model learns configurational rules from vast scene images described by global scene representations. The rules are then used to predict the focus of attention for the purpose of searching for a given object instance with special scale and pose. Finally, the image synthesis results are obtained by putting the object instance into the scene at the position that attracts most attention. Promising experimental results demonstrate the effectiveness of the proposed model.

© 2012 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. C. Ko and J.-Y. Nam, J. Opt. Soc. Am. A 23, 2462 (2006).
    [CrossRef]
  2. L. Itti, G. Gold, and C. Koch, Opt. Eng. 40, 1784 (2001).
    [CrossRef]
  3. V. Pimenov, J. Opt. Technol. 77, 684 (2010).
    [CrossRef]
  4. Y. Xu, Y. Zhao, C. Jin, Z. Qu, L. Liu, and X. Sun, Opt. Lett. 35, 475 (2010).
    [CrossRef]
  5. Y. Sun and R. Fisher, Artif. Intell. 146, 77 (2003).
    [CrossRef]
  6. A. Torralba, J. Opt. Soc. Am. A 20, 1407 (2003).
    [CrossRef]
  7. A. Oliva and A. Torralba, Prog. Brain Res. 155, 23 (2006).
    [CrossRef]
  8. C. Chang and C. Lin, http://www.csie.ntu.edu.tw/~jlin/libsvm.

2010 (2)

2006 (2)

B. C. Ko and J.-Y. Nam, J. Opt. Soc. Am. A 23, 2462 (2006).
[CrossRef]

A. Oliva and A. Torralba, Prog. Brain Res. 155, 23 (2006).
[CrossRef]

2003 (2)

A. Torralba, J. Opt. Soc. Am. A 20, 1407 (2003).
[CrossRef]

Y. Sun and R. Fisher, Artif. Intell. 146, 77 (2003).
[CrossRef]

2001 (1)

L. Itti, G. Gold, and C. Koch, Opt. Eng. 40, 1784 (2001).
[CrossRef]

Artif. Intell. (1)

Y. Sun and R. Fisher, Artif. Intell. 146, 77 (2003).
[CrossRef]

J. Opt. Soc. Am. A (2)

J. Opt. Technol. (1)

Opt. Eng. (1)

L. Itti, G. Gold, and C. Koch, Opt. Eng. 40, 1784 (2001).
[CrossRef]

Opt. Lett. (1)

Prog. Brain Res. (1)

A. Oliva and A. Torralba, Prog. Brain Res. 155, 23 (2006).
[CrossRef]

Other (1)

C. Chang and C. Lin, http://www.csie.ntu.edu.tw/~jlin/libsvm.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (3)

Fig. 1.
Fig. 1.

Results of putting a person instance into a scene. (a) the person instance; (b) the scene; (c) the first result of putting the instance into the scene; (d) the second result.

Fig. 2.
Fig. 2.

An overview of our instance-based attention model. We slide the object instance across the scene, and then we calculate the corresponding probability using the learned configurational rules. Focus of attention is given by applying a threshold.

Fig. 3.
Fig. 3.

Attention results of street scenes when searching for different car instances.

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

vo,s,k,l(I)=1NNWHu=kW/N(k+1)W/N1v=lH/N(l+1)H/N1Mo,s,
V(I)={vo,s,k,l(I)}.
Ri,j=Ri,j·F+T·(1F).
Ii,j(x,y)={Ri,j(xi+w2,yj+h2)(x,y)Ri,j,Ii,j(x,y)(x,y)Ri,j,
P=argmaxi,jci,j=argmaxi,jf(V(Ii,j)).

Metrics