Abstract

A two-stage keypoint registration approach is proposed to achieve frame-rate performance, while maintaining high accuracy under large perspective and scale variations. First, an agglomerative clustering algorithm based on an effective edge significance measure is adopted to derive the corresponding regions for keypoint detection. Next, a light-weight detector and a compact descriptor are utilized to obtain the exact location of the keypoints. In conjunction with the point transferring method, the proposed approach can perform registration task in textureless regions robustly. Experiments are conducted to demonstrate that the approach can handle the real-time tracking tasks.

© 2009 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
    [CrossRef]
  2. K. Mikolajczyk and C. Schmid, “Performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005).
    [CrossRef] [PubMed]
  3. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
    [CrossRef]
  4. V. Lepetit and P. Fua, “Keypoint recognition using randomized trees,” IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006).
    [CrossRef] [PubMed]
  5. M. Özuysal, P. Fua, and V. Lepetit, “Fast keypoint recognition in ten lines of code,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Minneapolis, United States, 2007), pp. 1–8.
  6. J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
    [CrossRef]
  7. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
    [CrossRef]
  8. P. E. Forssén, “Maximally stable colour regions for recognition and matching,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Minneapolis, United States, 2007), pp. 1220–1227.
  9. C. Boncelet, “Image noise models,” in Handbook of image and video processing (second edition), A. C. Bovic, ed. (Elsevier Academic Press, San Diego, United States, 2005).
  10. P. E. Forssén and A. Moe, “View matching with blob features,” Image Vis. Comput. 27(1-2), 99–107 (2009).
    [CrossRef]
  11. Visual Geometry Group, “Affine covariant regions datasets,” http://www.robots.ox.ac.uk/~vgg/data/ .
  12. T. Liu, A. W. Moore, A. Gray, and K. Yang, “An investigation of practical approximate nearest neighbor algorithms,” in Advances in Neural Information Processing Systems, L. K. Saul, Y. Weiss and L. Bottou, eds. (MIT Press, Cambridge, 2005).
    [PubMed]
  13. T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002).
    [CrossRef]
  14. V. Lepetit, and P. Fua, “Towards recognizing feature points using classification trees,” in Technical Report IC/2004/74. (EPFL, 2004).
  15. E. Rosten, and T. Drummond, “Fusing points and lines for high performance tracking,” in Proceedings of the IEEE International Conference on Computer Vision (Beijing, China, 2005), pp. 1508–1515.
  16. E. Rosten, and T. Drummond, “Machine learning for high-speed corner detection,” in Proceedings of the European Conference on Computer Vision (Graz, Austria, 2006), pp. 430–443.
  17. Z. Li, W. Gong, A. Y. C. Nee, and S. K. Ong, “The effectiveness of detector combinations,” Opt. Express 17(9), 7407–7418 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-17-9-7407 .
    [CrossRef] [PubMed]
  18. M. L. Yuan, S. K. Ong, and A. Y. C. Nee, “Registration using natural features for augmented reality systems,” IEEE Trans. Vis. Comput. Graph. 12(4), 569–580 (2006).
    [CrossRef] [PubMed]

2009 (2)

2008 (1)

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
[CrossRef]

2006 (2)

V. Lepetit and P. Fua, “Keypoint recognition using randomized trees,” IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006).
[CrossRef] [PubMed]

M. L. Yuan, S. K. Ong, and A. Y. C. Nee, “Registration using natural features for augmented reality systems,” IEEE Trans. Vis. Comput. Graph. 12(4), 569–580 (2006).
[CrossRef] [PubMed]

2005 (2)

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

K. Mikolajczyk and C. Schmid, “Performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005).
[CrossRef] [PubMed]

2004 (2)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
[CrossRef]

2002 (1)

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002).
[CrossRef]

Bay, H.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
[CrossRef]

Chuma, O.

J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
[CrossRef]

Ess, A.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
[CrossRef]

Forssén, P. E.

P. E. Forssén and A. Moe, “View matching with blob features,” Image Vis. Comput. 27(1-2), 99–107 (2009).
[CrossRef]

Fua, P.

V. Lepetit and P. Fua, “Keypoint recognition using randomized trees,” IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006).
[CrossRef] [PubMed]

Gong, W.

Gool, L. V.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

Kadir, T.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

Lepetit, V.

V. Lepetit and P. Fua, “Keypoint recognition using randomized trees,” IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006).
[CrossRef] [PubMed]

Li, Z.

Lowe, D. G.

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Maenpaa, T.

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002).
[CrossRef]

Matas, J.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
[CrossRef]

Mikolajczyk, K.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

K. Mikolajczyk and C. Schmid, “Performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005).
[CrossRef] [PubMed]

Moe, A.

P. E. Forssén and A. Moe, “View matching with blob features,” Image Vis. Comput. 27(1-2), 99–107 (2009).
[CrossRef]

Nee, A. Y. C.

Z. Li, W. Gong, A. Y. C. Nee, and S. K. Ong, “The effectiveness of detector combinations,” Opt. Express 17(9), 7407–7418 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-17-9-7407 .
[CrossRef] [PubMed]

M. L. Yuan, S. K. Ong, and A. Y. C. Nee, “Registration using natural features for augmented reality systems,” IEEE Trans. Vis. Comput. Graph. 12(4), 569–580 (2006).
[CrossRef] [PubMed]

Ojala, T.

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002).
[CrossRef]

Ong, S. K.

Z. Li, W. Gong, A. Y. C. Nee, and S. K. Ong, “The effectiveness of detector combinations,” Opt. Express 17(9), 7407–7418 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-17-9-7407 .
[CrossRef] [PubMed]

M. L. Yuan, S. K. Ong, and A. Y. C. Nee, “Registration using natural features for augmented reality systems,” IEEE Trans. Vis. Comput. Graph. 12(4), 569–580 (2006).
[CrossRef] [PubMed]

Pajdla, T.

J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
[CrossRef]

Pietikainen, M.

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002).
[CrossRef]

Schaffalitzky, F.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

Schmid, C.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

K. Mikolajczyk and C. Schmid, “Performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005).
[CrossRef] [PubMed]

Tuytelaars, T.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
[CrossRef]

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

Urbana, M.

J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
[CrossRef]

Van Gool, L.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
[CrossRef]

Yuan, M. L.

M. L. Yuan, S. K. Ong, and A. Y. C. Nee, “Registration using natural features for augmented reality systems,” IEEE Trans. Vis. Comput. Graph. 12(4), 569–580 (2006).
[CrossRef] [PubMed]

Zisserman, A.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

Comput. Vis. Image Underst. (1)

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110(3), 346–359 (2008).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (3)

V. Lepetit and P. Fua, “Keypoint recognition using randomized trees,” IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006).
[CrossRef] [PubMed]

K. Mikolajczyk and C. Schmid, “Performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005).
[CrossRef] [PubMed]

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002).
[CrossRef]

IEEE Trans. Vis. Comput. Graph. (1)

M. L. Yuan, S. K. Ong, and A. Y. C. Nee, “Registration using natural features for augmented reality systems,” IEEE Trans. Vis. Comput. Graph. 12(4), 569–580 (2006).
[CrossRef] [PubMed]

Image Vis. Comput. (2)

P. E. Forssén and A. Moe, “View matching with blob features,” Image Vis. Comput. 27(1-2), 99–107 (2009).
[CrossRef]

J. Matas, O. Chuma, M. Urbana, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image Vis. Comput. 22(10), 761–767 (2004).
[CrossRef]

Int. J. Comput. Vis. (2)

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “Comparison of affine region detectors,” Int. J. Comput. Vis. 65(1-2), 43–72 (2005).
[CrossRef]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[CrossRef]

Opt. Express (1)

Other (8)

Visual Geometry Group, “Affine covariant regions datasets,” http://www.robots.ox.ac.uk/~vgg/data/ .

T. Liu, A. W. Moore, A. Gray, and K. Yang, “An investigation of practical approximate nearest neighbor algorithms,” in Advances in Neural Information Processing Systems, L. K. Saul, Y. Weiss and L. Bottou, eds. (MIT Press, Cambridge, 2005).
[PubMed]

V. Lepetit, and P. Fua, “Towards recognizing feature points using classification trees,” in Technical Report IC/2004/74. (EPFL, 2004).

E. Rosten, and T. Drummond, “Fusing points and lines for high performance tracking,” in Proceedings of the IEEE International Conference on Computer Vision (Beijing, China, 2005), pp. 1508–1515.

E. Rosten, and T. Drummond, “Machine learning for high-speed corner detection,” in Proceedings of the European Conference on Computer Vision (Graz, Austria, 2006), pp. 430–443.

M. Özuysal, P. Fua, and V. Lepetit, “Fast keypoint recognition in ten lines of code,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Minneapolis, United States, 2007), pp. 1–8.

P. E. Forssén, “Maximally stable colour regions for recognition and matching,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Minneapolis, United States, 2007), pp. 1220–1227.

C. Boncelet, “Image noise models,” in Handbook of image and video processing (second edition), A. C. Bovic, ed. (Elsevier Academic Press, San Diego, United States, 2005).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1
Fig. 1

Reference image (a) and the detected feature regions for exact keypoint detection (b). The reference image is from the ‘Graffiti’ sequence provided by the Visual Geometry Group [11]. The cardinality of each feature region is restricted from 900 to 36000.

Fig. 2
Fig. 2

Keypoint location.

Fig. 3
Fig. 3

Performance comparisons on the ‘Graffiti’ sequence.

Fig. 4
Fig. 4

Application of markerless AR tracking. (a) shows the reference frame, and (b) shows the detected keypoints and the initial location of the virtual object. (c) and (d), (e) and (f), and (g) and (h), show the object tracking under viewpoint change, rotation and scale variations respectively.

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

P ( I / μ ) = G ( μ , a μ ) ,
P ( μ ( P ) > μ ( Q ) ) = 1 Φ ( μ ( P ) μ ( Q ) μ ( P ) + μ ( Q ) ) = Φ ( μ ( P ) μ ( Q ) μ ( P ) + μ ( Q ) ) ,
d = | μ ( P ) μ ( Q ) | μ ( P ) + μ ( Q ) = | I ( P ) I ( Q ) | I ( P ) + I ( Q ) .
c ( x ) = P ( d < x ) = 2 π 0 x / λ e y 2 d y .
d thr ( t ) = c 1 ( t / T )        t [ 0 , T ] ,
r t = C ( t + Δ ) C ( t ) d thr ( t + Δ ) d thr ( t ) ,
R ( M , C ) = { X : ( X M ) T C 1 ( X M ) 4 } ,
M = 1 μ 00 ( μ 01 μ 10 ) ,      C = 1 μ 00 ( μ 02 μ 11 μ 11 μ 20 ) M M T .
D i = 0.5 × I R i c w + I R i + 0.5 × I R i c c w        i [ 1 , 16 ] .
D i = 0.5 × G R i c w P + G R i P + 0.5 × G R i c c w P        i [ 1 , 16 ] .

Metrics