Abstract

Line matching in widely separated views is challenging because of large perspective distortion and violation of the planarity assumption in local regions. We introduce a novel method of wide-baseline image matching based on the coplanar line intersections for poorly textured and/or nonplanar structured scenes. The local areas of the coplanar line pairs are normalized into canonical frames by rectifying the coplanar line pairs to be orthogonal. Then, the 3D interpretation of the intersection context of the coplanar line pairs helps to match the nonplanar local regions. Furthermore, for calibrated stereo cameras, we propose a matching criterion based on 3D planar homography to improve the matching accuracy while reconstructing most likely physically existing planar patches. Experimental results demonstrate the effectiveness of the proposed method for real-world scenes.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
    [CrossRef]
  2. C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.
  3. B. Micusík and J. Kosecka, “Piecewise planar city 3D modeling from street view panoramic sequences,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (2009), pp. 2906–2912.
  4. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
    [CrossRef]
  5. J. Ponce, M. Hebert, C. Schmid, and A. Zisserman, eds., Toward Category-Level Object Recognition, Lecture Notes in Computer Science (Springer, 2006), Vol. 4170.
  6. F. Zhou, H. B. L. Duh, and M. Billinghurst, “Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR,” in 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (2008), pp. 193–202.
  7. J. Matas, O. Chum, U. Martin, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proceedings of British Machine Vision Conference, London, UK (2002), pp. 384–393.
  8. T. Tuytelaars and L. Van Gool, “Matching widely separated views based on affine invariant regions,” Int. J. Comput. Vis. 59, 61–85 (2004).
    [CrossRef]
  9. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
    [CrossRef]
  10. A. Vedaldi and S. Soatto, “Features for recognition: viewpoint invariance for non-planar scenes,” in Proceedings of the International Conference on Computer Vision (ICCV) (2005), pp. 1474–1481.
  11. E. Kim, G. Medioni, and S. Lee, “Planar patch based 3D environment modeling with stereo camera,” in 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju Island, Korea (2008), pp. 516–521.
  12. C. Schmid and A. Zisserman, “Automatic line matching across views,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1997), pp. 666–671.
  13. T. Werner and A. Zisserman, “New techniques for automated architectural reconstruction from photographs,” in Proceedings of the 7th European Conference on Computer Vision. Part II, London, UK (Springer-Verlag, 2002), pp. 541–555.
  14. H. Bay, V. Ferrari, and L. Van Gool, “Wide-baseline stereo matching with line segments,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), pp. 329–336.
  15. V. Ferrari, T. Tuytelaars, and L. V. Gool, “Wide-baseline muliple-view correspondences,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2003), pp. 718–728.
  16. L. Wang, U. Neumann, and S. You, “Wide-baseline image matching using line signatures,” in Proceedings of the International Conference on Computer Vision (ICCV), Kyoto (2009).
  17. Z. Wang, F. Wu, and Z. Hu, “MSLD: a robust descriptor for line matching,” Pattern Recogn. 42, 941–953 (2009).
    [CrossRef]
  18. E. Vincent and R. Laganière, “Junction matching and fundamental matrix recovery in widely separated views,” in Proceedings of the British Machine Vision Conference, London, UK (2004), pp. 77–86.
  19. H. Bay, A. Ess, A. Neubeck, and L. V. Gool, “3D from line segments in two poorly-textured, uncalibrated images,” in Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Chapel Hill, North Carolina (2006).
  20. B. Micusík, H. Wildenauer, and J. Kosecka, “Detection and matching of rectilinear structures,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (2008).
  21. H. Kim and S. Lee, “A novel line matching method based on intersection context,” in IEEE International Conference on Robotics and Automation (ICRA), Anchorage, Alaska (2010).
  22. H. Kim and S. Lee, “Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs,” Pattern Recogn. Lett. 33, 1349–1363 (2012).
    [CrossRef]
  23. T. Werner, “Lmatch: Matlab toolbox for matching line segments accross multiple calibrated images,” http://cmp.felk.cvut.cz/~werner/software/lmatch/ (2007).
  24. P. Moreels and P. Perona, “Evaluation of features detectors and descriptors based on 3D objects,” Int. J. Comput. Vis. 73, 263–284 (2007).
    [CrossRef]
  25. F. Zhao, Q. Huang, and W. Gao, “Image matching by multiscale oriented corner correlation,” in Asian Conference on Computer Vision (2006), pp. 928–937.
  26. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).
  27. R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2004).
  28. M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
    [CrossRef]
  29. A. Vedaldi and B. Fulkerson, “VLFeat: an open and portable library of computer vision algorithms,” http://www.vlfeat.org/ (2008).
  30. H. Shao, T. Svoboda, and L. V. Gool, “Zubud-zurich buildings database for image based recognition,” (Swiss Federal Institute of Technology, 2003).

2012 (1)

H. Kim and S. Lee, “Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs,” Pattern Recogn. Lett. 33, 1349–1363 (2012).
[CrossRef]

2009 (1)

Z. Wang, F. Wu, and Z. Hu, “MSLD: a robust descriptor for line matching,” Pattern Recogn. 42, 941–953 (2009).
[CrossRef]

2008 (1)

R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
[CrossRef]

2007 (1)

P. Moreels and P. Perona, “Evaluation of features detectors and descriptors based on 3D objects,” Int. J. Comput. Vis. 73, 263–284 (2007).
[CrossRef]

2005 (1)

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

2004 (2)

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
[CrossRef]

T. Tuytelaars and L. Van Gool, “Matching widely separated views based on affine invariant regions,” Int. J. Comput. Vis. 59, 61–85 (2004).
[CrossRef]

2003 (1)

M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
[CrossRef]

Argyros, A. A.

M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
[CrossRef]

Baillard, C.

C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.

Bay, H.

H. Bay, V. Ferrari, and L. Van Gool, “Wide-baseline stereo matching with line segments,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), pp. 329–336.

H. Bay, A. Ess, A. Neubeck, and L. V. Gool, “3D from line segments in two poorly-textured, uncalibrated images,” in Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Chapel Hill, North Carolina (2006).

Billinghurst, M.

F. Zhou, H. B. L. Duh, and M. Billinghurst, “Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR,” in 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (2008), pp. 193–202.

Chum, O.

J. Matas, O. Chum, U. Martin, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proceedings of British Machine Vision Conference, London, UK (2002), pp. 384–393.

Datta, R.

R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
[CrossRef]

Duh, H. B. L.

F. Zhou, H. B. L. Duh, and M. Billinghurst, “Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR,” in 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (2008), pp. 193–202.

England, O. O.

C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.

Ess, A.

H. Bay, A. Ess, A. Neubeck, and L. V. Gool, “3D from line segments in two poorly-textured, uncalibrated images,” in Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Chapel Hill, North Carolina (2006).

Ferrari, V.

H. Bay, V. Ferrari, and L. Van Gool, “Wide-baseline stereo matching with line segments,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), pp. 329–336.

V. Ferrari, T. Tuytelaars, and L. V. Gool, “Wide-baseline muliple-view correspondences,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2003), pp. 718–728.

Fitzgibbon, A.

C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.

Gao, W.

F. Zhao, Q. Huang, and W. Gao, “Image matching by multiscale oriented corner correlation,” in Asian Conference on Computer Vision (2006), pp. 928–937.

Gonzalez, R. C.

R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).

Gool, L. V.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

H. Bay, A. Ess, A. Neubeck, and L. V. Gool, “3D from line segments in two poorly-textured, uncalibrated images,” in Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Chapel Hill, North Carolina (2006).

V. Ferrari, T. Tuytelaars, and L. V. Gool, “Wide-baseline muliple-view correspondences,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2003), pp. 718–728.

H. Shao, T. Svoboda, and L. V. Gool, “Zubud-zurich buildings database for image based recognition,” (Swiss Federal Institute of Technology, 2003).

Hartley, R. I.

R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2004).

Hu, Z.

Z. Wang, F. Wu, and Z. Hu, “MSLD: a robust descriptor for line matching,” Pattern Recogn. 42, 941–953 (2009).
[CrossRef]

Huang, Q.

F. Zhao, Q. Huang, and W. Gao, “Image matching by multiscale oriented corner correlation,” in Asian Conference on Computer Vision (2006), pp. 928–937.

Joshi, D.

R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
[CrossRef]

Kadir, T.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

Kim, E.

E. Kim, G. Medioni, and S. Lee, “Planar patch based 3D environment modeling with stereo camera,” in 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju Island, Korea (2008), pp. 516–521.

Kim, H.

H. Kim and S. Lee, “Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs,” Pattern Recogn. Lett. 33, 1349–1363 (2012).
[CrossRef]

H. Kim and S. Lee, “A novel line matching method based on intersection context,” in IEEE International Conference on Robotics and Automation (ICRA), Anchorage, Alaska (2010).

Kosecka, J.

B. Micusík, H. Wildenauer, and J. Kosecka, “Detection and matching of rectilinear structures,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (2008).

B. Micusík and J. Kosecka, “Piecewise planar city 3D modeling from street view panoramic sequences,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (2009), pp. 2906–2912.

Laganière, R.

E. Vincent and R. Laganière, “Junction matching and fundamental matrix recovery in widely separated views,” in Proceedings of the British Machine Vision Conference, London, UK (2004), pp. 77–86.

Lee, S.

H. Kim and S. Lee, “Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs,” Pattern Recogn. Lett. 33, 1349–1363 (2012).
[CrossRef]

H. Kim and S. Lee, “A novel line matching method based on intersection context,” in IEEE International Conference on Robotics and Automation (ICRA), Anchorage, Alaska (2010).

E. Kim, G. Medioni, and S. Lee, “Planar patch based 3D environment modeling with stereo camera,” in 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju Island, Korea (2008), pp. 516–521.

Li, J.

R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
[CrossRef]

Lourakis, M. I. A.

M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
[CrossRef]

Lowe, D. G.

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
[CrossRef]

Martin, U.

J. Matas, O. Chum, U. Martin, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proceedings of British Machine Vision Conference, London, UK (2002), pp. 384–393.

Matas, J.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

J. Matas, O. Chum, U. Martin, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proceedings of British Machine Vision Conference, London, UK (2002), pp. 384–393.

Medioni, G.

E. Kim, G. Medioni, and S. Lee, “Planar patch based 3D environment modeling with stereo camera,” in 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju Island, Korea (2008), pp. 516–521.

Micusík, B.

B. Micusík and J. Kosecka, “Piecewise planar city 3D modeling from street view panoramic sequences,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (2009), pp. 2906–2912.

B. Micusík, H. Wildenauer, and J. Kosecka, “Detection and matching of rectilinear structures,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (2008).

Mikolajczyk, K.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

Moreels, P.

P. Moreels and P. Perona, “Evaluation of features detectors and descriptors based on 3D objects,” Int. J. Comput. Vis. 73, 263–284 (2007).
[CrossRef]

Neubeck, A.

H. Bay, A. Ess, A. Neubeck, and L. V. Gool, “3D from line segments in two poorly-textured, uncalibrated images,” in Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Chapel Hill, North Carolina (2006).

Neumann, U.

L. Wang, U. Neumann, and S. You, “Wide-baseline image matching using line signatures,” in Proceedings of the International Conference on Computer Vision (ICCV), Kyoto (2009).

Orphanoudakis, S. C.

M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
[CrossRef]

Pajdla, T.

J. Matas, O. Chum, U. Martin, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proceedings of British Machine Vision Conference, London, UK (2002), pp. 384–393.

Perona, P.

P. Moreels and P. Perona, “Evaluation of features detectors and descriptors based on 3D objects,” Int. J. Comput. Vis. 73, 263–284 (2007).
[CrossRef]

Schaffalitzky, F.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

Schmid, C.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

C. Schmid and A. Zisserman, “Automatic line matching across views,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1997), pp. 666–671.

C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.

Shao, H.

H. Shao, T. Svoboda, and L. V. Gool, “Zubud-zurich buildings database for image based recognition,” (Swiss Federal Institute of Technology, 2003).

Soatto, S.

A. Vedaldi and S. Soatto, “Features for recognition: viewpoint invariance for non-planar scenes,” in Proceedings of the International Conference on Computer Vision (ICCV) (2005), pp. 1474–1481.

Svoboda, T.

H. Shao, T. Svoboda, and L. V. Gool, “Zubud-zurich buildings database for image based recognition,” (Swiss Federal Institute of Technology, 2003).

Tuytelaars, T.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

T. Tuytelaars and L. Van Gool, “Matching widely separated views based on affine invariant regions,” Int. J. Comput. Vis. 59, 61–85 (2004).
[CrossRef]

V. Ferrari, T. Tuytelaars, and L. V. Gool, “Wide-baseline muliple-view correspondences,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2003), pp. 718–728.

Tzurbakis, S. V.

M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
[CrossRef]

Van Gool, L.

T. Tuytelaars and L. Van Gool, “Matching widely separated views based on affine invariant regions,” Int. J. Comput. Vis. 59, 61–85 (2004).
[CrossRef]

H. Bay, V. Ferrari, and L. Van Gool, “Wide-baseline stereo matching with line segments,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), pp. 329–336.

Vedaldi, A.

A. Vedaldi and S. Soatto, “Features for recognition: viewpoint invariance for non-planar scenes,” in Proceedings of the International Conference on Computer Vision (ICCV) (2005), pp. 1474–1481.

Vincent, E.

E. Vincent and R. Laganière, “Junction matching and fundamental matrix recovery in widely separated views,” in Proceedings of the British Machine Vision Conference, London, UK (2004), pp. 77–86.

Wang, J. Z.

R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
[CrossRef]

Wang, L.

L. Wang, U. Neumann, and S. You, “Wide-baseline image matching using line signatures,” in Proceedings of the International Conference on Computer Vision (ICCV), Kyoto (2009).

Wang, Z.

Z. Wang, F. Wu, and Z. Hu, “MSLD: a robust descriptor for line matching,” Pattern Recogn. 42, 941–953 (2009).
[CrossRef]

Werner, T.

T. Werner and A. Zisserman, “New techniques for automated architectural reconstruction from photographs,” in Proceedings of the 7th European Conference on Computer Vision. Part II, London, UK (Springer-Verlag, 2002), pp. 541–555.

Wildenauer, H.

B. Micusík, H. Wildenauer, and J. Kosecka, “Detection and matching of rectilinear structures,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (2008).

Woods, R. E.

R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).

Wu, F.

Z. Wang, F. Wu, and Z. Hu, “MSLD: a robust descriptor for line matching,” Pattern Recogn. 42, 941–953 (2009).
[CrossRef]

You, S.

L. Wang, U. Neumann, and S. You, “Wide-baseline image matching using line signatures,” in Proceedings of the International Conference on Computer Vision (ICCV), Kyoto (2009).

Zhao, F.

F. Zhao, Q. Huang, and W. Gao, “Image matching by multiscale oriented corner correlation,” in Asian Conference on Computer Vision (2006), pp. 928–937.

Zhou, F.

F. Zhou, H. B. L. Duh, and M. Billinghurst, “Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR,” in 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (2008), pp. 193–202.

Zisserman, A.

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

C. Schmid and A. Zisserman, “Automatic line matching across views,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1997), pp. 666–671.

C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.

R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2004).

T. Werner and A. Zisserman, “New techniques for automated architectural reconstruction from photographs,” in Proceedings of the 7th European Conference on Computer Vision. Part II, London, UK (Springer-Verlag, 2002), pp. 541–555.

ACM Comput. Surv. (1)

R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Comput. Surv. 40, 5 (2008).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

M. I. A. Lourakis, S. V. Tzurbakis, A. A. Argyros, and S. C. Orphanoudakis, “Feature transfer and matching in disparate stereo views through the use of plane homographies,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 271–276 (2003).
[CrossRef]

Int. J. Comput. Vis. (4)

P. Moreels and P. Perona, “Evaluation of features detectors and descriptors based on 3D objects,” Int. J. Comput. Vis. 73, 263–284 (2007).
[CrossRef]

T. Tuytelaars and L. Van Gool, “Matching widely separated views based on affine invariant regions,” Int. J. Comput. Vis. 59, 61–85 (2004).
[CrossRef]

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool, “A comparison of affine region detectors,” Int. J. Comput. Vis. 65, 43–72 (2005).
[CrossRef]

D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
[CrossRef]

Pattern Recogn. (1)

Z. Wang, F. Wu, and Z. Hu, “MSLD: a robust descriptor for line matching,” Pattern Recogn. 42, 941–953 (2009).
[CrossRef]

Pattern Recogn. Lett. (1)

H. Kim and S. Lee, “Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs,” Pattern Recogn. Lett. 33, 1349–1363 (2012).
[CrossRef]

Other (22)

T. Werner, “Lmatch: Matlab toolbox for matching line segments accross multiple calibrated images,” http://cmp.felk.cvut.cz/~werner/software/lmatch/ (2007).

C. Baillard, C. Schmid, A. Zisserman, A. Fitzgibbon, and O. O. England, “Automatic line matching and 3D reconstruction of buildings from multiple views,” in ISPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery (1999), pp. 69–80.

B. Micusík and J. Kosecka, “Piecewise planar city 3D modeling from street view panoramic sequences,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (2009), pp. 2906–2912.

F. Zhao, Q. Huang, and W. Gao, “Image matching by multiscale oriented corner correlation,” in Asian Conference on Computer Vision (2006), pp. 928–937.

R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed. (Prentice-Hall, 2006).

R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. (Cambridge University, 2004).

A. Vedaldi and B. Fulkerson, “VLFeat: an open and portable library of computer vision algorithms,” http://www.vlfeat.org/ (2008).

H. Shao, T. Svoboda, and L. V. Gool, “Zubud-zurich buildings database for image based recognition,” (Swiss Federal Institute of Technology, 2003).

E. Vincent and R. Laganière, “Junction matching and fundamental matrix recovery in widely separated views,” in Proceedings of the British Machine Vision Conference, London, UK (2004), pp. 77–86.

H. Bay, A. Ess, A. Neubeck, and L. V. Gool, “3D from line segments in two poorly-textured, uncalibrated images,” in Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Chapel Hill, North Carolina (2006).

B. Micusík, H. Wildenauer, and J. Kosecka, “Detection and matching of rectilinear structures,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (2008).

H. Kim and S. Lee, “A novel line matching method based on intersection context,” in IEEE International Conference on Robotics and Automation (ICRA), Anchorage, Alaska (2010).

J. Ponce, M. Hebert, C. Schmid, and A. Zisserman, eds., Toward Category-Level Object Recognition, Lecture Notes in Computer Science (Springer, 2006), Vol. 4170.

F. Zhou, H. B. L. Duh, and M. Billinghurst, “Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR,” in 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (2008), pp. 193–202.

J. Matas, O. Chum, U. Martin, and T. Pajdla, “Robust wide baseline stereo from maximally stable extremal regions,” in Proceedings of British Machine Vision Conference, London, UK (2002), pp. 384–393.

A. Vedaldi and S. Soatto, “Features for recognition: viewpoint invariance for non-planar scenes,” in Proceedings of the International Conference on Computer Vision (ICCV) (2005), pp. 1474–1481.

E. Kim, G. Medioni, and S. Lee, “Planar patch based 3D environment modeling with stereo camera,” in 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju Island, Korea (2008), pp. 516–521.

C. Schmid and A. Zisserman, “Automatic line matching across views,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1997), pp. 666–671.

T. Werner and A. Zisserman, “New techniques for automated architectural reconstruction from photographs,” in Proceedings of the 7th European Conference on Computer Vision. Part II, London, UK (Springer-Verlag, 2002), pp. 541–555.

H. Bay, V. Ferrari, and L. Van Gool, “Wide-baseline stereo matching with line segments,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), pp. 329–336.

V. Ferrari, T. Tuytelaars, and L. V. Gool, “Wide-baseline muliple-view correspondences,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2003), pp. 718–728.

L. Wang, U. Neumann, and S. You, “Wide-baseline image matching using line signatures,” in Proceedings of the International Conference on Computer Vision (ICCV), Kyoto (2009).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (19)

Fig. 1.
Fig. 1.

Robust region descriptors of LICFs and the 3D interpretation. (a) Canonical frames, (b) rotation and scale variations of a canonical frame, and (c) 2D LICFs w.r.t. 3D nonplanar structures.

Fig. 2.
Fig. 2.

LICF examples and analysis of 3D nonplanar structures. (a) 3D classified feature positions of nonplanar structures, (b) 3D classified canonical frames, (c) matching score (NCC), (d) 3D class estimation, (e) rotation (class) estimation, and (f) scale estimation.

Fig. 3.
Fig. 3.

Relationship among LICFs and 3D planar patches. (a) Candidate plane reconstruction, (b) LICF line-pair reconstruction, and (c) planar patch projection into retina planes.

Fig. 4.
Fig. 4.

Four configurations of end-point matching.

Fig. 5.
Fig. 5.

Two examples of line end-point matching.

Fig. 6.
Fig. 6.

Three different LICF matching candidates. The red boxes are the backprojected planar patches by the reconstructed 3D planar patches from matching LICF candidates.

Fig. 7.
Fig. 7.

Test image sequences for widely separated views. The reference views are boxed in red. The sequence “Zubud” is taken from scene #157 in the Zubud data set. (a) The sequence “Zubud.” Views 1–5, in order. The reference view is View 1. (b) The sequence “Kampa.” Views 1–7, in order. The reference view is View 1. (c) The sequence ”INRIA.” Views 1–7, in order. The reference view is View 3.

Fig. 8.
Fig. 8.

Matching results of the sequence “Zubud.” In the top and bottom rows, the reference view (view 1) and view 5 are presented, respectively. In the first and second columns, matching results of LICFs and SIFTs are shown, respectively. The matching points are coded in the same color with matching index number. The red lines correspond to epipolar lines. In the last column, we select a point on the wall of the right building from the reference view and transfer the point into the other view using epipolar geometry using the estimated fundamental matrix from matching LICFs (in red line) and SIFTs (in blue line). The red epipolar line estimated by the matching LICFs meets the points on the wall of the right building, but the blue line fails to match the points.

Fig. 9.
Fig. 9.

Matching results of the scene “Kampa.” (a) Matching lines, matching LICFs, and matching SIFT features, in order, between the reference view and view 4. (b) Between the reference view and view 6. The matching points are coded in the same color with matching index number. The red lines correspond to epipolar lines.

Fig. 10.
Fig. 10.

Quantitative results. (Left) The sequence “ZuBud.” (Middle) The sequence “Kampa.” (Right) The sequence “INRIA.” (a) The number of matching features. (b) The symmetric transfer error of the estimated fundamental matrix.

Fig. 11.
Fig. 11.

Matching results of the scene “INRIA.” Matching lines, matching LICFs, and matching SIFT features, in order, between the reference view and view 7.

Fig. 12.
Fig. 12.

Test image sequences of poorly textured scenes. (a) The sequence “Dining_Table.” Views 1–7 in order. The reference view is View 1. (b) The sequence “Kitchen.” Views 1–6 in order. The reference view is View 1. (c) The sequence “Tea_Table.” Views 1–8 in order. The reference view is View 1.

Fig. 13.
Fig. 13.

Quantitative results. (Left) The sequence “Dining_Table.” (Middle) The sequence “Kitchen.” (Right) The sequence “Tea_Table.” (a) The number of matching features. (b) The symmetric transfer error of the estimated fundamental matrix.

Fig. 14.
Fig. 14.

Matching results of the scene “Dining_Table” between the reference view and view 4. Matching lines, matching LICFs, and matching SIFTs, in order, between the reference view and View 4.

Fig. 15.
Fig. 15.

Matching results of the scene “Kitchen.” Matching lines, matching LICFs, and matching SIFT features, in order, between the reference view and view 6.

Fig. 16.
Fig. 16.

Matching results of the scene “Tea_Table.” Matching lines, matching LICFs, and matching SIFT features, in order, between the reference view and view 8.

Fig. 17.
Fig. 17.

Matching and 3D reconstruction results for poorly textured scenes, “table” and “sink.” The original image pair, matching lines with backprojection, matching LICFs, and matching SIFTs with the corresponding epipolar constraints, in that order, are presented in the first two rows. In the bottom row, the 3D line reconstruction results are presented. (a) Matching and 3D reconstruction results of the scene “Table.” (b) Matching and 3D reconstruction results of the scene “Sink.”

Fig. 18.
Fig. 18.

Comparison study results of line matching between the original method and the proposed method. The result of the original method and that of the proposed method are given alternatively for the scenes, “lab,” “table,” and “sink.”

Fig. 19.
Fig. 19.

Comparison of matching criteria. In the first and second rows, the left (reference) image and the right (target) image are presented, with overlaid line segments, respectively. (a) Stereo image. (b) LICF (reference image) and its matching candidates (target image) located on the epipolar line. (c) Falsely matched line pairs by the canonical-frame-based method. (d),(e) Matching candidates before and after applying 3D planar homography criterion. (f) Correctly matched line pairs based on 3D planar homography criterion.

Tables (2)

Tables Icon

Table 1. Comparison of the Proposed Method with the SIFT Method and the Simple NCC Version

Tables Icon

Table 2. Matching Rates

Equations (14)

Equations on this page are rendered with MathJax. Learn more.

Lpair,k={lπ(k)1,lπ(k)2},(π(k)1,π(k)2)Π,
F{xk,P(xk,σ),Lpair,k}={xk,P(xk),lπ(k)1,lπ(k)2},
CkC(xk,σ)=P(Hkyk),ykRxk,σ,
lπ(k)1cHkTlπ(k)1,
lπ(k)2cHkTlπ(k)2,
(HkTlπ(k)1)·(HkTlπ(k)2)=0
|HkTlπ(k)1||(HkTlπ(k)1)·(lπ(k)1)|=|HkTlπ(k)2||(HkTlπ(k)2)·(lπ(k)2)|.
SksP(SsHkxk).
RSks,rPk(RrSsHkxk),
Tks,r,wWwRSks,rWwPk(RrSsHkxk),
NCC(Fi,Fj)=1Nncc1Nncc(P(xi)P(x¯i))·(P(xj)P(x¯j))σP(xi)·σP(xj),
Mx,x={xm,xm;Lpair,m,Lpair,m},
Ex,x=1Mi=1Md(xi,Fxi)2+d(xi,FTxi)2,
Ml,l={lψ(m),lψ(m)},ψ(m)Π,ψ(m)Π,

Metrics