Abstract

The ultra-field infrared and visible image registration is a challenging task due to its nonlinear imaging and multi-modal image features. In this paper, a robust registration method is proposed for the ultra-field infrared and visible images. First, control points are extracted utilizing phase congruency and optimized based on the guidance map, which is proposed according to significant structures information. Second, ROI pair matching is accomplished based on epipolar curve. Its effect is equivalent to a search window that is popular in methods with the standard field of view, and it can overcome the content differences in the search window caused by nonlinear imaging and vision disparity. Third, a descriptor, named multiple phase congruency directional pattern (MPCDP), is established and composed of distribution information and main direction. The phase congruency amplitudes are encoded as binary patterns, and then they are represented as weighted histogram for distribution information. Six pairs of ultra-field infrared and visible images are employed for registration experiments, and the results demonstrate that the performance of the proposed is robust and accurate in five types of ultra-field scenes and two different camera relationships.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. W. M. Zhang, J. Zhao, M. Chen, Y. M. Chen, K. Yan, L. Y. Li, J. B. Qi, X. Y. Wang, J. H. Luo, and Q. Chu, “Registration of optical imagery and Li DAR data using an inherent geometrical constraint,” Opt. Express 23(6), 7694–7702 (2015).
    [Crossref]
  2. Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
    [Crossref]
  3. S. G. Li, “Binocular Spherical Stereo,” IEEE Trans. Intell. Transport. Syst. 9(4), 589–600 (2008).
    [Crossref]
  4. X. H. Wang, K. H. Wu, and S. Z. Wang, “Research on Panoramic Image Registration Approach based on Spherical Model,” IJSIP 6(6), 297–308 (2013).
    [Crossref]
  5. Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
    [Crossref]
  6. H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
    [Crossref]
  7. Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
    [Crossref]
  8. Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
    [Crossref]
  9. J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
    [Crossref]
  10. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
    [Crossref]
  11. S. K. S. Fan, Y. C. Chuang, and J. R. Wu, “A New Cross-Correlation Based Image Registration Method,” Appl. Mech. Mater. 58-60(60), 1979–1984 (2011).
    [Crossref]
  12. Y. Hel-Or, H. Hel-Or, and E. David, “Fast template matching in non-linear tone-mapped images,” in Proc. IEEE Int. Conf. Comput. Vis., (2011), pp. 1355–1362.
  13. D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
    [Crossref]
  14. M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features,” In Proceedings of the European Conference on Computer Vision, (ECCV), (2010), pp. 1–4.
  15. E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” In Proceedings of the European Conference on Computer Vision (ECCV), (2006), pp. 1–2.
  16. C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
    [Crossref]
  17. Z. L. Song, S. Li, and Thomas F. George, “Remote sensing image registration approach based on a retrofitted SIFT algorithm and Lissajous-curve trajectories,” Opt. Express 18(2), 513–522 (2010).
    [Crossref]
  18. F. Lv and K. Ren, “Automatic registration of airborne Li DAR point cloud data and optical imagery depth map based on line and points features,” Infrared Phys. Technol. 71, 457–463 (2015).
    [Crossref]
  19. T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Machine Intell. 24(7), 971–987 (2002).
    [Crossref]
  20. E. Shechtman and M. Irani, “Matching local self-similarities across images and videos,” in Proc. IEEE Comput. Vis. Pattern Recognit. (2007), pp. 1–8.
  21. M. I. Patel, V. K. Thakar, and S. K. Shah, “Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF,” Procedia Comput. Sci. 93, 382–388 (2016).
    [Crossref]
  22. S. W. Teng, M. T. Hossain, and G. J. Lu, “Multimodal image registration technique based on improved local feature descriptors,” J. Electron. Imaging 24(1), 013013 (2015).
    [Crossref]
  23. C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
    [Crossref]
  24. P. Kovesi, “Image features from phase congruency,” Videre: Journal of Computer Vision Research 1(3), 1–26 (1999).
  25. X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
    [Crossref]
  26. A. Wong and D. A. Clausi, “ARRSI: Automatic Registration of Remote-Sensing Images,” IEEE Trans. Geosci. Electron. 45(5), 1483–1493 (2007).
    [Crossref]
  27. S. G. Li, “Real-Time Spherical Stereo,” in 18th International Conference on Pattern Recognition (ICPR’06), (2006), pp. 1–4.
  28. J. Moreau, S. Ambellouis, and Y. Ruichek, “3D reconstruction of urban environments based on fisheye stereovision,” in 8th International Conference on Signal Image Technology and Internet Based System, (2012), pp. 36–41.
  29. J. Bouguet, “Camera calibration toolbox for matlab,” (2014). http://www.vision.caltech.edu/bouguetj/calib_doc/index.html , (visited January 2020).
  30. F. Ahmed, “Gradient directional pattern: a robust feature descriptor for facial expression recognition,” Electron. Lett. 48(19), 1203 (2012).
    [Crossref]
  31. L. Yu, D. R. Zhang, and E. J. Holden, “A fast and fully automatic registration approach based on point features for multi-source remote-sensing images,” Comput. Geosci. 34(7), 838–848 (2008).
    [Crossref]
  32. C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37(2), 151–172 (2000).
    [Crossref]
  33. A. R. Rivera, J. R. Castillo, and O. Chae, “Local directional texture pattern image descriptor,” Pattern Recognit. Lett. 51, 94–100 (2015).
    [Crossref]

2018 (4)

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
[Crossref]

X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
[Crossref]

2017 (2)

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
[Crossref]

2016 (3)

M. I. Patel, V. K. Thakar, and S. K. Shah, “Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF,” Procedia Comput. Sci. 93, 382–388 (2016).
[Crossref]

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

2015 (4)

S. W. Teng, M. T. Hossain, and G. J. Lu, “Multimodal image registration technique based on improved local feature descriptors,” J. Electron. Imaging 24(1), 013013 (2015).
[Crossref]

W. M. Zhang, J. Zhao, M. Chen, Y. M. Chen, K. Yan, L. Y. Li, J. B. Qi, X. Y. Wang, J. H. Luo, and Q. Chu, “Registration of optical imagery and Li DAR data using an inherent geometrical constraint,” Opt. Express 23(6), 7694–7702 (2015).
[Crossref]

A. R. Rivera, J. R. Castillo, and O. Chae, “Local directional texture pattern image descriptor,” Pattern Recognit. Lett. 51, 94–100 (2015).
[Crossref]

F. Lv and K. Ren, “Automatic registration of airborne Li DAR point cloud data and optical imagery depth map based on line and points features,” Infrared Phys. Technol. 71, 457–463 (2015).
[Crossref]

2013 (1)

X. H. Wang, K. H. Wu, and S. Z. Wang, “Research on Panoramic Image Registration Approach based on Spherical Model,” IJSIP 6(6), 297–308 (2013).
[Crossref]

2012 (1)

F. Ahmed, “Gradient directional pattern: a robust feature descriptor for facial expression recognition,” Electron. Lett. 48(19), 1203 (2012).
[Crossref]

2011 (1)

S. K. S. Fan, Y. C. Chuang, and J. R. Wu, “A New Cross-Correlation Based Image Registration Method,” Appl. Mech. Mater. 58-60(60), 1979–1984 (2011).
[Crossref]

2010 (1)

2008 (2)

L. Yu, D. R. Zhang, and E. J. Holden, “A fast and fully automatic registration approach based on point features for multi-source remote-sensing images,” Comput. Geosci. 34(7), 838–848 (2008).
[Crossref]

S. G. Li, “Binocular Spherical Stereo,” IEEE Trans. Intell. Transport. Syst. 9(4), 589–600 (2008).
[Crossref]

2007 (1)

A. Wong and D. A. Clausi, “ARRSI: Automatic Registration of Remote-Sensing Images,” IEEE Trans. Geosci. Electron. 45(5), 1483–1493 (2007).
[Crossref]

2005 (1)

Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
[Crossref]

2004 (1)

D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

2002 (1)

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Machine Intell. 24(7), 971–987 (2002).
[Crossref]

2000 (1)

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37(2), 151–172 (2000).
[Crossref]

1999 (1)

P. Kovesi, “Image features from phase congruency,” Videre: Journal of Computer Vision Research 1(3), 1–26 (1999).

1997 (1)

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

Ahmed, F.

F. Ahmed, “Gradient directional pattern: a robust feature descriptor for facial expression recognition,” Electron. Lett. 48(19), 1203 (2012).
[Crossref]

Ai, Y. F.

X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
[Crossref]

Ambellouis, S.

J. Moreau, S. Ambellouis, and Y. Ruichek, “3D reconstruction of urban environments based on fisheye stereovision,” in 8th International Conference on Signal Image Technology and Internet Based System, (2012), pp. 36–41.

Bauckhage, C.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37(2), 151–172 (2000).
[Crossref]

Bentoutou, Y.

Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
[Crossref]

Bi, G. L.

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Bouguet, J.

J. Bouguet, “Camera calibration toolbox for matlab,” (2014). http://www.vision.caltech.edu/bouguetj/calib_doc/index.html , (visited January 2020).

Bruzzone, L.

Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
[Crossref]

Calonder, M.

M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features,” In Proceedings of the European Conference on Computer Vision, (ECCV), (2010), pp. 1–4.

Cao, Y. C.

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

Castillo, J. R.

A. R. Rivera, J. R. Castillo, and O. Chae, “Local directional texture pattern image descriptor,” Pattern Recognit. Lett. 51, 94–100 (2015).
[Crossref]

Chae, O.

A. R. Rivera, J. R. Castillo, and O. Chae, “Local directional texture pattern image descriptor,” Pattern Recognit. Lett. 51, 94–100 (2015).
[Crossref]

Chen, F. C.

Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
[Crossref]

Chen, M.

Chen, X. T.

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Chen, Y. M.

Chu, Q.

Chuang, Y. C.

S. K. S. Fan, Y. C. Chuang, and J. R. Wu, “A New Cross-Correlation Based Image Registration Method,” Appl. Mech. Mater. 58-60(60), 1979–1984 (2011).
[Crossref]

Clausi, D. A.

A. Wong and D. A. Clausi, “ARRSI: Automatic Registration of Remote-Sensing Images,” IEEE Trans. Geosci. Electron. 45(5), 1483–1493 (2007).
[Crossref]

Collignon, A.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

David, E.

Y. Hel-Or, H. Hel-Or, and E. David, “Fast template matching in non-linear tone-mapped images,” in Proc. IEEE Int. Conf. Comput. Vis., (2011), pp. 1355–1362.

Drummond, T.

E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” In Proceedings of the European Conference on Computer Vision (ECCV), (2006), pp. 1–2.

Fan, J. W.

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

Fan, S. K. S.

S. K. S. Fan, Y. C. Chuang, and J. R. Wu, “A New Cross-Correlation Based Image Registration Method,” Appl. Mech. Mater. 58-60(60), 1979–1984 (2011).
[Crossref]

Fua, P.

M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features,” In Proceedings of the European Conference on Computer Vision, (ECCV), (2010), pp. 1–4.

George, Thomas F.

He, B.

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Hel-Or, H.

Y. Hel-Or, H. Hel-Or, and E. David, “Fast template matching in non-linear tone-mapped images,” in Proc. IEEE Int. Conf. Comput. Vis., (2011), pp. 1355–1362.

Hel-Or, Y.

Y. Hel-Or, H. Hel-Or, and E. David, “Fast template matching in non-linear tone-mapped images,” in Proc. IEEE Int. Conf. Comput. Vis., (2011), pp. 1355–1362.

Holden, E. J.

L. Yu, D. R. Zhang, and E. J. Holden, “A fast and fully automatic registration approach based on point features for multi-source remote-sensing images,” Comput. Geosci. 34(7), 838–848 (2008).
[Crossref]

Hossain, M. T.

S. W. Teng, M. T. Hossain, and G. J. Lu, “Multimodal image registration technique based on improved local feature descriptors,” J. Electron. Imaging 24(1), 013013 (2015).
[Crossref]

Irani, M.

E. Shechtman and M. Irani, “Matching local self-similarities across images and videos,” in Proc. IEEE Comput. Vis. Pattern Recognit. (2007), pp. 1–8.

Kovesi, P.

P. Kovesi, “Image features from phase congruency,” Videre: Journal of Computer Vision Research 1(3), 1–26 (1999).

Kpalma, K.

Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
[Crossref]

Lepetit, V.

M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features,” In Proceedings of the European Conference on Computer Vision, (ECCV), (2010), pp. 1–4.

Li, B.

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

Li, K. J.

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Li, L. Y.

Li, M.

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

Li, S.

Li, S. G.

S. G. Li, “Binocular Spherical Stereo,” IEEE Trans. Intell. Transport. Syst. 9(4), 589–600 (2008).
[Crossref]

S. G. Li, “Real-Time Spherical Stereo,” in 18th International Conference on Pattern Recognition (ICPR’06), (2006), pp. 1–4.

Liang, H. D.

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Liang, W. K.

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

Liu, C. L.

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Liu, J. H.

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

Liu, X. Z.

X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
[Crossref]

Lowe, D.

D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

Lu, G. J.

S. W. Teng, M. T. Hossain, and G. J. Lu, “Multimodal image registration technique based on improved local feature descriptors,” J. Electron. Imaging 24(1), 013013 (2015).
[Crossref]

Luo, J. H.

Lv, F.

F. Lv and K. Ren, “Automatic registration of airborne Li DAR point cloud data and optical imagery depth map based on line and points features,” Infrared Phys. Technol. 71, 457–463 (2015).
[Crossref]

Lv, J. F.

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

Maenpaa, T.

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Machine Intell. 24(7), 971–987 (2002).
[Crossref]

Maes, F.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

Marchal, G.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

Meng, H. Y.

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Mohr, R.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37(2), 151–172 (2000).
[Crossref]

Moreau, J.

J. Moreau, S. Ambellouis, and Y. Ruichek, “3D reconstruction of urban environments based on fisheye stereovision,” in 8th International Conference on Signal Image Technology and Internet Based System, (2012), pp. 36–41.

Nie, T.

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Ojala, T.

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Machine Intell. 24(7), 971–987 (2002).
[Crossref]

Patel, M. I.

M. I. Patel, V. K. Thakar, and S. K. Shah, “Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF,” Procedia Comput. Sci. 93, 382–388 (2016).
[Crossref]

Pietikainen, M.

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Machine Intell. 24(7), 971–987 (2002).
[Crossref]

Qi, J. B.

Ren, K.

F. Lv and K. Ren, “Automatic registration of airborne Li DAR point cloud data and optical imagery depth map based on line and points features,” Infrared Phys. Technol. 71, 457–463 (2015).
[Crossref]

Rivera, A. R.

A. R. Rivera, J. R. Castillo, and O. Chae, “Local directional texture pattern image descriptor,” Pattern Recognit. Lett. 51, 94–100 (2015).
[Crossref]

Ronsin, J.

Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
[Crossref]

Rosten, E.

E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” In Proceedings of the European Conference on Computer Vision (ECCV), (2006), pp. 1–2.

Ruichek, Y.

J. Moreau, S. Ambellouis, and Y. Ruichek, “3D reconstruction of urban environments based on fisheye stereovision,” in 8th International Conference on Signal Image Technology and Internet Based System, (2012), pp. 36–41.

Schmid, C.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37(2), 151–172 (2000).
[Crossref]

Shah, S. K.

M. I. Patel, V. K. Thakar, and S. K. Shah, “Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF,” Procedia Comput. Sci. 93, 382–388 (2016).
[Crossref]

Shan, J.

Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
[Crossref]

Shechtman, E.

E. Shechtman and M. Irani, “Matching local self-similarities across images and videos,” in Proc. IEEE Comput. Vis. Pattern Recognit. (2007), pp. 1–8.

Shen, L.

Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
[Crossref]

Song, Z. L.

Strecha, C.

M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features,” In Proceedings of the European Conference on Computer Vision, (ECCV), (2010), pp. 1–4.

Su, C.

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

Suetens, P.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

Sun, M. C.

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

Sun, S. J.

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

Sun, S. L.

Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
[Crossref]

Taleb, N.

Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
[Crossref]

Tang, C. Q.

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Teng, S. W.

S. W. Teng, M. T. Hossain, and G. J. Lu, “Multimodal image registration technique based on improved local feature descriptors,” J. Electron. Imaging 24(1), 013013 (2015).
[Crossref]

Thakar, V. K.

M. I. Patel, V. K. Thakar, and S. K. Shah, “Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF,” Procedia Comput. Sci. 93, 382–388 (2016).
[Crossref]

Tian, G. Y.

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Vandermeulen, D.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

Wang, S. Z.

X. H. Wang, K. H. Wu, and S. Z. Wang, “Research on Panoramic Image Registration Approach based on Spherical Model,” IJSIP 6(6), 297–308 (2013).
[Crossref]

Wang, X.

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

Wang, X. H.

X. H. Wang, K. H. Wu, and S. Z. Wang, “Research on Panoramic Image Registration Approach based on Spherical Model,” IJSIP 6(6), 297–308 (2013).
[Crossref]

Wang, X. Y.

Wang, Z. P.

X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
[Crossref]

Wong, A.

A. Wong and D. A. Clausi, “ARRSI: Automatic Registration of Remote-Sensing Images,” IEEE Trans. Geosci. Electron. 45(5), 1483–1493 (2007).
[Crossref]

Wu, D. M.

Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
[Crossref]

Wu, J. B.

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Wu, J. R.

S. K. S. Fan, Y. C. Chuang, and J. R. Wu, “A New Cross-Correlation Based Image Registration Method,” Appl. Mech. Mater. 58-60(60), 1979–1984 (2011).
[Crossref]

Wu, K. H.

X. H. Wang, K. H. Wu, and S. Z. Wang, “Research on Panoramic Image Registration Approach based on Spherical Model,” IJSIP 6(6), 297–308 (2013).
[Crossref]

Wu, Y.

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

Yan, K.

Yang, M. Y.

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

Ye, Y. X.

Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
[Crossref]

Yu, L.

L. Yu, D. R. Zhang, and E. J. Holden, “A fast and fully automatic registration approach based on point features for multi-source remote-sensing images,” Comput. Geosci. 34(7), 838–848 (2008).
[Crossref]

Yu, Q. H.

Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
[Crossref]

Zhang, D. R.

L. Yu, D. R. Zhang, and E. J. Holden, “A fast and fully automatic registration approach based on point features for multi-source remote-sensing images,” Comput. Geosci. 34(7), 838–848 (2008).
[Crossref]

Zhang, J. L.

X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
[Crossref]

Zhang, W. M.

Zhao, C. Y.

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

Zhao, H. C.

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

Zhao, J.

Zuo, Y. J.

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

Appl. Mech. Mater. (1)

S. K. S. Fan, Y. C. Chuang, and J. R. Wu, “A New Cross-Correlation Based Image Registration Method,” Appl. Mech. Mater. 58-60(60), 1979–1984 (2011).
[Crossref]

Chin. Opt. Lett. (1)

Q. H. Yu, D. M. Wu, F. C. Chen, and S. L. Sun, “Design of a wide-field target detection and tracking system using the segmented planar imaging detector for electro-optical reconnaissance,” Chin. Opt. Lett. 16(7), 071101-1 (2018).
[Crossref]

Comput. Geosci. (1)

L. Yu, D. R. Zhang, and E. J. Holden, “A fast and fully automatic registration approach based on point features for multi-source remote-sensing images,” Comput. Geosci. 34(7), 838–848 (2008).
[Crossref]

Electron. Lett. (1)

F. Ahmed, “Gradient directional pattern: a robust feature descriptor for facial expression recognition,” Electron. Lett. 48(19), 1203 (2012).
[Crossref]

IEEE Trans. Geosci. Electron. (4)

A. Wong and D. A. Clausi, “ARRSI: Automatic Registration of Remote-Sensing Images,” IEEE Trans. Geosci. Electron. 45(5), 1483–1493 (2007).
[Crossref]

Y. X. Ye, J. Shan, L. Bruzzone, and L. Shen, “Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity,” IEEE Trans. Geosci. Electron. 55(5), 2941–2958 (2017).
[Crossref]

Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An Automatic Image Registration for Applications in Remote Sensing,” IEEE Trans. Geosci. Electron. 43(9), 2127–2137 (2005).
[Crossref]

J. W. Fan, Y. Wu, M. Li, W. K. Liang, and Y. C. Cao, “SAR and Optical Image Registration Using Nonlinear Diffusion and Phase Congruency Structural Descriptor,” IEEE Trans. Geosci. Electron. 56(9), 5368–5379 (2018).
[Crossref]

IEEE Trans. Intell. Transport. Syst. (1)

S. G. Li, “Binocular Spherical Stereo,” IEEE Trans. Intell. Transport. Syst. 9(4), 589–600 (2008).
[Crossref]

IEEE Trans. Med. Imaging (1)

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16(2), 187–198 (1997).
[Crossref]

IEEE Trans. Pattern Anal. Machine Intell. (1)

T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal. Machine Intell. 24(7), 971–987 (2002).
[Crossref]

IJSIP (1)

X. H. Wang, K. H. Wu, and S. Z. Wang, “Research on Panoramic Image Registration Approach based on Spherical Model,” IJSIP 6(6), 297–308 (2013).
[Crossref]

Infrared Phys. Technol. (3)

H. D. Liang, C. L. Liu, B. He, T. Nie, G. L. Bi, and C. Su, “A Binary Method of Multisensor Image Registration Based on Angle Traversal,” Infrared Phys. Technol. 95, 189–198 (2018).
[Crossref]

F. Lv and K. Ren, “Automatic registration of airborne Li DAR point cloud data and optical imagery depth map based on line and points features,” Infrared Phys. Technol. 71, 457–463 (2015).
[Crossref]

C. Q. Tang, G. Y. Tian, X. T. Chen, J. B. Wu, K. J. Li, and H. Y. Meng, “Infrared and Visible Images Registration withAdaptable Local-Global Feature Integration for RailInspection,” Infrared Phys. Technol. 87, 31–39 (2017).
[Crossref]

Int. J. Comput. Vis. (2)

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37(2), 151–172 (2000).
[Crossref]

D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004).
[Crossref]

J. Electron. Imaging (1)

S. W. Teng, M. T. Hossain, and G. J. Lu, “Multimodal image registration technique based on improved local feature descriptors,” J. Electron. Imaging 24(1), 013013 (2015).
[Crossref]

Neurocomputing (1)

C. Y. Zhao, H. C. Zhao, J. F. Lv, S. J. Sun, and B. Li, “Multimodal image matching based on Multimodality Robust Line Segment Descriptor,” Neurocomputing 177(6), 290–303 (2016).
[Crossref]

Opt. Eng. (1)

Y. J. Zuo, J. H. Liu, M. Y. Yang, X. Wang, and M. C. Sun, “Algorithm for unmanned aerial vehicle aerial different-source image matching,” Opt. Eng. 55(12), 123111 (2016).
[Crossref]

Opt. Express (2)

Pattern Recognit. Lett. (1)

A. R. Rivera, J. R. Castillo, and O. Chae, “Local directional texture pattern image descriptor,” Pattern Recognit. Lett. 51, 94–100 (2015).
[Crossref]

Procedia Comput. Sci. (1)

M. I. Patel, V. K. Thakar, and S. K. Shah, “Image Registration of Satellite Images with Varying Illumination Level Using HOG Descriptor Based SURF,” Procedia Comput. Sci. 93, 382–388 (2016).
[Crossref]

Remote Sens. (1)

X. Z. Liu, Y. F. Ai, J. L. Zhang, and Z. P. Wang, “A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration,” Remote Sens. 10(4), 658 (2018).
[Crossref]

Videre: Journal of Computer Vision Research (1)

P. Kovesi, “Image features from phase congruency,” Videre: Journal of Computer Vision Research 1(3), 1–26 (1999).

Other (7)

S. G. Li, “Real-Time Spherical Stereo,” in 18th International Conference on Pattern Recognition (ICPR’06), (2006), pp. 1–4.

J. Moreau, S. Ambellouis, and Y. Ruichek, “3D reconstruction of urban environments based on fisheye stereovision,” in 8th International Conference on Signal Image Technology and Internet Based System, (2012), pp. 36–41.

J. Bouguet, “Camera calibration toolbox for matlab,” (2014). http://www.vision.caltech.edu/bouguetj/calib_doc/index.html , (visited January 2020).

E. Shechtman and M. Irani, “Matching local self-similarities across images and videos,” in Proc. IEEE Comput. Vis. Pattern Recognit. (2007), pp. 1–8.

M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary Robust Independent Elementary Features,” In Proceedings of the European Conference on Computer Vision, (ECCV), (2010), pp. 1–4.

E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” In Proceedings of the European Conference on Computer Vision (ECCV), (2006), pp. 1–2.

Y. Hel-Or, H. Hel-Or, and E. David, “Fast template matching in non-linear tone-mapped images,” in Proc. IEEE Int. Conf. Comput. Vis., (2011), pp. 1355–1362.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1.
Fig. 1. Comparison of the phase congruency and gradient in infrared and visible images
Fig. 2.
Fig. 2. Principle of the proposed CP extraction
Fig. 3.
Fig. 3. Ultra-field infrared and visible binocular model
Fig. 4.
Fig. 4. Ultra-field infrared and visible binocular calibration results
Fig. 5.
Fig. 5. Epipolar curve based on calibration results
Fig. 6.
Fig. 6. The quantizing process of phase congruency
Fig. 7.
Fig. 7. Flow of the proposed registration method
Fig. 8.
Fig. 8. Average repeatability of all the detection methods
Fig. 9.
Fig. 9. CP extraction results for rotation change
Fig. 10.
Fig. 10. Robust ultra-field infrared and visible CP extraction results
Fig. 11.
Fig. 11. Reprojection errors comparison in ultra-field infrared and visible cameras
Fig. 12.
Fig. 12. Average matching errors of image pairs No.1-6
Fig. 13.
Fig. 13. Standard deviations of matching errors of image pairs No.1-6
Fig. 14.
Fig. 14. Five descriptors CMR of image pairs No.1-6
Fig. 15.
Fig. 15. Final registration results of six image pairs

Tables (1)

Tables Icon

Table 1. Calibration results of infrared and visible ultra-field cameras

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

P ( x , y ) = n W ( x , y ) A n ( x , y ) Δ Φ ( x , y ) T n A n ( x , y ) + ε
Δ Φ ( x , y ) = cos ( ϕ n ( x , y ) ϕ ¯ ( x , y ) ) | sin ( ϕ n ( x , y ) ϕ ¯ ( x , y ) ) |
I ( x , y ) t = d i v ( c ( x , y , t ) I )
c ( x , y , t ) = g ( | I ( x , y , t ) | )
g = { 1 | I | 2 0 1 exp ( 3.315 ( | I | / k ) 3 ) | I | 2 > 0
p l  =  { ρ l = x 2 + y 2 ω l = arctan y l x l , P l  =  { R φ l = p r o j l 1 ( ρ l , ω l ) θ l , P l r a y  =  λ l { R sin φ l sin θ l R sin φ l cos θ l R cos φ l
P r r a y  =  R r o t a t i o n [ λ l { R sin φ l sin θ l R sin φ l cos θ l R cos φ l ] T + T t r a n s l a t i o n , p e p i p o l a r = p r o j r ( P r r a y )
H M P C D P ( i ) = x = 1 M y = 1 N f ( M P C D P ( x , y ) , i ) ,   f ( a , b ) = { 1 a = b 0 o t h e r w i s e
M P C D P = { ω 1 P C D P 1 , ω 2 P C D P 2 , ω n P C D P n t h d i r e c t i o n }
C M R = N u m c o r r e c t N u m a l l