Abstract

A new image registration method for multimodal images is proposed in this paper. This method is a combination of the modified scale invariant feature transform (SIFT) feature extraction algorithm and the shape-context feature descriptor. Salient points of multimodal images are extracted by using the modified SIFT feature extraction algorithm. Then each salient point is described by using the shape-context descriptor that formed a feature vector from the orientation histograms of the subregion around each salient point. After salient points matching by using Euclidean distance, random sample consensus algorithm is used to eliminate wrong corresponding pairs. At last, multimodal images registration is achieved by affine transformation and bilinear interpolation. Experimental results for registration of IR images and electro-optical images show that this method has a good registration result.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. Zitová and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21, 977–1000 (2003).
    [CrossRef]
  2. A. Tian, Z. Zhao, and Q. Gao, “Electrical equipment IR and visible images registration method based on SIFT,” Electric Power Sci. Eng. 24, 13–15 (2008).
  3. L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).
  4. R. Istenic, D. Heric, S. Ribaric, D. Zazula, “Thermal and visual image registration in Hough parameter space,” presented at the 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, Maribor, Slovenia, 26–30 June 2007.
  5. P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).
  6. Y. Chen and Y-p. Tian, “The research of image registration in image fusion,” Laser Infrared 34, 34–36 (2004).
  7. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
    [CrossRef] [PubMed]
  8. C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37, 151–172 (2000).
    [CrossRef]
  9. D. G. Lowe, “Object recognition from local scale-invariant features,” presented at the International Conference on Computer Vision, Corfu, Greece, 20 September 1999.
  10. D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
    [CrossRef]
  11. N. Seo and D. A. Schug, “Image matching using scale invariant feature transform (SIFT),” http://note.sonots.com/SciSoftware/SIFT.html.
  12. S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Machine Intell. 24, 509–522 (2002).
    [CrossRef]
  13. C. Harris and M. Stephens, “A combined corner and edge detector,” presented at the Fourth Alvey Vision Conference, Manchester, UK, 31 August 1988.
  14. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395(1981).
    [CrossRef]

2008

A. Tian, Z. Zhao, and Q. Gao, “Electrical equipment IR and visible images registration method based on SIFT,” Electric Power Sci. Eng. 24, 13–15 (2008).

2005

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

2004

Y. Chen and Y-p. Tian, “The research of image registration in image fusion,” Laser Infrared 34, 34–36 (2004).

D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
[CrossRef]

2003

B. Zitová and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21, 977–1000 (2003).
[CrossRef]

L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).

2002

S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Machine Intell. 24, 509–522 (2002).
[CrossRef]

2000

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37, 151–172 (2000).
[CrossRef]

1997

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

1981

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395(1981).
[CrossRef]

Bauckhage, C.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37, 151–172 (2000).
[CrossRef]

Belongie, S.

S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Machine Intell. 24, 509–522 (2002).
[CrossRef]

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395(1981).
[CrossRef]

Chen, Y.

Y. Chen and Y-p. Tian, “The research of image registration in image fusion,” Laser Infrared 34, 34–36 (2004).

Collignon, A.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

Cui, S-f.

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

Ding, M.

L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395(1981).
[CrossRef]

Flusser, J.

B. Zitová and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21, 977–1000 (2003).
[CrossRef]

Gao, Q.

A. Tian, Z. Zhao, and Q. Gao, “Electrical equipment IR and visible images registration method based on SIFT,” Electric Power Sci. Eng. 24, 13–15 (2008).

Ge, Y-x.

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

Harris, C.

C. Harris and M. Stephens, “A combined corner and edge detector,” presented at the Fourth Alvey Vision Conference, Manchester, UK, 31 August 1988.

Heric, D.

R. Istenic, D. Heric, S. Ribaric, D. Zazula, “Thermal and visual image registration in Hough parameter space,” presented at the 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, Maribor, Slovenia, 26–30 June 2007.

Istenic, R.

R. Istenic, D. Heric, S. Ribaric, D. Zazula, “Thermal and visual image registration in Hough parameter space,” presented at the 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, Maribor, Slovenia, 26–30 June 2007.

Lowe, D.

D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
[CrossRef]

Lowe, D. G.

D. G. Lowe, “Object recognition from local scale-invariant features,” presented at the International Conference on Computer Vision, Corfu, Greece, 20 September 1999.

Maes, F.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

Malik, J.

S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Machine Intell. 24, 509–522 (2002).
[CrossRef]

Marchal, G.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

Mohr, R.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37, 151–172 (2000).
[CrossRef]

Ping, X.

L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).

Puzicha, J.

S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Machine Intell. 24, 509–522 (2002).
[CrossRef]

Ribaric, S.

R. Istenic, D. Heric, S. Ribaric, D. Zazula, “Thermal and visual image registration in Hough parameter space,” presented at the 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, Maribor, Slovenia, 26–30 June 2007.

Schmid, C.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37, 151–172 (2000).
[CrossRef]

Schug, D. A.

N. Seo and D. A. Schug, “Image matching using scale invariant feature transform (SIFT),” http://note.sonots.com/SciSoftware/SIFT.html.

Seo, N.

N. Seo and D. A. Schug, “Image matching using scale invariant feature transform (SIFT),” http://note.sonots.com/SciSoftware/SIFT.html.

Shu, L.

L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).

Stephens, M.

C. Harris and M. Stephens, “A combined corner and edge detector,” presented at the Fourth Alvey Vision Conference, Manchester, UK, 31 August 1988.

Suetens, P.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

Tian, A.

A. Tian, Z. Zhao, and Q. Gao, “Electrical equipment IR and visible images registration method based on SIFT,” Electric Power Sci. Eng. 24, 13–15 (2008).

Tian, Y-p.

Y. Chen and Y-p. Tian, “The research of image registration in image fusion,” Laser Infrared 34, 34–36 (2004).

Vandermeulen, D.

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

Wang, Y-w.

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

Yu, P.

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

Zazula, D.

R. Istenic, D. Heric, S. Ribaric, D. Zazula, “Thermal and visual image registration in Hough parameter space,” presented at the 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, Maribor, Slovenia, 26–30 June 2007.

Zhang, K.

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

Zhao, Z.

A. Tian, Z. Zhao, and Q. Gao, “Electrical equipment IR and visible images registration method based on SIFT,” Electric Power Sci. Eng. 24, 13–15 (2008).

Zhou, C.

L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).

Zitová, B.

B. Zitová and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21, 977–1000 (2003).
[CrossRef]

Commun. ACM

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395(1981).
[CrossRef]

Electric Power Sci. Eng.

A. Tian, Z. Zhao, and Q. Gao, “Electrical equipment IR and visible images registration method based on SIFT,” Electric Power Sci. Eng. 24, 13–15 (2008).

IEEE Trans. Med. Imaging

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imaging 16, 187–198 (1997).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Machine Intell.

S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” IEEE Trans. Pattern Anal. Machine Intell. 24, 509–522 (2002).
[CrossRef]

Image Vis. Comput.

B. Zitová and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. 21, 977–1000 (2003).
[CrossRef]

Int. J. Comput. Vis.

C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of interest point detectors,” Int. J. Comput. Vis. 37, 151–172 (2000).
[CrossRef]

D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60, 91–110 (2004).
[CrossRef]

J. Huazhong Univ. Sci. Tech.

L. Shu, X. Ping, C. Zhou, and M. Ding, “Method of automatic IR/visual image registration,” J. Huazhong Univ. Sci. Tech. 31, 92–94 (2003).

Laser Infrared

P. Yu, S-f. Cui, Y-x. Ge, Y-w. Wang, and K. Zhang, “Image registration based on alignment metric,” Laser Infrared 38, 737–740 (2005).

Y. Chen and Y-p. Tian, “The research of image registration in image fusion,” Laser Infrared 34, 34–36 (2004).

Other

R. Istenic, D. Heric, S. Ribaric, D. Zazula, “Thermal and visual image registration in Hough parameter space,” presented at the 6th EURASIP Conference focused on Speech and Image Processing, Multimedia Communications and Services, Maribor, Slovenia, 26–30 June 2007.

N. Seo and D. A. Schug, “Image matching using scale invariant feature transform (SIFT),” http://note.sonots.com/SciSoftware/SIFT.html.

D. G. Lowe, “Object recognition from local scale-invariant features,” presented at the International Conference on Computer Vision, Corfu, Greece, 20 September 1999.

C. Harris and M. Stephens, “A combined corner and edge detector,” presented at the Fourth Alvey Vision Conference, Manchester, UK, 31 August 1988.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1

Diagram of the proposed algorithm.

Fig. 2
Fig. 2

Log-polar location grids of the shape-context descriptor.

Fig. 3
Fig. 3

Image pairs used for the experiment. (a) EO image. (b) IR image.

Fig. 4
Fig. 4

Feature points matching results of the proposed algorithm. (a) Original matching result. (b) Result of eliminating the wrong corresponding points.

Fig. 5
Fig. 5

Registration results of different methods. (a) Registration result of the proposed algotithm. (b) Registration result of the manually selected points. (c) Registration result of the SIFT algorithm.

Fig. 6
Fig. 6

Feature points matching results of the SIFT algorithm. (a) Original matching result. (b) Result of eliminating the wrong corresponding points.

Fig. 7
Fig. 7

IR image and EO image of the same scene. (a) Original IR image. (b) Reversal IR image. (c) EO image.

Fig. 8
Fig. 8

Feature points matching results of the proposed algorithm and SIFT algorithm. (a) Original matching result of the proposed algorithm. (b) Result of eliminating the wrong corresponding points of the proposed algorithm. (c) Registration result of the proposed algorithm. (d) Original matching result of the SIFT algorithm.

Tables (1)

Tables Icon

Table 1 Parameters of Affine Transformation of Different Algorithms

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

D ( x , y , k σ ) = ( G ( x , y , k σ ) G ( x , y , σ ) ) ( x , y ) = L ( x , y , k σ ) L ( x , y , σ ) ,
G ( x , y , k σ ) = 1 / ( 2 π σ 2 ) × exp ( ( x 2 + y 2 ) / 2 σ 2 ) .
H = [ D x x D x y D x y D y y ] ,
Tr ( H ) = D x x + D y y = λ 1 + λ 2 ,
Det ( H ) = D x x D y y ( D x y ) 2 = λ 1 · λ 2.
Tr ( H ) 2 Det ( H ) = ( λ 1 + λ 2 ) 2 λ 1 · λ 2 = ( r + 1 ) 2 r .
w ( d , r , σ ) = exp ( ( d / r ) 2 / σ 2 ) .
[ X 2 Y 2 ] = [ A 11 A 12 A 21 A 22 ] [ X 1 Y 1 ] + [ T x T y ] .

Metrics