Abstract

Medical image fusion has been used to derive useful information from multimodality medical image data. In this research, we propose a novel method for multimodality medical image fusion. Using wavelet transform, we achieved a fusion scheme. A fusion rule is proposed and used for calculating the wavelet transformation modulus maxima of input images at different bandwidths and levels. To evaluate the fusion result, a metric based on mutual information (MI) is presented for measuring fusion effect. The performances of other two methods of image fusion based on wavelet transform are briefly described for comparison. The experiment results demonstrate the effectiveness of the fusion scheme.

© 2001 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M.A. Hurn and K.V. Mardia, “Bayesian fused classification of medical images,” IEEE Trans. Med. Imag. 15, 850–858 (1996)
    [Crossref]
  2. J. Chanutsot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for linear features detection in mltitemporal SAR imaged,” IEEE Trans. Geoscience and Remote Sensing 37, 1350–1359 (1999)
  3. L. Bruzzone, F. Prieto, and S. Serpico, “A neural statistical approac to multitemporal and multisource remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing 37, 1292–1305 (1999)
    [Crossref]
  4. P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Trans. on Communications COM-31, 532–540(1983)
    [Crossref]
  5. Tang Zhi Wei, Wang Jian Guo, and Huang Shun Ji, “The wavelet transformation application for image fusion,” in Wavelet Application VII, H. H. Szu, ed., Proc. SPIE4056, 462–469 (2000)
  6. H. Li, B.S. Manjunath, and S.K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing 57, 235–245 (1995)
    [Crossref]
  7. George P. Lemeshewsky, “Multispectral multisensor image fusion using wavelet transforms,” in Visual Information Processing VIII, S.D. Park, ed., Proc. SPIE3716, 214–222 (1999)
  8. L. Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24, 325–376 (1992)
    [Crossref]
  9. P. Elsen, E. Pol, and M. Viergever, “Medical image matching-A review with classification,” IEEE Eng, Med. Biol.26–39 (1993)
    [Crossref]
  10. D. MarrVision ( W.H. Freeman and Co., San Fransisco, 1982)
  11. S. Zhong and S. Mallat, “Characterization of signals from multiscale edges,” IEEE Trans. PAMI. 14. 710–732 (1992)
    [Crossref]
  12. S. Mallat, A wavelet tour of singnal processing (Academic Press,1998)
  13. W. B. Pennebaker and J. L. Mitchell, JPEG - still image data compression standards, Van Nostrand Reinhold, 1993.
  14. W. D. Withers, “A rapid entropy coding algorithm”, (Technical report, Pegasus Imaging Corporation) ftp://www.pegasusimaging.com/pub/ELSCODER.PDF.
  15. R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. on Information Theory 38, 713–718(1992)
    [Crossref]

1999 (2)

J. Chanutsot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for linear features detection in mltitemporal SAR imaged,” IEEE Trans. Geoscience and Remote Sensing 37, 1350–1359 (1999)

L. Bruzzone, F. Prieto, and S. Serpico, “A neural statistical approac to multitemporal and multisource remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing 37, 1292–1305 (1999)
[Crossref]

1996 (1)

M.A. Hurn and K.V. Mardia, “Bayesian fused classification of medical images,” IEEE Trans. Med. Imag. 15, 850–858 (1996)
[Crossref]

1995 (1)

H. Li, B.S. Manjunath, and S.K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing 57, 235–245 (1995)
[Crossref]

1992 (3)

L. Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24, 325–376 (1992)
[Crossref]

S. Zhong and S. Mallat, “Characterization of signals from multiscale edges,” IEEE Trans. PAMI. 14. 710–732 (1992)
[Crossref]

R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. on Information Theory 38, 713–718(1992)
[Crossref]

1983 (1)

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Trans. on Communications COM-31, 532–540(1983)
[Crossref]

Adelson, E. H.

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Trans. on Communications COM-31, 532–540(1983)
[Crossref]

Brown, L.

L. Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24, 325–376 (1992)
[Crossref]

Bruzzone, L.

L. Bruzzone, F. Prieto, and S. Serpico, “A neural statistical approac to multitemporal and multisource remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing 37, 1292–1305 (1999)
[Crossref]

Burt, P. J.

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Trans. on Communications COM-31, 532–540(1983)
[Crossref]

Chanutsot, J.

J. Chanutsot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for linear features detection in mltitemporal SAR imaged,” IEEE Trans. Geoscience and Remote Sensing 37, 1350–1359 (1999)

Coifman, R. R.

R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. on Information Theory 38, 713–718(1992)
[Crossref]

Elsen, P.

P. Elsen, E. Pol, and M. Viergever, “Medical image matching-A review with classification,” IEEE Eng, Med. Biol.26–39 (1993)
[Crossref]

Freeman, W.H.

D. MarrVision ( W.H. Freeman and Co., San Fransisco, 1982)

Guo, Wang Jian

Tang Zhi Wei, Wang Jian Guo, and Huang Shun Ji, “The wavelet transformation application for image fusion,” in Wavelet Application VII, H. H. Szu, ed., Proc. SPIE4056, 462–469 (2000)

Hurn, M.A.

M.A. Hurn and K.V. Mardia, “Bayesian fused classification of medical images,” IEEE Trans. Med. Imag. 15, 850–858 (1996)
[Crossref]

Ji, Huang Shun

Tang Zhi Wei, Wang Jian Guo, and Huang Shun Ji, “The wavelet transformation application for image fusion,” in Wavelet Application VII, H. H. Szu, ed., Proc. SPIE4056, 462–469 (2000)

Lambert, P.

J. Chanutsot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for linear features detection in mltitemporal SAR imaged,” IEEE Trans. Geoscience and Remote Sensing 37, 1350–1359 (1999)

Lemeshewsky, George P.

George P. Lemeshewsky, “Multispectral multisensor image fusion using wavelet transforms,” in Visual Information Processing VIII, S.D. Park, ed., Proc. SPIE3716, 214–222 (1999)

Li, H.

H. Li, B.S. Manjunath, and S.K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing 57, 235–245 (1995)
[Crossref]

Mallat, S.

S. Zhong and S. Mallat, “Characterization of signals from multiscale edges,” IEEE Trans. PAMI. 14. 710–732 (1992)
[Crossref]

S. Mallat, A wavelet tour of singnal processing (Academic Press,1998)

Manjunath, B.S.

H. Li, B.S. Manjunath, and S.K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing 57, 235–245 (1995)
[Crossref]

Mardia, K.V.

M.A. Hurn and K.V. Mardia, “Bayesian fused classification of medical images,” IEEE Trans. Med. Imag. 15, 850–858 (1996)
[Crossref]

Marr, D.

D. MarrVision ( W.H. Freeman and Co., San Fransisco, 1982)

Mauris, G.

J. Chanutsot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for linear features detection in mltitemporal SAR imaged,” IEEE Trans. Geoscience and Remote Sensing 37, 1350–1359 (1999)

Mitchell, J. L.

W. B. Pennebaker and J. L. Mitchell, JPEG - still image data compression standards, Van Nostrand Reinhold, 1993.

Mitra, S.K.

H. Li, B.S. Manjunath, and S.K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing 57, 235–245 (1995)
[Crossref]

Pennebaker, W. B.

W. B. Pennebaker and J. L. Mitchell, JPEG - still image data compression standards, Van Nostrand Reinhold, 1993.

Pol, E.

P. Elsen, E. Pol, and M. Viergever, “Medical image matching-A review with classification,” IEEE Eng, Med. Biol.26–39 (1993)
[Crossref]

Prieto, F.

L. Bruzzone, F. Prieto, and S. Serpico, “A neural statistical approac to multitemporal and multisource remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing 37, 1292–1305 (1999)
[Crossref]

Serpico, S.

L. Bruzzone, F. Prieto, and S. Serpico, “A neural statistical approac to multitemporal and multisource remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing 37, 1292–1305 (1999)
[Crossref]

Viergever, M.

P. Elsen, E. Pol, and M. Viergever, “Medical image matching-A review with classification,” IEEE Eng, Med. Biol.26–39 (1993)
[Crossref]

Wei, Tang Zhi

Tang Zhi Wei, Wang Jian Guo, and Huang Shun Ji, “The wavelet transformation application for image fusion,” in Wavelet Application VII, H. H. Szu, ed., Proc. SPIE4056, 462–469 (2000)

Wickerhauser, M. V.

R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. on Information Theory 38, 713–718(1992)
[Crossref]

Withers, W. D.

W. D. Withers, “A rapid entropy coding algorithm”, (Technical report, Pegasus Imaging Corporation) ftp://www.pegasusimaging.com/pub/ELSCODER.PDF.

Zhong, S.

S. Zhong and S. Mallat, “Characterization of signals from multiscale edges,” IEEE Trans. PAMI. 14. 710–732 (1992)
[Crossref]

ACM Comput. Surv. (1)

L. Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24, 325–376 (1992)
[Crossref]

Graphical Models and Image Processing (1)

H. Li, B.S. Manjunath, and S.K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing 57, 235–245 (1995)
[Crossref]

IEEE Trans. Geoscience and Remote Sensing (2)

J. Chanutsot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for linear features detection in mltitemporal SAR imaged,” IEEE Trans. Geoscience and Remote Sensing 37, 1350–1359 (1999)

L. Bruzzone, F. Prieto, and S. Serpico, “A neural statistical approac to multitemporal and multisource remote sensing image classification,” IEEE Trans. Geoscience and Remote Sensing 37, 1292–1305 (1999)
[Crossref]

IEEE Trans. Med. Imag. (1)

M.A. Hurn and K.V. Mardia, “Bayesian fused classification of medical images,” IEEE Trans. Med. Imag. 15, 850–858 (1996)
[Crossref]

IEEE Trans. on Communications (1)

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Trans. on Communications COM-31, 532–540(1983)
[Crossref]

IEEE Trans. on Information Theory (1)

R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. on Information Theory 38, 713–718(1992)
[Crossref]

IEEE Trans. PAMI. (1)

S. Zhong and S. Mallat, “Characterization of signals from multiscale edges,” IEEE Trans. PAMI. 14. 710–732 (1992)
[Crossref]

Other (7)

S. Mallat, A wavelet tour of singnal processing (Academic Press,1998)

W. B. Pennebaker and J. L. Mitchell, JPEG - still image data compression standards, Van Nostrand Reinhold, 1993.

W. D. Withers, “A rapid entropy coding algorithm”, (Technical report, Pegasus Imaging Corporation) ftp://www.pegasusimaging.com/pub/ELSCODER.PDF.

Tang Zhi Wei, Wang Jian Guo, and Huang Shun Ji, “The wavelet transformation application for image fusion,” in Wavelet Application VII, H. H. Szu, ed., Proc. SPIE4056, 462–469 (2000)

George P. Lemeshewsky, “Multispectral multisensor image fusion using wavelet transforms,” in Visual Information Processing VIII, S.D. Park, ed., Proc. SPIE3716, 214–222 (1999)

P. Elsen, E. Pol, and M. Viergever, “Medical image matching-A review with classification,” IEEE Eng, Med. Biol.26–39 (1993)
[Crossref]

D. MarrVision ( W.H. Freeman and Co., San Fransisco, 1982)

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1.

Image fusion scheme in this study

Fig. 2.
Fig. 2.

The original matched images

Fig. 3.
Fig. 3.

The distribution of the wavelet transform modulus maxima sets of the input images

Fig.4
Fig.4

The new fused image

Fig. 5.
Fig. 5.

The marginal distributions of the original images

Fig.6
Fig.6

The joint probability distributions

Tables (2)

Tables Icon

Table 1. The fusion performance assessing results

Tables Icon

Table 2 fusion performance-assessing results of the two approaches

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

( W 1 f ( u , v , 2 j ) W 2 f ( u , v , 2 j ) ) = ( f * ψ ¯ 2 f 1 ( u , v ) f * ψ ¯ 2 f 2 ( u , v ) ) = 2 j ( f * θ ¯ 2 f ) ( u , v ) ,
M f ( u , v , 2 j ) = W 1 f ( u , v , 2 j ) + W 2 f ( u , v , 2 j ) ,
A f ( u , v , 2 j ) = { α W 1 ( u , v , 2 j ) 0 π α W 1 ( u , v , 2 j ) < 0 ,
α = { tan 1 ( W 2 f ( u , v , 2 j ) W 1 f ( u , v , 2 j ) ) , when W 1 f ( u , v , 2 j ) 0 ± π 2 , otherwise ,
M k f ( u j , p , v j , p , 2 j ) = < f , ψ j , p k > for 1 j 2 ,
M k f ˜ ( u j , p , v j , p , 2 j ) = < f ˜ , ψ j , p k > = < f , ψ j , p k > ,
g = L f ˜ = k = 1 2 j , p < f , ψ j , p k > ψ j , p k ,
C F k ( u , v ) = mean ( C A k ( u , v ) + C B k ( u , v ) )
D F k ( u , v ) = max { D A k ( u , v ) , D B k ( u , v ) }
I A B ( x , y ) = x , y p A B ( x , y ) log p A B ( x , y ) p A ( x ) p B ( y ) ,
M F A B = I F A ( f , a ) + I F B ( f , b ) ,
I F A ( f , a ) = f , a p F A ( f , a ) log p F A ( f , a ) p F ( f ) p A ( a ) ,
I FB ( f , b ) = f , b p FB ( f , b ) log p FB ( f , b ) p F ( f ) p B ( b ) ,

Metrics