Abstract

A new technique is developed for the merging and data fusion of two images. Two spatially registered images with differing spatial resolutions and color content are merged by combining multiresolution wavelet-decomposition components from each and then reconstructing the merged image by means of the inverse wavelet transform. The wavelet merger can employ a variety of wavelet bases, but in presentation of the concept, simple orthonormal sets—Haar and Daubechies wavelets—are explored. The wavelet technique is compared with the intensity–hue–saturation merging technique by means of multispectral and panchromatic test images. The results of the comparison show the wavelet merger performing better in combining and preserving spectral–spatial information for the test images.

© 1995 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. A. Rose, “The sensitivity performance of the human eye on an absolute scale,” J. Opt. Soc. Am. 38, 196–208 (1948).
    [CrossRef] [PubMed]
  2. D. L. Hall, Mathematical Techniques in Multisensor Data Fusion (Artech House, Boston, 1992).
  3. R. Haydn, G. W. Dalke, J. Henkel, J. E. Bare, “Application of the IHS color transform to the processing of multi-sensor data and image enhancement,” in Proceedings of the International Symposium on Remote Sensing of Arid and Semi-Arid Lands, Cairo, Egypt (Environmental Research Institute, Ann Arbor, Mich., 1982), pp. 599–616.
  4. W. J. Carper, T. M. Lillesand, R. W. Kiefer, “The use of intensity–hue–saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogram. Eng. Remote Sens. 56, 459–467 (1990).
  5. V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogram. Eng. Remote Sens. 58, 561–567 (1992).
  6. The viewpoint of the author is from a remote-sensing perspective of the terms multispectral and panchromatic. In signal processing, multispectral can be confused with Fourier information, also referred to as spectral information. In this paper multispectral and spectral deal only with the wavelength, electromagnetic spectrum sense of the word, which is also synonymous with the term multichannel.
  7. H. Wechsler, Computational Vision (Academic, San Diego, Calif., 1990).
  8. M. D. Levine, Vision in Man and Machine (McGraw-Hill, San Francisco, Calif., 1985).
  9. P. J. Burt, “Multiresolution techniques for image representation, analysis, and ‘smart’ transmission,” Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1199, 2–15 (1989).
    [CrossRef]
  10. S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 674–693 (1989).
    [CrossRef]
  11. I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math. 91, 909–996 (1988).
    [CrossRef]
  12. M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, “Image coding using vector quantization in the wavelet transform domain,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM (Institute of Electrical and Electronics Engineers, New York, 1990), pp. 2297–2300.
    [CrossRef]
  13. J. Froment, S. Mallat, “Second generation compact image coding with wavelets,” in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed. (Academic, San Diego, Calif., 1992), pp. 655–678.
  14. J. Rosiene, I. Greenshields, “Standard wavelet basis compression of images,” Opt. Eng. 33, 2572–2578 (1994).
    [CrossRef]
  15. A. Toet, L. J. van Ruyven, J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28, 789–792 (1989).
    [CrossRef]
  16. O. Rioul, M. Vetterli, “Wavelets and signal processing,” IEEE Signal Process. Mag. 8(4), 14–38 (1991).
    [CrossRef]
  17. S. Mallat, S. Zhong, “Characterization of signals from multiscale edges,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 710–732 (1992).
    [CrossRef]
  18. P. S. Chavez, “Digital merging of Landsat-TM and digitized NHAP data for 1:24,000-scale image mapping,” Photogram. Eng. Remote Sens. 52, 140–146 (1986).
  19. G. Cliche, F. Bonn, P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogram. Eng. Remote Sens. 51, 311–316 (1985).
  20. A. R. Smith, “Color gamut transform pairs,” Comput. Graphics 12, 12–19 (1978).
    [CrossRef]

1994 (1)

J. Rosiene, I. Greenshields, “Standard wavelet basis compression of images,” Opt. Eng. 33, 2572–2578 (1994).
[CrossRef]

1992 (2)

V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogram. Eng. Remote Sens. 58, 561–567 (1992).

S. Mallat, S. Zhong, “Characterization of signals from multiscale edges,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 710–732 (1992).
[CrossRef]

1991 (1)

O. Rioul, M. Vetterli, “Wavelets and signal processing,” IEEE Signal Process. Mag. 8(4), 14–38 (1991).
[CrossRef]

1990 (1)

W. J. Carper, T. M. Lillesand, R. W. Kiefer, “The use of intensity–hue–saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogram. Eng. Remote Sens. 56, 459–467 (1990).

1989 (2)

S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 674–693 (1989).
[CrossRef]

A. Toet, L. J. van Ruyven, J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28, 789–792 (1989).
[CrossRef]

1988 (1)

I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math. 91, 909–996 (1988).
[CrossRef]

1986 (1)

P. S. Chavez, “Digital merging of Landsat-TM and digitized NHAP data for 1:24,000-scale image mapping,” Photogram. Eng. Remote Sens. 52, 140–146 (1986).

1985 (1)

G. Cliche, F. Bonn, P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogram. Eng. Remote Sens. 51, 311–316 (1985).

1978 (1)

A. R. Smith, “Color gamut transform pairs,” Comput. Graphics 12, 12–19 (1978).
[CrossRef]

1948 (1)

Antonini, M.

M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, “Image coding using vector quantization in the wavelet transform domain,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM (Institute of Electrical and Electronics Engineers, New York, 1990), pp. 2297–2300.
[CrossRef]

Bare, J. E.

R. Haydn, G. W. Dalke, J. Henkel, J. E. Bare, “Application of the IHS color transform to the processing of multi-sensor data and image enhancement,” in Proceedings of the International Symposium on Remote Sensing of Arid and Semi-Arid Lands, Cairo, Egypt (Environmental Research Institute, Ann Arbor, Mich., 1982), pp. 599–616.

Barlaud, M.

M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, “Image coding using vector quantization in the wavelet transform domain,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM (Institute of Electrical and Electronics Engineers, New York, 1990), pp. 2297–2300.
[CrossRef]

Bonn, F.

G. Cliche, F. Bonn, P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogram. Eng. Remote Sens. 51, 311–316 (1985).

Burt, P. J.

P. J. Burt, “Multiresolution techniques for image representation, analysis, and ‘smart’ transmission,” Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1199, 2–15 (1989).
[CrossRef]

Carper, W. J.

W. J. Carper, T. M. Lillesand, R. W. Kiefer, “The use of intensity–hue–saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogram. Eng. Remote Sens. 56, 459–467 (1990).

Chavez, P. S.

P. S. Chavez, “Digital merging of Landsat-TM and digitized NHAP data for 1:24,000-scale image mapping,” Photogram. Eng. Remote Sens. 52, 140–146 (1986).

Cliche, G.

G. Cliche, F. Bonn, P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogram. Eng. Remote Sens. 51, 311–316 (1985).

Dalke, G. W.

R. Haydn, G. W. Dalke, J. Henkel, J. E. Bare, “Application of the IHS color transform to the processing of multi-sensor data and image enhancement,” in Proceedings of the International Symposium on Remote Sensing of Arid and Semi-Arid Lands, Cairo, Egypt (Environmental Research Institute, Ann Arbor, Mich., 1982), pp. 599–616.

Daubechies, I.

I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math. 91, 909–996 (1988).
[CrossRef]

M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, “Image coding using vector quantization in the wavelet transform domain,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM (Institute of Electrical and Electronics Engineers, New York, 1990), pp. 2297–2300.
[CrossRef]

Froment, J.

J. Froment, S. Mallat, “Second generation compact image coding with wavelets,” in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed. (Academic, San Diego, Calif., 1992), pp. 655–678.

Greenshields, I.

J. Rosiene, I. Greenshields, “Standard wavelet basis compression of images,” Opt. Eng. 33, 2572–2578 (1994).
[CrossRef]

Hall, D. L.

D. L. Hall, Mathematical Techniques in Multisensor Data Fusion (Artech House, Boston, 1992).

Haydn, R.

R. Haydn, G. W. Dalke, J. Henkel, J. E. Bare, “Application of the IHS color transform to the processing of multi-sensor data and image enhancement,” in Proceedings of the International Symposium on Remote Sensing of Arid and Semi-Arid Lands, Cairo, Egypt (Environmental Research Institute, Ann Arbor, Mich., 1982), pp. 599–616.

Henkel, J.

R. Haydn, G. W. Dalke, J. Henkel, J. E. Bare, “Application of the IHS color transform to the processing of multi-sensor data and image enhancement,” in Proceedings of the International Symposium on Remote Sensing of Arid and Semi-Arid Lands, Cairo, Egypt (Environmental Research Institute, Ann Arbor, Mich., 1982), pp. 599–616.

Kiefer, R. W.

W. J. Carper, T. M. Lillesand, R. W. Kiefer, “The use of intensity–hue–saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogram. Eng. Remote Sens. 56, 459–467 (1990).

Levine, M. D.

M. D. Levine, Vision in Man and Machine (McGraw-Hill, San Francisco, Calif., 1985).

Lillesand, T. M.

W. J. Carper, T. M. Lillesand, R. W. Kiefer, “The use of intensity–hue–saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogram. Eng. Remote Sens. 56, 459–467 (1990).

Mallat, S.

S. Mallat, S. Zhong, “Characterization of signals from multiscale edges,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 710–732 (1992).
[CrossRef]

J. Froment, S. Mallat, “Second generation compact image coding with wavelets,” in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed. (Academic, San Diego, Calif., 1992), pp. 655–678.

Mallat, S. G.

S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 674–693 (1989).
[CrossRef]

Mathieu, P.

M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, “Image coding using vector quantization in the wavelet transform domain,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM (Institute of Electrical and Electronics Engineers, New York, 1990), pp. 2297–2300.
[CrossRef]

Rioul, O.

O. Rioul, M. Vetterli, “Wavelets and signal processing,” IEEE Signal Process. Mag. 8(4), 14–38 (1991).
[CrossRef]

Rose, A.

Rosiene, J.

J. Rosiene, I. Greenshields, “Standard wavelet basis compression of images,” Opt. Eng. 33, 2572–2578 (1994).
[CrossRef]

Shettigara, V. K.

V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogram. Eng. Remote Sens. 58, 561–567 (1992).

Smith, A. R.

A. R. Smith, “Color gamut transform pairs,” Comput. Graphics 12, 12–19 (1978).
[CrossRef]

Teillet, P.

G. Cliche, F. Bonn, P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogram. Eng. Remote Sens. 51, 311–316 (1985).

Toet, A.

A. Toet, L. J. van Ruyven, J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28, 789–792 (1989).
[CrossRef]

Valeton, J. M.

A. Toet, L. J. van Ruyven, J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28, 789–792 (1989).
[CrossRef]

van Ruyven, L. J.

A. Toet, L. J. van Ruyven, J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28, 789–792 (1989).
[CrossRef]

Vetterli, M.

O. Rioul, M. Vetterli, “Wavelets and signal processing,” IEEE Signal Process. Mag. 8(4), 14–38 (1991).
[CrossRef]

Wechsler, H.

H. Wechsler, Computational Vision (Academic, San Diego, Calif., 1990).

Zhong, S.

S. Mallat, S. Zhong, “Characterization of signals from multiscale edges,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 710–732 (1992).
[CrossRef]

Commun. Pure Appl. Math. (1)

I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Commun. Pure Appl. Math. 91, 909–996 (1988).
[CrossRef]

Comput. Graphics (1)

A. R. Smith, “Color gamut transform pairs,” Comput. Graphics 12, 12–19 (1978).
[CrossRef]

IEEE Signal Process. Mag. (1)

O. Rioul, M. Vetterli, “Wavelets and signal processing,” IEEE Signal Process. Mag. 8(4), 14–38 (1991).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

S. Mallat, S. Zhong, “Characterization of signals from multiscale edges,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 710–732 (1992).
[CrossRef]

S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 674–693 (1989).
[CrossRef]

J. Opt. Soc. Am. (1)

Opt. Eng. (2)

J. Rosiene, I. Greenshields, “Standard wavelet basis compression of images,” Opt. Eng. 33, 2572–2578 (1994).
[CrossRef]

A. Toet, L. J. van Ruyven, J. M. Valeton, “Merging thermal and visual images by a contrast pyramid,” Opt. Eng. 28, 789–792 (1989).
[CrossRef]

Photogram. Eng. Remote Sens. (4)

P. S. Chavez, “Digital merging of Landsat-TM and digitized NHAP data for 1:24,000-scale image mapping,” Photogram. Eng. Remote Sens. 52, 140–146 (1986).

G. Cliche, F. Bonn, P. Teillet, “Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement,” Photogram. Eng. Remote Sens. 51, 311–316 (1985).

W. J. Carper, T. M. Lillesand, R. W. Kiefer, “The use of intensity–hue–saturation transformations for merging SPOT panchromatic and multispectral image data,” Photogram. Eng. Remote Sens. 56, 459–467 (1990).

V. K. Shettigara, “A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set,” Photogram. Eng. Remote Sens. 58, 561–567 (1992).

Other (8)

The viewpoint of the author is from a remote-sensing perspective of the terms multispectral and panchromatic. In signal processing, multispectral can be confused with Fourier information, also referred to as spectral information. In this paper multispectral and spectral deal only with the wavelength, electromagnetic spectrum sense of the word, which is also synonymous with the term multichannel.

H. Wechsler, Computational Vision (Academic, San Diego, Calif., 1990).

M. D. Levine, Vision in Man and Machine (McGraw-Hill, San Francisco, Calif., 1985).

P. J. Burt, “Multiresolution techniques for image representation, analysis, and ‘smart’ transmission,” Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1199, 2–15 (1989).
[CrossRef]

M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, “Image coding using vector quantization in the wavelet transform domain,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Albuquerque, NM (Institute of Electrical and Electronics Engineers, New York, 1990), pp. 2297–2300.
[CrossRef]

J. Froment, S. Mallat, “Second generation compact image coding with wavelets,” in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed. (Academic, San Diego, Calif., 1992), pp. 655–678.

D. L. Hall, Mathematical Techniques in Multisensor Data Fusion (Artech House, Boston, 1992).

R. Haydn, G. W. Dalke, J. Henkel, J. E. Bare, “Application of the IHS color transform to the processing of multi-sensor data and image enhancement,” in Proceedings of the International Symposium on Remote Sensing of Arid and Semi-Arid Lands, Cairo, Egypt (Environmental Research Institute, Ann Arbor, Mich., 1982), pp. 599–616.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Multiresolution wavelet decomposition of the mandrill image. The final 1/8 resolution approximation is in the top-left corner. The others are detail images for each resolution level.

Plate 1
Plate 1

(a) Original mandrill image, (b) low-resolution color image of the mandrill, (c) high-resolution panchromatic image of the mandrill.

Plate 2
Plate 2

Comparison of the MS image and Haar wavelet mergers: (a) low-resolution multispectral image, (b) merged image with use of H11, (c) merged image with use of H21, (d) merged image with use of H31.

Plate 3
Plate 3

Comparison of the MS image and Daubechies wavelet mergers: (a) low-resolution multispeetral image, (b) merged image with use of D611, (c) merged image with use of D621, (d) merged image with use of D631.

Plate 4
Plate 4

Comparison of the original mandrill image and reconstructed merged images: (a) original mandrill image, (b) IHS merged image, (c) MWD merged image with use of D631.

Tables (5)

Tables Icon

Table 1 Comparison of the MWD and IHS Mergers by Means of the RMS Pixel Error over the Entire Imagea

Tables Icon

Table 2 Best-Case Image Mergera

Tables Icon

Table 3 Comparison of the D6rp and IHS Mergers by Means of the RMS Pixel Error Averaged over the Entire Imagea

Tables Icon

Table 4 Comparison of the RMS Pixel Error of MWD Merged Images with the Original Imagea

Tables Icon

Table 5 RMS Pixel Error of MWD and IHS Merged Images with Various Magnitudes of Translation in the Horizontal Direction Compared with the Original Image

Equations (20)

Equations on this page are rendered with MathJax. Learn more.

ϕ 2 j ( x ) = 2 j ϕ ( 2 j x ) ;
2 - j ϕ 2 j ( x - 2 - j n ) ( n , j ) Z 2
ψ 2 j ( x ) = 2 j ψ ( 2 j x )
2 - j ψ 2 j ( x - 2 - j n ) ( n , j ) Z 2
f ^ 2 j ( x ) = k { ϕ 2 - 1 ( x ) , ϕ [ x - ( k - 2 n ) ] × f ( x ) , ϕ 2 j + 1 ( x - 2 - j - 1 k ) } ,
h ( n ) = ϕ 2 - 1 ( x ) , ϕ ( x - n )
f ^ 2 j ( x ) = k h ˜ ( 2 x - k ) f ^ 2 j + 1 ( k ) ,
d f 2 j ( x ) = k { ψ 2 - 1 ( x ) , ϕ [ x - ( k - 2 n ) ] × f ( x ) , ϕ 2 j + 1 ( x - 2 - j - 1 k ) } .
g ( n ) = ψ 2 - 1 ( x ) , ϕ ( x - n ) ,
d f 2 j ( x ) = k g ˜ ( 2 x - k ) f ^ 2 j + 1 ( k ) .
f ^ 2 j + 1 ( x ) = k h ˜ ( 2 k - x ) f ^ 2 j ( k ) + g ˜ ( 2 k - x ) d f 2 j ( k ) .
Φ ( x , y ) = ϕ ( x ) ϕ ( y ) .
ψ 1 ( x , y ) = ϕ ( x ) ψ ( y ) , ψ 2 ( x , y ) = ψ ( x ) ϕ ( y ) , ψ 3 ( x , y ) = ψ ( x ) ψ ( y ) .
f ^ 2 j + 1 ( x ) = k h ˜ ( 2 k - x ) α ^ 2 j ( k ) + g ˜ ( 2 k - x ) d f 2 j ( k ) ,
f ^ 2 j + 1 ( x ) = k h ˜ ( 2 k - x ) f ^ 2 j ( k ) + g ˜ ( 2 k - x ) Δ f 2 j ( k ) ,
I ( i , j , λ k ) = λ x y I ( x , y , λ ) R k ( λ ) Γ ( x , y ; i , j ) ,
I ( x , y ) = a I ( x , y , λ 1 ) + b I ( x , y , λ 2 ) + c I ( x , y , λ 3 )
I ( x , y ) = λ I ( x , y , λ ) R ( λ ) ,
ξ = ( c - c ^ ) 2 ,
ψ ( x ) = { 1 if 0 x < 1 / 2 - 1 if 1 / 2 x < 1 0 otherwise .

Metrics