Abstract

Fast nonuniform interpolation based super-resolution (SR) has traditionally been limited to applications with translational interframe motion. This is in part because such methods are based on an underlying assumption that the warping and blurring components in the observation model commute. For translational motion this is the case, but it is not true in general. This presents a problem for applications such as airborne imaging where translation may be insufficient. Here we present a new Fourier domain analysis to show that, for many image systems, an affine warping model with limited zoom and shear approximately commutes with the point spread function when diffraction effects are modeled. Based on this important result, we present a new fast adaptive Wiener filter (AWF) SR algorithm for non-translational motion and study its performance with affine motion. The fast AWF SR method employs a new smart observation window that allows us to precompute all the needed filter weights for any type of motion without sacrificing much of the full performance of the AWF. We evaluate the proposed algorithm using simulated data and real infrared airborne imagery that contains a thermal resolution target allowing for objective resolution analysis.

© 2011 OSA

Full Article  |  PDF Article
OSA Recommended Articles
Fast super-resolution using an adaptive Wiener filter with robustness to local motion

Russell C. Hardie and Kenneth J. Barnard
Opt. Express 20(19) 21053-21073 (2012)

Computationally efficient video restoration for Nyquist sampled imaging sensors combining an affine-motion-based temporal Kalman filter and adaptive Wiener filter

Michael Rucci, Russell C. Hardie, and Kenneth J. Barnard
Appl. Opt. 53(13) C1-C13 (2014)

Adaptive Wiener filter super-resolution of color filter array images

Barry K. Karch and Russell C. Hardie
Opt. Express 21(16) 18820-18841 (2013)

More Recommended Articles

References

  • View by:
  • Article Order
  • |
  • Year
  • |
  • Author
  • |
  • Publication
  1. S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Mag. 20, 21–36 (2003).
    [Crossref]
  2. D. P. Capel, “Image mosaicing and super-resolution,” Ph.D. thesis, University of Oxford (2001).
  3. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
    [Crossref]
  4. G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
    [Crossref]
  5. R. D. Fiete, “Image quality and λ FN/ p for remote sensing systems,” Optical Engineering 38, 1229–1240 (1999).
    [Crossref]
  6. B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
    [Crossref]
  7. R. C. Hardie, “A fast super-resolution algorithm using an adaptive wiener filter,” IEEE Trans. Image Processing 16, 2953–2964 (2007).
    [Crossref]
  8. F. O. Baxley, K. J. Barnard, R. C. Hardie, and M. A. Bicknell, “Flight test results of a rapid step-stare and microscan midwave infrared sensor concept for persistent surveillance,” in Proceedings of MSS Passive Sensors, (Orlando, FL, 2010).
  9. M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Processing 6, 1646–1658 (1997).
    [Crossref]
  10. R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
    [Crossref]
  11. E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision (Prentice Hall, 1998).
  12. B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in International Joint Conference on Artificial Intelligence, 674–679 (Vancouver, 1981).
  13. J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the Second European Conference on Computer Vision, 237–252 (Springer-Verlag, 1992).
  14. J. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1968).
  15. R. A. Emmert and C. D. McGillem, “Multitemporal geometric distortion correction utilizing the affine transformation,” LARS Technical Reports, Paper 114 (1973).
  16. C. W. Therrian, Discrete Random Signals and Statistical Signal Processing (Prentice Hall, 1992).
  17. J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Reduction of aliasing in staring infrared imagers utilizing subpixel techniques,” Optical Engineering 34, 3130–3137 (1995).
    [Crossref]
  18. M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
    [Crossref]
  19. S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using delaunay triangulation,” IEEE Trans. Image Processing 11, 1427–1441 (2002).
    [Crossref]

2007 (2)

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
[Crossref]

R. C. Hardie, “A fast super-resolution algorithm using an adaptive wiener filter,” IEEE Trans. Image Processing 16, 2953–2964 (2007).
[Crossref]

2006 (1)

G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
[Crossref]

2004 (1)

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
[Crossref]

2003 (1)

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Mag. 20, 21–36 (2003).
[Crossref]

2002 (1)

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using delaunay triangulation,” IEEE Trans. Image Processing 11, 1427–1441 (2002).
[Crossref]

2000 (1)

M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
[Crossref]

1999 (1)

R. D. Fiete, “Image quality and λ FN/ p for remote sensing systems,” Optical Engineering 38, 1229–1240 (1999).
[Crossref]

1998 (1)

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

1997 (1)

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Processing 6, 1646–1658 (1997).
[Crossref]

1995 (1)

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Reduction of aliasing in staring infrared imagers utilizing subpixel techniques,” Optical Engineering 34, 3130–3137 (1995).
[Crossref]

Alam, M. S.

M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
[Crossref]

Anandan, P.

J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the Second European Conference on Computer Vision, 237–252 (Springer-Verlag, 1992).

Armstrong, E. E.

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

Barnard, K. J.

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

F. O. Baxley, K. J. Barnard, R. C. Hardie, and M. A. Bicknell, “Flight test results of a rapid step-stare and microscan midwave infrared sensor concept for persistent surveillance,” in Proceedings of MSS Passive Sensors, (Orlando, FL, 2010).

Barner, K. E.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
[Crossref]

Baxley, F. O.

F. O. Baxley, K. J. Barnard, R. C. Hardie, and M. A. Bicknell, “Flight test results of a rapid step-stare and microscan midwave infrared sensor concept for persistent surveillance,” in Proceedings of MSS Passive Sensors, (Orlando, FL, 2010).

Bergen, J. R.

J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the Second European Conference on Computer Vision, 237–252 (Springer-Verlag, 1992).

Besnerais, G. L.

G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
[Crossref]

Bicknell, M. A.

F. O. Baxley, K. J. Barnard, R. C. Hardie, and M. A. Bicknell, “Flight test results of a rapid step-stare and microscan midwave infrared sensor concept for persistent surveillance,” in Proceedings of MSS Passive Sensors, (Orlando, FL, 2010).

Bognar, J. G.

M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
[Crossref]

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

Bose, N. K.

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using delaunay triangulation,” IEEE Trans. Image Processing 11, 1427–1441 (2002).
[Crossref]

Capel, D. P.

D. P. Capel, “Image mosaicing and super-resolution,” Ph.D. thesis, University of Oxford (2001).

Champagnat, F.

G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
[Crossref]

Elad, M.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
[Crossref]

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Processing 6, 1646–1658 (1997).
[Crossref]

Emmert, R. A.

R. A. Emmert and C. D. McGillem, “Multitemporal geometric distortion correction utilizing the affine transformation,” LARS Technical Reports, Paper 114 (1973).

Farsiu, S.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
[Crossref]

Feuer, A.

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Processing 6, 1646–1658 (1997).
[Crossref]

Fiete, R. D.

R. D. Fiete, “Image quality and λ FN/ p for remote sensing systems,” Optical Engineering 38, 1229–1240 (1999).
[Crossref]

franois Giovannelli, J.

G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
[Crossref]

Gillette, J. C.

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Reduction of aliasing in staring infrared imagers utilizing subpixel techniques,” Optical Engineering 34, 3130–3137 (1995).
[Crossref]

Hanna, K. J.

J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the Second European Conference on Computer Vision, 237–252 (Springer-Verlag, 1992).

Hardie, R. C.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
[Crossref]

R. C. Hardie, “A fast super-resolution algorithm using an adaptive wiener filter,” IEEE Trans. Image Processing 16, 2953–2964 (2007).
[Crossref]

M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
[Crossref]

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Reduction of aliasing in staring infrared imagers utilizing subpixel techniques,” Optical Engineering 34, 3130–3137 (1995).
[Crossref]

F. O. Baxley, K. J. Barnard, R. C. Hardie, and M. A. Bicknell, “Flight test results of a rapid step-stare and microscan midwave infrared sensor concept for persistent surveillance,” in Proceedings of MSS Passive Sensors, (Orlando, FL, 2010).

Hingorani, R.

J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the Second European Conference on Computer Vision, 237–252 (Springer-Verlag, 1992).

Kanade, T.

B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in International Joint Conference on Artificial Intelligence, 674–679 (Vancouver, 1981).

Kang, M. G.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Mag. 20, 21–36 (2003).
[Crossref]

Lertrattanapanich, S.

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using delaunay triangulation,” IEEE Trans. Image Processing 11, 1427–1441 (2002).
[Crossref]

Lucas, B. D.

B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in International Joint Conference on Artificial Intelligence, 674–679 (Vancouver, 1981).

McGillem, C. D.

R. A. Emmert and C. D. McGillem, “Multitemporal geometric distortion correction utilizing the affine transformation,” LARS Technical Reports, Paper 114 (1973).

Milanfar, P.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
[Crossref]

Narayanan, B.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
[Crossref]

Park, M. K.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Mag. 20, 21–36 (2003).
[Crossref]

Park, S. C.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Mag. 20, 21–36 (2003).
[Crossref]

Robinson, D.

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
[Crossref]

Rochefort, G.

G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
[Crossref]

Shao, M.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
[Crossref]

Stadtmiller, T. M.

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Reduction of aliasing in staring infrared imagers utilizing subpixel techniques,” Optical Engineering 34, 3130–3137 (1995).
[Crossref]

Therrian, C. W.

C. W. Therrian, Discrete Random Signals and Statistical Signal Processing (Prentice Hall, 1992).

Trucco, E.

E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision (Prentice Hall, 1998).

Verri, A.

E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision (Prentice Hall, 1998).

Watson, E. A.

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

Yasuda, B. J.

M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
[Crossref]

IEEE Signal Processing Mag. (1)

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Processing Mag. 20, 21–36 (2003).
[Crossref]

IEEE Trans. Circuits Syst. Video Technol. (1)

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Trans. Circuits Syst. Video Technol. 17, 621–634 (2007).
[Crossref]

IEEE Trans. Image Processing (5)

R. C. Hardie, “A fast super-resolution algorithm using an adaptive wiener filter,” IEEE Trans. Image Processing 16, 2953–2964 (2007).
[Crossref]

S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super-resolution,” IEEE Trans. Image Processing 13, 1327–1344 (2004).
[Crossref]

G. Rochefort, F. Champagnat, G. L. Besnerais, and J. franois Giovannelli, “An improved observation model for super-resolution under affine motion,” IEEE Trans. Image Processing 15, 3325–3337 (2006).
[Crossref]

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Processing 6, 1646–1658 (1997).
[Crossref]

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using delaunay triangulation,” IEEE Trans. Image Processing 11, 1427–1441 (2002).
[Crossref]

IEEE Trans. Instrum. Meas. (1)

M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000).
[Crossref]

Optical Engineering (3)

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Reduction of aliasing in staring infrared imagers utilizing subpixel techniques,” Optical Engineering 34, 3130–3137 (1995).
[Crossref]

R. D. Fiete, “Image quality and λ FN/ p for remote sensing systems,” Optical Engineering 38, 1229–1240 (1999).
[Crossref]

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Optical Engineering 37, 247–260 (1998).
[Crossref]

Other (8)

E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision (Prentice Hall, 1998).

B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in International Joint Conference on Artificial Intelligence, 674–679 (Vancouver, 1981).

J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the Second European Conference on Computer Vision, 237–252 (Springer-Verlag, 1992).

J. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1968).

R. A. Emmert and C. D. McGillem, “Multitemporal geometric distortion correction utilizing the affine transformation,” LARS Technical Reports, Paper 114 (1973).

C. W. Therrian, Discrete Random Signals and Statistical Signal Processing (Prentice Hall, 1992).

D. P. Capel, “Image mosaicing and super-resolution,” Ph.D. thesis, University of Oxford (2001).

F. O. Baxley, K. J. Barnard, R. C. Hardie, and M. A. Bicknell, “Flight test results of a rapid step-stare and microscan midwave infrared sensor concept for persistent surveillance,” in Proceedings of MSS Passive Sensors, (Orlando, FL, 2010).

Supplementary Material (4)

» Media 1: AVI (521 KB)     
» Media 2: AVI (3686 KB)     
» Media 3: AVI (3811 KB)     
» Media 4: AVI (3695 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1
Fig. 1

Warp-then-blur observation model relating a desired 2-D continuous scene, d(x,y), to a set of corresponding LR frames. This model follows the physical image acquisition process and is the basis for most of the iterative SR algorithms.

Download Full Size | PPT Slide | PDF

Fig. 2
Fig. 2

(a) Cross-section of the MWIR system MTF and its components. (b) MWIR system PSF model.

Download Full Size | PPT Slide | PDF

Fig. 3
Fig. 3

Blur-then-warp observation model that performs nonuniform sampling of a single blurred image. This version of the observation model is valid when the motion model and PSF blurring commute. The fast interpolation-restoration SR methods (including the AWF) are based on this model.

Download Full Size | PPT Slide | PDF

Fig. 4
Fig. 4

Error-to-signal energy spectral density ratio analysis based on Eq. (15). (a) 2-D MWIR imaging system OTF, H(u). Γ(u) for the MWIR system with (b) a rotation of 20 degrees (c) zoom of 10%, (d) horizontal shear of 10%.

Download Full Size | PPT Slide | PDF

Fig. 5
Fig. 5

Peak ESR value for the MWIR imaging system with various f-numbers (Q values) for (a) rotation, (b) zoom, and (c) shear. The actual system has an f-number of 2.30 (Q=0.472).

Download Full Size | PPT Slide | PDF

Fig. 6
Fig. 6

Fast AWF for non-translational motion block diagram.

Download Full Size | PPT Slide | PDF

Fig. 7
Fig. 7

Minimum MSE weights for estimating the position shown with the red plus sign for four different population patterns ( Media 1). Only samples in the green boxes are assumed to be available and the grayscale value represents the weight with background gray corresponding to zero. The pattern in (a) is the case of no motion or a single frame (the reference frame).

Download Full Size | PPT Slide | PDF

Fig. 8
Fig. 8

Expected fraction of the HR grid populated as a function of the number of frames with uniform motion.

Download Full Size | PPT Slide | PDF

Fig. 9
Fig. 9

Subwindows designed using the forward sequential selection method for Lx = Ly = 3, ρ = 0.7, σ d 2 / σ n 2 = 100, K = 10 and M = 16. The estimation position is shown with a red plus sign and the reference grid samples are shown in green. The selected positions are shown with blue numbers in order of selection.

Download Full Size | PPT Slide | PDF

Fig. 10
Fig. 10

Theoretical expected MSE values normalized by σ d 2 as a function of M for the subwindows in Figs. 9(a), 9(b), and 9(d).

Download Full Size | PPT Slide | PDF

Fig. 11
Fig. 11

Flight data regions of interest processed. (a) 50Hz sequence frame (b) 16Hz sequence frame. The thermal resolution target area is boxed in each image.

Download Full Size | PPT Slide | PDF

Fig. 12
Fig. 12

SR results for 50Hz MWIR flight sequence for Lx = Ly = 3 and K = 10. (a) Bicubic interpolation, (b) partially populated HR grid, (c) fast AWF SR method (d) WNN, (e) Delaunay, (f) RLS (20 interations).

Download Full Size | PPT Slide | PDF

Fig. 13
Fig. 13

SR results for 50Hz MWIR data simulating no camera motion by repeating a single frame K = 10 times. Algorithm tuning parameters are the same as in Fig. 12. The results are for (a) fast AWF SR method, (b) WNN, (c) Delaunay, (d) RLS (20 interations).

Download Full Size | PPT Slide | PDF

Fig. 14
Fig. 14

SR results for K = 10 frames of the 16Hz MWIR sequence using the fast AWF SR method with different motion models. Algorithm parameters are the same as in Fig. 12. The results are for (a) bicubic interpolation ( Media 2, left), (b) the fast AWF SR with affine motion model ( Media 2, right), (c) rotation and translation, (d) translation only.

Download Full Size | PPT Slide | PDF

Fig. 15
Fig. 15

Single-frame excerpts from video results using the proposed fast AWF SR method with data from the visible grayscale camera described in Section 2.3. (a) 2-D chirp sequence ( Media 3). (b) bookshelf sequence ( Media 4).

Download Full Size | PPT Slide | PDF

Tables (1)

Tables Icon

Table 1 MSE for SR with affine motion for aerial image (Lx = Ly = 3 and K = 10).

View Table

Equations (24)

Equations on this page are rendered with MathJax. Learn more.

A k = [ A 1 , 1 ( k ) A 1 , 2 ( k ) A 2 , 1 ( k ) A 2 , 2 ( k ) ]
d k ( x ) = d ( x ˜ ( k ) ) = d ( A k x + t k ) .
d k ( x ) ≈ d ( x ) + ( x ˜ ( k ) − x ) g x ( x ) + ( y ˜ ( k ) − y ) g y ( x ) .
M k = [ x 1 g x ( x 1 ) y 1 g x ( x 1 ) g x ( x 1 ) x 1 g y ( x 1 ) y 1 g y ( x 1 ) g y ( x 1 ) x 2 g x ( x 2 ) y 2 g x ( x 2 ) g x ( x 2 ) x 2 g y ( x 2 ) y 2 g y ( x 2 ) g y ( x 2 ) ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ x N g x ( x N ) y N g x ( x N ) g x ( x N ) x N g y ( x N ) y N g y ( x N ) g y ( x N ) ] ,
b k = [ d k ( x 1 ) − d ( x 1 ) + x 1 g x ( x 1 ) + y 1 g y ( x 1 ) d k ( x 2 ) − d ( x 2 ) + x 2 g x ( x 2 ) + y 2 g y ( x 2 ) ⋮ d k ( x N ) − d ( x N ) + x N g x ( x N ) + y N g y ( x N ) ] .
a ^ k = ( M k T M k ) − 1 M k T b k .
H ( u , v ) = H dif ( u , v ) H det ( u , v ) ,
H dif ( u , v ) = { 2 π [ cos − 1 ( ω / ω c ) − ( ω / ω c ) 1 − ( ω / ω c ) 2 ] for   ω < ω c 0 else ,
f k ( x ) = d k ( x ) * h ( x ) = d ( A k x + t k ) * h ( x ) .
D k ( u ) = 1 | A k | e j 2 π u ⋅ A k − 1 t k D ( A k − T u ) ,
F k ( u ) = 1 | A k | e j 2 π u ⋅ A k − 1 t k D ( A k − T u ) H ( u ) .
F ˜ k ( u ) = 1 | A k | e j 2 π u ⋅ A k − 1 t k F ( A k − T u ) = 1 | A k | e j 2 π u ⋅ A k − 1 t k D ( A k − T u ) H ( A k − T u ) .
E k ( u ) = D k ( u ) [ H ( u ) − H ( A k − T u ) ] .
Φ E k ( u ) = | D k ( u ) | 2 | [ H ( u ) − H ( A k − T u ) ] | 2 .
Γ ( u ) = Φ E k ( u ) Φ D k ( u ) = | [ H ( u ) − H ( A k − T u ) ] | 2 .
J ( w | Ψ = ψ ( i ) ) = E { ( d i − d ^ i ) 2 | Ψ = ψ ( i ) } = E { ( d i − w T g i ) 2 | Ψ = ψ ( i ) } ,
J ( w | Ψ = ψ ( i ) ) = E { d i 2 } − 2 w T E { d i g i | Ψ = ψ ( i ) } + w T E { g i g i T | Ψ = ψ ( i ) } w .
J ( w | Ψ = ψ ( i ) ) = E { d i 2 } − 2 w T p ψ ( i ) + w T R ψ ( i ) w .
w ψ ( i ) = R ψ ( i ) − 1 p ψ ( i ) .
J ( w ψ ( i ) | Ψ = ψ ( i ) ) = E { d i 2 } − p ψ ( i ) T R ψ ( i ) − T p ψ ( i ) .
r d d ( x , y ) = σ d 2 ρ x 2 + y 2 ,
r d f ( x , y ) = r d d ( x , y ) * h ( x , y ) .
r f f ( x , y ) = r d d ( x , y ) * h ( x , y ) * h ( − x , − y ) .
J = E { J ( w ψ | Ψ ) } = ∑ ψ = 1 2 M J ( w ψ | Ψ = ψ ) Pr { Ψ = ψ } ,

Metrics