Abstract

We propose a framework for scene-based nonuniformity correction (NUC) and nonuniformity correction and enhancement (NUCE) that is required for focal-plane array-like sensors to obtain clean and enhanced-quality images. The core of the proposed framework is a novel registration-based nonuniformity correction super-resolution (NUCSR) method that is bootstrapped by statistical scene-based NUC methods. Based on a comprehensive imaging model and an accurate parametric motion estimation, we are able to remove severe/structured nonuniformity and in the presence of subpixel motion to simultaneously improve image resolution. One important feature of our NUCSR method is the adoption of a parametric motion model that allows us to (1) handle many practical scenarios where parametric motions are present and (2) carry out perfect super-resolution in principle by exploring available subpixel motions. Experiments with real data demonstrate the efficiency of the proposed NUCE framework and the effectiveness of the NUCSR method.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Teich, “Digital infrared imaging for medicine: Recent advances in IR focal plane array imaging technology,” 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (1996), pp. 2079-2080.
  2. R. Helfrich, “Programmable compensation techniques for staring arrays,” Proc. SPIE 178, 110-121 (1979).
  3. Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
    [CrossRef]
  4. “Studies on future broadcasting services and fundamental technologies,” Nihon Hohsoh Kyokai (NHK) Science & Technology Research Lab (STRL) Annual Report 2004 (NHK, 2005), pp. 35-149.
  5. J. Caulfield, “Next generation IR focal plane arrays and applications,” in Proceedings of the Annual Applied Imagery Pattern Recognition Workshop (2003).
    [CrossRef]
  6. S. Cain, E. Armstrong, and B. Yasuda, “Joint estimation of image, shift, and nonuniformities from infrared images,” in Proceedings of the Meeting of the Infrared Information Symposium (IRIS) Speciality Group on Passive Sensors (1997), pp. 121-132.
  7. D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
    [CrossRef]
  8. A. Milton, F. Barone, and M. Kruer, “Influence of nonuniformity on infrared focal plane array performance,” Opt. Eng. 24, 855-862 (1985).
  9. S. Tzimopoulou and A. Lettington, “Scene based techniques for nonuniformity correction of infrared focal plane arrays,” Proc. SPIE 3436, 173-183 (1998).
  10. R. Hardie, M. Hayat, E. Armstrong, and B. Yasuda, “Scene-based nonuniformity correction with video sequences and registration,” Appl. Opt. 39, 1241-1250 (2000).
    [CrossRef]
  11. B. Ratliff, M. Hayat, and J. Tyo, “Generalized algebraic scene-based nonuniformity correction algorithm,” J. Opt. Soc. Am. A 22, 239-249 (2005).
    [CrossRef]
  12. W. Zhao and C. Zhang, “Efficient scene-based nonuniformity correction and enhancement,” in Proceedings of the International Conference on Image Processing (2006), pp. 2873-2876.
  13. R. Schultz and R. Stevenson, “Extraction of high resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996-1011 (1996).
    [CrossRef] [PubMed]
  14. B. Bascle, A. Blake, and A. Zisserman, “Motion deblurring and super-resolution from an image sequence,” in Proceedings of the European Conference on Computer Vision, (1996), pp. 573-581.
  15. S. Cain, M. Hayat, and E. Armstrong, “Projection-based image registration in the presence of fixed-pattern noise,” IEEE Trans. Image Process. 10, 1860-1872 (2001).
    [CrossRef]
  16. E. Armstrong, M. Hayat, R. Hardie, S. Torres, and B. Yasuda, “Non-uniformity correction for improved registration and high-resolution image reconstruction in IR imagery,” Proc. SPIE 3808, 150-161 (1999).
    [CrossRef]
  17. J. Harris and Y. Chiang, “Nonuniformity correction of infrared image sequences using the constant-statistics constraint,” IEEE Trans. Image Process. 8, 1148-1151 (1999).
    [CrossRef]
  18. M. Hayat, S. Torres, E. Armstrong, S. Cain, and B. Yasuda, “Statistical algorithm for nonuniformity correction in focal-plane arrays,” Appl. Opt. 38, 772-780 (1999).
    [CrossRef]
  19. S. Torres and M. Hayat, “Kalman filtering for adaptive nonuniformity correction in infrared focal-plane arrays,” J. Opt. Soc. Am. A 20, 470-480 (2003).
    [CrossRef]
  20. P. Narendra and N. Foss, “Shutterless fixed pattern noise correction for infrared imaging array,” Proc. SPIE 282, 44-51 (1981).
  21. B. Narayanan, R. Hardie, and R. Muse, “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architectures,” Appl. Opt. 44, 3482-3491 (2005).
    [CrossRef] [PubMed]
  22. W. O'Neil, “Dither image scanner with compensation for individual detector response and gain correction,” U. S. patent 5,514,865 (May 7, 1996).
  23. B. Ratliff, M. Hayat, and R. Hardie, “An algebraic algorithm for nonuniformity correction in focal-plane arrays,” J. Opt. Soc. Am. A 19, 1737-1747 (2002).
    [CrossRef]
  24. B. Ratliff, M. Hayat, and J. Tyo, “Radiometrically accurate scene-based nonuniformity correction for array sensors,” J. Opt. Soc. Am. A 20, 1890-1899 (2003).
    [CrossRef]
  25. R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
    [CrossRef]
  26. J. Gillette, T. Stadtmiller, and R. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34, 3130-3137 (1995).
    [CrossRef]
  27. W. Zhao and H. Sawhney, “Is optical flow based super-resolution feasible?” in Proceedings of the European Conference on Computer Vision (2002).
  28. G. Wolberg, Digital Image Warping (IEEE Computer Society Press, 1990).
  29. M. Irani and S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” J. Visual Commun. Image Represent 4, 324-335 (1993).
    [CrossRef]
  30. R. Young, An Introduction to Nonharmonic Fourier Series (Academic, 1980).
  31. R. Y. Tsai, and T. S. Huang, “Multiframe image restoration and registration” in Advances in Computer Vision and Image Processing (JAI Press Inc., 1984).
  32. A. Patti, M. Sezan, and M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, pp. 1064-1076 (1997).
    [CrossRef] [PubMed]
  33. P. Marziliano and M. Vetterlli, “Reconstruction of irregularly sampled discrete-time bandlimited signals,” IEEE Trans. Image Process. 3462-3471 (1999).
  34. M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy and undersampled measured images,” IEEE Trans. Image Process. 6, 1646-1658 (1997).
    [CrossRef] [PubMed]
  35. J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

2006 (1)

W. Zhao and C. Zhang, “Efficient scene-based nonuniformity correction and enhancement,” in Proceedings of the International Conference on Image Processing (2006), pp. 2873-2876.

2005 (2)

2003 (3)

2002 (3)

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

W. Zhao and H. Sawhney, “Is optical flow based super-resolution feasible?” in Proceedings of the European Conference on Computer Vision (2002).

B. Ratliff, M. Hayat, and R. Hardie, “An algebraic algorithm for nonuniformity correction in focal-plane arrays,” J. Opt. Soc. Am. A 19, 1737-1747 (2002).
[CrossRef]

2001 (1)

S. Cain, M. Hayat, and E. Armstrong, “Projection-based image registration in the presence of fixed-pattern noise,” IEEE Trans. Image Process. 10, 1860-1872 (2001).
[CrossRef]

2000 (1)

1999 (4)

E. Armstrong, M. Hayat, R. Hardie, S. Torres, and B. Yasuda, “Non-uniformity correction for improved registration and high-resolution image reconstruction in IR imagery,” Proc. SPIE 3808, 150-161 (1999).
[CrossRef]

J. Harris and Y. Chiang, “Nonuniformity correction of infrared image sequences using the constant-statistics constraint,” IEEE Trans. Image Process. 8, 1148-1151 (1999).
[CrossRef]

M. Hayat, S. Torres, E. Armstrong, S. Cain, and B. Yasuda, “Statistical algorithm for nonuniformity correction in focal-plane arrays,” Appl. Opt. 38, 772-780 (1999).
[CrossRef]

P. Marziliano and M. Vetterlli, “Reconstruction of irregularly sampled discrete-time bandlimited signals,” IEEE Trans. Image Process. 3462-3471 (1999).

1998 (2)

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

S. Tzimopoulou and A. Lettington, “Scene based techniques for nonuniformity correction of infrared focal plane arrays,” Proc. SPIE 3436, 173-183 (1998).

1997 (3)

S. Cain, E. Armstrong, and B. Yasuda, “Joint estimation of image, shift, and nonuniformities from infrared images,” in Proceedings of the Meeting of the Infrared Information Symposium (IRIS) Speciality Group on Passive Sensors (1997), pp. 121-132.

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy and undersampled measured images,” IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

A. Patti, M. Sezan, and M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, pp. 1064-1076 (1997).
[CrossRef] [PubMed]

1996 (3)

J. Teich, “Digital infrared imaging for medicine: Recent advances in IR focal plane array imaging technology,” 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (1996), pp. 2079-2080.

R. Schultz and R. Stevenson, “Extraction of high resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

B. Bascle, A. Blake, and A. Zisserman, “Motion deblurring and super-resolution from an image sequence,” in Proceedings of the European Conference on Computer Vision, (1996), pp. 573-581.

1995 (1)

J. Gillette, T. Stadtmiller, and R. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34, 3130-3137 (1995).
[CrossRef]

1993 (1)

M. Irani and S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” J. Visual Commun. Image Represent 4, 324-335 (1993).
[CrossRef]

1992 (1)

J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

1990 (2)

G. Wolberg, Digital Image Warping (IEEE Computer Society Press, 1990).

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

1985 (1)

A. Milton, F. Barone, and M. Kruer, “Influence of nonuniformity on infrared focal plane array performance,” Opt. Eng. 24, 855-862 (1985).

1984 (1)

R. Y. Tsai, and T. S. Huang, “Multiframe image restoration and registration” in Advances in Computer Vision and Image Processing (JAI Press Inc., 1984).

1981 (1)

P. Narendra and N. Foss, “Shutterless fixed pattern noise correction for infrared imaging array,” Proc. SPIE 282, 44-51 (1981).

1980 (1)

R. Young, An Introduction to Nonharmonic Fourier Series (Academic, 1980).

1979 (1)

R. Helfrich, “Programmable compensation techniques for staring arrays,” Proc. SPIE 178, 110-121 (1979).

Anadan, P.

J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

Armstrong, E.

S. Cain, M. Hayat, and E. Armstrong, “Projection-based image registration in the presence of fixed-pattern noise,” IEEE Trans. Image Process. 10, 1860-1872 (2001).
[CrossRef]

R. Hardie, M. Hayat, E. Armstrong, and B. Yasuda, “Scene-based nonuniformity correction with video sequences and registration,” Appl. Opt. 39, 1241-1250 (2000).
[CrossRef]

E. Armstrong, M. Hayat, R. Hardie, S. Torres, and B. Yasuda, “Non-uniformity correction for improved registration and high-resolution image reconstruction in IR imagery,” Proc. SPIE 3808, 150-161 (1999).
[CrossRef]

M. Hayat, S. Torres, E. Armstrong, S. Cain, and B. Yasuda, “Statistical algorithm for nonuniformity correction in focal-plane arrays,” Appl. Opt. 38, 772-780 (1999).
[CrossRef]

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

S. Cain, E. Armstrong, and B. Yasuda, “Joint estimation of image, shift, and nonuniformities from infrared images,” in Proceedings of the Meeting of the Infrared Information Symposium (IRIS) Speciality Group on Passive Sensors (1997), pp. 121-132.

Barnard, K.

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

Barone, F.

A. Milton, F. Barone, and M. Kruer, “Influence of nonuniformity on infrared focal plane array performance,” Opt. Eng. 24, 855-862 (1985).

Bascle, B.

B. Bascle, A. Blake, and A. Zisserman, “Motion deblurring and super-resolution from an image sequence,” in Proceedings of the European Conference on Computer Vision, (1996), pp. 573-581.

Bergen, J.

J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

Blake, A.

B. Bascle, A. Blake, and A. Zisserman, “Motion deblurring and super-resolution from an image sequence,” in Proceedings of the European Conference on Computer Vision, (1996), pp. 573-581.

Bognar, J.

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

Cain, S.

S. Cain, M. Hayat, and E. Armstrong, “Projection-based image registration in the presence of fixed-pattern noise,” IEEE Trans. Image Process. 10, 1860-1872 (2001).
[CrossRef]

M. Hayat, S. Torres, E. Armstrong, S. Cain, and B. Yasuda, “Statistical algorithm for nonuniformity correction in focal-plane arrays,” Appl. Opt. 38, 772-780 (1999).
[CrossRef]

S. Cain, E. Armstrong, and B. Yasuda, “Joint estimation of image, shift, and nonuniformities from infrared images,” in Proceedings of the Meeting of the Infrared Information Symposium (IRIS) Speciality Group on Passive Sensors (1997), pp. 121-132.

Caulfield, J.

J. Caulfield, “Next generation IR focal plane arrays and applications,” in Proceedings of the Annual Applied Imagery Pattern Recognition Workshop (2003).
[CrossRef]

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

Chiang, Y.

J. Harris and Y. Chiang, “Nonuniformity correction of infrared image sequences using the constant-statistics constraint,” IEEE Trans. Image Process. 8, 1148-1151 (1999).
[CrossRef]

Elad, M.

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy and undersampled measured images,” IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Feuer, A.

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy and undersampled measured images,” IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

Foss, N.

P. Narendra and N. Foss, “Shutterless fixed pattern noise correction for infrared imaging array,” Proc. SPIE 282, 44-51 (1981).

Gillette, J.

J. Gillette, T. Stadtmiller, and R. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34, 3130-3137 (1995).
[CrossRef]

Gridley, C.

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

Hanna, K.

J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

Hardie, R.

B. Narayanan, R. Hardie, and R. Muse, “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architectures,” Appl. Opt. 44, 3482-3491 (2005).
[CrossRef] [PubMed]

B. Ratliff, M. Hayat, and R. Hardie, “An algebraic algorithm for nonuniformity correction in focal-plane arrays,” J. Opt. Soc. Am. A 19, 1737-1747 (2002).
[CrossRef]

R. Hardie, M. Hayat, E. Armstrong, and B. Yasuda, “Scene-based nonuniformity correction with video sequences and registration,” Appl. Opt. 39, 1241-1250 (2000).
[CrossRef]

E. Armstrong, M. Hayat, R. Hardie, S. Torres, and B. Yasuda, “Non-uniformity correction for improved registration and high-resolution image reconstruction in IR imagery,” Proc. SPIE 3808, 150-161 (1999).
[CrossRef]

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

J. Gillette, T. Stadtmiller, and R. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34, 3130-3137 (1995).
[CrossRef]

Harris, J.

J. Harris and Y. Chiang, “Nonuniformity correction of infrared image sequences using the constant-statistics constraint,” IEEE Trans. Image Process. 8, 1148-1151 (1999).
[CrossRef]

Hayat, M.

Helfrich, R.

R. Helfrich, “Programmable compensation techniques for staring arrays,” Proc. SPIE 178, 110-121 (1979).

Hingorani, R.

J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

Horowitz, R.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

Huang, T. S.

R. Y. Tsai, and T. S. Huang, “Multiframe image restoration and registration” in Advances in Computer Vision and Image Processing (JAI Press Inc., 1984).

Irani, M.

M. Irani and S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” J. Visual Commun. Image Represent 4, 324-335 (1993).
[CrossRef]

Katz, G.

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

Kitching, J.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

Kruer, M.

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

A. Milton, F. Barone, and M. Kruer, “Influence of nonuniformity on infrared focal plane array performance,” Opt. Eng. 24, 855-862 (1985).

Lettington, A.

S. Tzimopoulou and A. Lettington, “Scene based techniques for nonuniformity correction of infrared focal plane arrays,” Proc. SPIE 3436, 173-183 (1998).

Mai, M.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

Majumdar, A.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

Marziliano, P.

P. Marziliano and M. Vetterlli, “Reconstruction of irregularly sampled discrete-time bandlimited signals,” IEEE Trans. Image Process. 3462-3471 (1999).

Milton, A.

A. Milton, F. Barone, and M. Kruer, “Influence of nonuniformity on infrared focal plane array performance,” Opt. Eng. 24, 855-862 (1985).

Muse, R.

Narayanan, B.

Narendra, P.

P. Narendra and N. Foss, “Shutterless fixed pattern noise correction for infrared imaging array,” Proc. SPIE 282, 44-51 (1981).

Norton, P.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

O'Neil, W.

W. O'Neil, “Dither image scanner with compensation for individual detector response and gain correction,” U. S. patent 5,514,865 (May 7, 1996).

Patti, A.

A. Patti, M. Sezan, and M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, pp. 1064-1076 (1997).
[CrossRef] [PubMed]

Peleg, S.

M. Irani and S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” J. Visual Commun. Image Represent 4, 324-335 (1993).
[CrossRef]

Ratliff, B.

Sarkay, K.

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

Sawhney, H.

W. Zhao and H. Sawhney, “Is optical flow based super-resolution feasible?” in Proceedings of the European Conference on Computer Vision (2002).

Schultz, R.

R. Schultz and R. Stevenson, “Extraction of high resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

Scribner, D.

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

Sezan, M.

A. Patti, M. Sezan, and M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, pp. 1064-1076 (1997).
[CrossRef] [PubMed]

Stadtmiller, T.

J. Gillette, T. Stadtmiller, and R. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34, 3130-3137 (1995).
[CrossRef]

Stevenson, R.

R. Schultz and R. Stevenson, “Extraction of high resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

Teich, J.

J. Teich, “Digital infrared imaging for medicine: Recent advances in IR focal plane array imaging technology,” 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (1996), pp. 2079-2080.

Tekalp, M.

A. Patti, M. Sezan, and M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, pp. 1064-1076 (1997).
[CrossRef] [PubMed]

Torres, S.

Tsai, R. Y.

R. Y. Tsai, and T. S. Huang, “Multiframe image restoration and registration” in Advances in Computer Vision and Image Processing (JAI Press Inc., 1984).

Tyo, J.

Tzimopoulou, S.

S. Tzimopoulou and A. Lettington, “Scene based techniques for nonuniformity correction of infrared focal plane arrays,” Proc. SPIE 3436, 173-183 (1998).

Varesi, J.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

Vetterlli, M.

P. Marziliano and M. Vetterlli, “Reconstruction of irregularly sampled discrete-time bandlimited signals,” IEEE Trans. Image Process. 3462-3471 (1999).

Watson, R.

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

Wolberg, G.

G. Wolberg, Digital Image Warping (IEEE Computer Society Press, 1990).

Yasuda, B.

R. Hardie, M. Hayat, E. Armstrong, and B. Yasuda, “Scene-based nonuniformity correction with video sequences and registration,” Appl. Opt. 39, 1241-1250 (2000).
[CrossRef]

E. Armstrong, M. Hayat, R. Hardie, S. Torres, and B. Yasuda, “Non-uniformity correction for improved registration and high-resolution image reconstruction in IR imagery,” Proc. SPIE 3808, 150-161 (1999).
[CrossRef]

M. Hayat, S. Torres, E. Armstrong, S. Cain, and B. Yasuda, “Statistical algorithm for nonuniformity correction in focal-plane arrays,” Appl. Opt. 38, 772-780 (1999).
[CrossRef]

S. Cain, E. Armstrong, and B. Yasuda, “Joint estimation of image, shift, and nonuniformities from infrared images,” in Proceedings of the Meeting of the Infrared Information Symposium (IRIS) Speciality Group on Passive Sensors (1997), pp. 121-132.

Young, R.

R. Young, An Introduction to Nonharmonic Fourier Series (Academic, 1980).

Zhang, C.

W. Zhao and C. Zhang, “Efficient scene-based nonuniformity correction and enhancement,” in Proceedings of the International Conference on Image Processing (2006), pp. 2873-2876.

Zhao, W.

W. Zhao and C. Zhang, “Efficient scene-based nonuniformity correction and enhancement,” in Proceedings of the International Conference on Image Processing (2006), pp. 2873-2876.

W. Zhao and H. Sawhney, “Is optical flow based super-resolution feasible?” in Proceedings of the European Conference on Computer Vision (2002).

Zhao, Y.

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

Zisserman, A.

B. Bascle, A. Blake, and A. Zisserman, “Motion deblurring and super-resolution from an image sequence,” in Proceedings of the European Conference on Computer Vision, (1996), pp. 573-581.

Appl. Opt. (3)

IEEE Trans. Image Process. (6)

A. Patti, M. Sezan, and M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, pp. 1064-1076 (1997).
[CrossRef] [PubMed]

P. Marziliano and M. Vetterlli, “Reconstruction of irregularly sampled discrete-time bandlimited signals,” IEEE Trans. Image Process. 3462-3471 (1999).

M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy and undersampled measured images,” IEEE Trans. Image Process. 6, 1646-1658 (1997).
[CrossRef] [PubMed]

R. Schultz and R. Stevenson, “Extraction of high resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996-1011 (1996).
[CrossRef] [PubMed]

S. Cain, M. Hayat, and E. Armstrong, “Projection-based image registration in the presence of fixed-pattern noise,” IEEE Trans. Image Process. 10, 1860-1872 (2001).
[CrossRef]

J. Harris and Y. Chiang, “Nonuniformity correction of infrared image sequences using the constant-statistics constraint,” IEEE Trans. Image Process. 8, 1148-1151 (1999).
[CrossRef]

J. Microelectromech. Syst. (1)

Y. Zhao, M. Mai, R. Horowitz, A. Majumdar, J. Varesi, P. Norton, and J. Kitching, “Optomechanical uncooled infrared imaging system: Design, microfabrication, and performance,” J. Microelectromech. Syst. 11, 136-146 (2002).
[CrossRef]

J. Opt. Soc. Am. A (4)

J. Visual Commun. Image Represent (1)

M. Irani and S. Peleg, “Motion analysis for image enhancement: Resolution, occlusion, and transparency,” J. Visual Commun. Image Represent 4, 324-335 (1993).
[CrossRef]

Opt. Eng. (3)

R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and R. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247-260 (1998).
[CrossRef]

J. Gillette, T. Stadtmiller, and R. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34, 3130-3137 (1995).
[CrossRef]

A. Milton, F. Barone, and M. Kruer, “Influence of nonuniformity on infrared focal plane array performance,” Opt. Eng. 24, 855-862 (1985).

Proc. SPIE (5)

S. Tzimopoulou and A. Lettington, “Scene based techniques for nonuniformity correction of infrared focal plane arrays,” Proc. SPIE 3436, 173-183 (1998).

R. Helfrich, “Programmable compensation techniques for staring arrays,” Proc. SPIE 178, 110-121 (1979).

D. Scribner, K. Sarkay, J. Caulfield, M. Kruer, G. Katz, and C. Gridley, “Nonuniformity correction of staring IR focal plane arrays using scene-based techniques,” Proc. SPIE 1308, 224-233 (1990).
[CrossRef]

E. Armstrong, M. Hayat, R. Hardie, S. Torres, and B. Yasuda, “Non-uniformity correction for improved registration and high-resolution image reconstruction in IR imagery,” Proc. SPIE 3808, 150-161 (1999).
[CrossRef]

P. Narendra and N. Foss, “Shutterless fixed pattern noise correction for infrared imaging array,” Proc. SPIE 282, 44-51 (1981).

Other (12)

W. O'Neil, “Dither image scanner with compensation for individual detector response and gain correction,” U. S. patent 5,514,865 (May 7, 1996).

W. Zhao and H. Sawhney, “Is optical flow based super-resolution feasible?” in Proceedings of the European Conference on Computer Vision (2002).

G. Wolberg, Digital Image Warping (IEEE Computer Society Press, 1990).

R. Young, An Introduction to Nonharmonic Fourier Series (Academic, 1980).

R. Y. Tsai, and T. S. Huang, “Multiframe image restoration and registration” in Advances in Computer Vision and Image Processing (JAI Press Inc., 1984).

J. Bergen, P. Anadan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision (1992) pp. 237-252.

W. Zhao and C. Zhang, “Efficient scene-based nonuniformity correction and enhancement,” in Proceedings of the International Conference on Image Processing (2006), pp. 2873-2876.

B. Bascle, A. Blake, and A. Zisserman, “Motion deblurring and super-resolution from an image sequence,” in Proceedings of the European Conference on Computer Vision, (1996), pp. 573-581.

J. Teich, “Digital infrared imaging for medicine: Recent advances in IR focal plane array imaging technology,” 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (1996), pp. 2079-2080.

“Studies on future broadcasting services and fundamental technologies,” Nihon Hohsoh Kyokai (NHK) Science & Technology Research Lab (STRL) Annual Report 2004 (NHK, 2005), pp. 35-149.

J. Caulfield, “Next generation IR focal plane arrays and applications,” in Proceedings of the Annual Applied Imagery Pattern Recognition Workshop (2003).
[CrossRef]

S. Cain, E. Armstrong, and B. Yasuda, “Joint estimation of image, shift, and nonuniformities from infrared images,” in Proceedings of the Meeting of the Infrared Information Symposium (IRIS) Speciality Group on Passive Sensors (1997), pp. 121-132.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Typical motion fields (images of motion vectors ( Δ t x , Δ t y ) ) computed based on [35] from two video sequences used in our experiments. (a) Two motion fields from two other frames to the reference frame for the real video clip A (Fig. 7 below). (b) Two motion fields from two other frames to the reference frame for the synthetic sequence D (Fig. 10 below).

Fig. 2
Fig. 2

Registration-based NUCSR method. Several iterations are typically required for the best image reconstruction in the iterative approach where each iteration carries out the processes described in (A), (B), and(C). Images of z t s r are produced by statistical NUC methods for registration.

Fig. 3
Fig. 3

Comparison of nonstructured and structured fixed-pattern noise: (a) Synthetic case, where Gaussian fixed-pattern noise is presented and (b) real case, where the offset image is recovered from a video clip acquired by an actual long-wave IR camera using the method proposed in this paper. To see why we call case (b) a structured fixed-pattern noise, we plot the horizontal profiles of both offset images. The numbers represent the grayscale levels of the profiles.

Fig. 4
Fig. 4

One challenging example of handling structured fixed-pattern noise when the image motion is roughly parallel to the noise-pattern structure [the white arrow in plot (a)]. In such a case, neither the (c) temporal average s a nor the (d) deblurred image s d is a good initial estimate. Both estimates have biases compared to the original clean image s r . However, (e)–(h) subsequent iterations are able to significantly reduce such estimation biases. Notice that the initial estimate based on (d) image deblurring is better than (c) temporal average in terms of image quality, for example, along the image border. Estimate [plot (b)] based on [10] achieves a result similar to plots (e) and (f) with more blurred image quality. Multiframe image deblurring is a special case of super-resolution with scale s = 1 please refer to Subsection 3B2.

Fig. 5
Fig. 5

Impact-of image deblurring (a special case of super-resolution with scale s = 1 ) upon the image quality of the final estimate of s r . Notice that image deblurring is able to sharpen the image and repair artifacts [plot (a) versus plot (b)]. Refined result based on [10] is similar to plot (c) with a few more artifacts.

Fig. 6
Fig. 6

Statistical methods for image recovery based on a small number of frames. Image registration cannot be applied based on images (a) in the noisy image sequence. While images (b) recovered from statistical methods do not have good quality in terms of the difference from the true s r [plot (c)], they are good enough for image registration and can be used to bootstrap the registration-based NUCSR method after only 100 frames.

Fig. 7
Fig. 7

Real noisy video clip A (image size of odd fields 720 × 240 ): NUCE result using the sensor referred to in Fig. 3b that has structured fixed-pattern noise. Comparing plot (d) against the original noisy image plot (a), we see that the NUCSR estimate is clean and has better resolution (e.g., less aliased along the edges of bricks). Comparing plot (d) against alternative s r estimates, we see that: (1) s a (b) is more blurred due to averaging and it still contains fixed-pattern noise inside the window area and (2) the refined estimate (c) has strong aliasing and creates artifacts on the top-middle image border and still contains random noise inside the window area.

Fig. 8
Fig. 8

Synthetic sequence B (image size 704 × 480 ): (a) Synthetic noisy image z r , (b) temporal average s a of 31 frames, (c) statistical estimate of s r using 100 frames, and (d) NUCSR estimate of s r using 31 frames. Image registration has been successfully applied after the 100th frame using images recovered from statistical methods (refer to Fig. 6).

Fig. 9
Fig. 9

Real noisy video clip C [original image (odd fields) size 720 × 240 ]: NUCSR result. Comparing plot (a) against plot (b), we notice that not only is noise removed but also the image resolution has been improved (sharper and much less aliasing).

Fig. 10
Fig. 10

Synthetic sequence D (original image size 360 × 244 ): (a) Cubic-interpolated noisy image, (b) NUCSR estimate of S r , and (c) comparison of zoomed-up views in region A from cubic-interpolated s a , NUCSR estimate of S r , and cubic-interpolated true s r .

Fig. 11
Fig. 11

Issue of nonlinearity (synthetic sequence B). (a) Estimate of s r after one iteration using a linear model and (b) estimate of s r after one iteration using a nonlinear model (ignoring the pixels with values outside the range [15:240]). Notice that plot (a) has more artifacts (vertical strips) than plot (b). Please note that we deliberately turned off the super-resolution step C (Fig. 2). That is, we want to expose the nonlinear issue and study its impact upon the image quality of the s r estimate. If step C is turned on, the difference will become less obvious but still exists.

Tables (1)

Tables Icon

Table 1 Comparison of Work on Scene-Based NUC and/Image Enhancement a

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

z t ( x , y ) = g t ( x , y ) s t ( x , y ) + b t ( x , y ) + N ( x , y ) ,
s t = { S t h } s ,
z t ( x , y ) = g ( x , y ) { S t ( x , y ) h } s + b ( x , y ) + N ( x , y ) .
s r ( x , y ) = s t ( x + Δ t x , y + Δ t y ) ,
s t = s r W t ,
{ S r * , g * , b * } = arg S r , g , b min t = 1 T ( z t g s t ( S r ) b ) 2 ,
s t ( S r ) = { ( S r ) F t h } s .
s a = 1 T t = 1 T ( g s t + b ) W t + 1 T N = s r 1 T t = 1 T g W t + 1 T t = 1 T b W t + 1 T N .
s a s r + 1 T t = 1 T b W t .
b a = 1 T t = 1 T ( z t s ̂ t ) .
I [ n + 1 ] = I [ n ] + 1 T t = 1 T { [ ( f t f t ( I [ n ] ) s ) ] F t p } ,
z t = { g s t + b + N , θ L g s t + b + N θ H θ H , g s t + b + N θ H θ L , g s t + b + N θ L ,

Metrics