Abstract

Staggered time delay and integration (TDI) scanning image acquisition systems are usually employed in low signal-to-noise situations such as thermal imaging. Analysis and restoration of images acquired by thermal staggered TDI sensors in the presence of mechanical vibrations that may cause space-variant image distortions (severe geometric warps and blur) are studied. The relative motion at each location in the degraded image is identified from the image when a differential technique is used. This information is then used to reconstruct the image by a technique of projection onto convex sets. The main novelty is the implementation of such methods to scanned images (columnwise). Restorations are performed with simulated and real mechanically degraded thermal images.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
    [CrossRef]
  2. A. Fenster, D. W. Holdsworth, M. Drangova, “Three-dimensional imaging using TDI CCD sensors,” in Charge-Coupled Devices and Solid State Optical Sensors II, M. M. Blouea, eds., Proc. SPIE1447, 28–33 (1991).
    [CrossRef]
  3. V. Sankaran, C. M. Weber, K. W. Tobin, “Inspection in Semiconductor manufacturing,” in Webster’s Encyclopedia of Electrical and Electronic Engineering (Wiley, New York, 1999), Vol. 10, pp. 242–262.
  4. H. S. Wong, Y. L. Yao, E. S. Schlig, “TDI charge coupled devices: design and applications,” IBM J. Res. Dev. 36, 83–104 (1992).
    [CrossRef]
  5. M. Berger, Y. Lauber, M. Citroen, J. M. Topaz, “Cos4 compensation and optimal TDI operation in multielement linear TDI IR detectors,” in Infrared Technology and Applications XXVIII, B. F. Andresen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 830–839 (2002).
    [CrossRef]
  6. M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
    [CrossRef]
  7. D. F. Barbe, “Charge-coupled imaging devices,” in Charge-Coupled Devices: Technology and Applications, R. Melen, D. Buss, eds. (Institute of Electrical and Electronics Engineers, New York, 1977), pp. 144–147.
  8. O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibrations. Part IV: Real time numerical calculation of the OTF and experimental verification,” Opt. Eng. 33, 566–578 (1994).
    [CrossRef]
  9. Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
    [CrossRef]
  10. Y. Yitzhaky, A. Stern, “Restoration of interlaced images degraded by variable velocity motion,” Opt. Eng. 42, 3557–3565 (2003).
    [CrossRef]
  11. J. Miettinen, H. Ailisto, “Using a time delay and integration camera at nonzero viewing angle under vibration condition,” Opt. Lasers Eng. 31, 223–238 (1999).
    [CrossRef]
  12. G. Wolberg, R. P. Loce, “Restoration of images scanned in the presence of vibrations,” J. Electron. Imaging 5, 50–61 (1996).
    [CrossRef]
  13. W. L. Barnes, T. S. Pagano, V. V. Salomonson, “Prelaunch characteristics of the moderate resolution imaging spectroradiometer (MODIS) on EOS-AM1,” IEEE Trans. Geosci. Remote Sens. 36, 1088–1100 (1998).
    [CrossRef]
  14. C. Stiller, J. Konrad, “Estimating motions in image sequences—a tutorial on modeling and computation of 2D motion,” IEEE Signal Process. Mag. 16, 70–91 (1999).
    [CrossRef]
  15. G. L. Brown, “A survey on image registration techniques,” ACM Comput. Surv. (CSUR) 24, 325–376 (1992).
    [CrossRef]
  16. M. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graphi. Models Image Process. 53, 231–239 (1991).
    [CrossRef]
  17. S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
    [CrossRef]
  18. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
    [CrossRef]
  19. C. L. Luengo Hendriks, L. J. van Vliet, “Improving resolution to reduce aliasing in an under sampled image sequence,” in Sensors, Cameras, and Systems, for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, G. M. Williams, N. Sampat, T. Yeh, eds., Proc. SPIE3965, 214–222 (2000).
    [CrossRef]
  20. M. Elad, P. Teo, Y. Hel-Or, “Optimal filters for gradient based motion estimation,” in Proceedings of the International Conference on Computer Vision, Corfu, Greece, 1999 (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 559–565.
  21. D. C. Youla, H. Webb, “Image restoration by method of convex projections: Part 1— theory,” IEEE Trans. Med. Imaging MI-1, 81–94 (1982).
    [CrossRef]
  22. H. Stark, P. Oskoui, “High-resolution image recovery from image-plane arrays, using convex projections,” J. Opt. Soc. Am. A 6, 1715–1726 (1989).
    [CrossRef] [PubMed]
  23. A. J. Patti, M. I. Sezan, A. M. Teklap, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1075 (1997).
    [CrossRef]
  24. A. Stern, Y. Porat, A. Ben-Dor, N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40, 4706–4715 (2001).
    [CrossRef]
  25. H. Stark, Y. Yang, Vector Space Projections: A Numerical Approach to Signal and Image Processing Neural Nets and Optics (Wiley, New York, 1998).

2003 (2)

Y. Yitzhaky, A. Stern, “Restoration of interlaced images degraded by variable velocity motion,” Opt. Eng. 42, 3557–3565 (2003).
[CrossRef]

S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
[CrossRef]

2001 (1)

2000 (1)

Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
[CrossRef]

1999 (2)

C. Stiller, J. Konrad, “Estimating motions in image sequences—a tutorial on modeling and computation of 2D motion,” IEEE Signal Process. Mag. 16, 70–91 (1999).
[CrossRef]

J. Miettinen, H. Ailisto, “Using a time delay and integration camera at nonzero viewing angle under vibration condition,” Opt. Lasers Eng. 31, 223–238 (1999).
[CrossRef]

1998 (1)

W. L. Barnes, T. S. Pagano, V. V. Salomonson, “Prelaunch characteristics of the moderate resolution imaging spectroradiometer (MODIS) on EOS-AM1,” IEEE Trans. Geosci. Remote Sens. 36, 1088–1100 (1998).
[CrossRef]

1997 (1)

A. J. Patti, M. I. Sezan, A. M. Teklap, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1075 (1997).
[CrossRef]

1996 (1)

G. Wolberg, R. P. Loce, “Restoration of images scanned in the presence of vibrations,” J. Electron. Imaging 5, 50–61 (1996).
[CrossRef]

1994 (1)

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibrations. Part IV: Real time numerical calculation of the OTF and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

1992 (2)

H. S. Wong, Y. L. Yao, E. S. Schlig, “TDI charge coupled devices: design and applications,” IBM J. Res. Dev. 36, 83–104 (1992).
[CrossRef]

G. L. Brown, “A survey on image registration techniques,” ACM Comput. Surv. (CSUR) 24, 325–376 (1992).
[CrossRef]

1991 (1)

M. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graphi. Models Image Process. 53, 231–239 (1991).
[CrossRef]

1989 (1)

1982 (1)

D. C. Youla, H. Webb, “Image restoration by method of convex projections: Part 1— theory,” IEEE Trans. Med. Imaging MI-1, 81–94 (1982).
[CrossRef]

1981 (1)

K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Admon, D.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Ailisto, H.

J. Miettinen, H. Ailisto, “Using a time delay and integration camera at nonzero viewing angle under vibration condition,” Opt. Lasers Eng. 31, 223–238 (1999).
[CrossRef]

Barbe, D. F.

D. F. Barbe, “Charge-coupled imaging devices,” in Charge-Coupled Devices: Technology and Applications, R. Melen, D. Buss, eds. (Institute of Electrical and Electronics Engineers, New York, 1977), pp. 144–147.

Barnes, W. L.

W. L. Barnes, T. S. Pagano, V. V. Salomonson, “Prelaunch characteristics of the moderate resolution imaging spectroradiometer (MODIS) on EOS-AM1,” IEEE Trans. Geosci. Remote Sens. 36, 1088–1100 (1998).
[CrossRef]

Ben-Dor, A.

Berger, M.

M. Berger, Y. Lauber, M. Citroen, J. M. Topaz, “Cos4 compensation and optimal TDI operation in multielement linear TDI IR detectors,” in Infrared Technology and Applications XXVIII, B. F. Andresen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 830–839 (2002).
[CrossRef]

Boshusha, G.

Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
[CrossRef]

Brown, G. L.

G. L. Brown, “A survey on image registration techniques,” ACM Comput. Surv. (CSUR) 24, 325–376 (1992).
[CrossRef]

Cavanaugh, J.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Citroen, M.

M. Berger, Y. Lauber, M. Citroen, J. M. Topaz, “Cos4 compensation and optimal TDI operation in multielement linear TDI IR detectors,” in Infrared Technology and Applications XXVIII, B. F. Andresen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 830–839 (2002).
[CrossRef]

Drangova, M.

A. Fenster, D. W. Holdsworth, M. Drangova, “Three-dimensional imaging using TDI CCD sensors,” in Charge-Coupled Devices and Solid State Optical Sensors II, M. M. Blouea, eds., Proc. SPIE1447, 28–33 (1991).
[CrossRef]

Dror, I.

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibrations. Part IV: Real time numerical calculation of the OTF and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Elad, M.

M. Elad, P. Teo, Y. Hel-Or, “Optimal filters for gradient based motion estimation,” in Proceedings of the International Conference on Computer Vision, Corfu, Greece, 1999 (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 559–565.

Fenster, A.

A. Fenster, D. W. Holdsworth, M. Drangova, “Three-dimensional imaging using TDI CCD sensors,” in Charge-Coupled Devices and Solid State Optical Sensors II, M. M. Blouea, eds., Proc. SPIE1447, 28–33 (1991).
[CrossRef]

Fraenkel, A.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Grush, R. C.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Hadar, O.

S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
[CrossRef]

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibrations. Part IV: Real time numerical calculation of the OTF and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Haski, J.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Hel-Or, Y.

M. Elad, P. Teo, Y. Hel-Or, “Optimal filters for gradient based motion estimation,” in Proceedings of the International Conference on Computer Vision, Corfu, Greece, 1999 (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 559–565.

Hoffman, J. W.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Holdsworth, D. W.

A. Fenster, D. W. Holdsworth, M. Drangova, “Three-dimensional imaging using TDI CCD sensors,” in Charge-Coupled Devices and Solid State Optical Sensors II, M. M. Blouea, eds., Proc. SPIE1447, 28–33 (1991).
[CrossRef]

Horn, K. P.

K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Irani, M.

M. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graphi. Models Image Process. 53, 231–239 (1991).
[CrossRef]

Keinan, M.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Konrad, J.

C. Stiller, J. Konrad, “Estimating motions in image sequences—a tutorial on modeling and computation of 2D motion,” IEEE Signal Process. Mag. 16, 70–91 (1999).
[CrossRef]

Kopeika, N. S.

S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
[CrossRef]

A. Stern, Y. Porat, A. Ben-Dor, N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40, 4706–4715 (2001).
[CrossRef]

Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
[CrossRef]

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibrations. Part IV: Real time numerical calculation of the OTF and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Lauber, Y.

M. Berger, Y. Lauber, M. Citroen, J. M. Topaz, “Cos4 compensation and optimal TDI operation in multielement linear TDI IR detectors,” in Infrared Technology and Applications XXVIII, B. F. Andresen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 830–839 (2002).
[CrossRef]

Levi, Y.

Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
[CrossRef]

Loce, R. P.

G. Wolberg, R. P. Loce, “Restoration of images scanned in the presence of vibrations,” J. Electron. Imaging 5, 50–61 (1996).
[CrossRef]

Luengo Hendriks, C. L.

C. L. Luengo Hendriks, L. J. van Vliet, “Improving resolution to reduce aliasing in an under sampled image sequence,” in Sensors, Cameras, and Systems, for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, G. M. Williams, N. Sampat, T. Yeh, eds., Proc. SPIE3965, 214–222 (2000).
[CrossRef]

Malkinson, E.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Manizade, K.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Miettinen, J.

J. Miettinen, H. Ailisto, “Using a time delay and integration camera at nonzero viewing angle under vibration condition,” Opt. Lasers Eng. 31, 223–238 (1999).
[CrossRef]

Oskoui, P.

Pagano, T. S.

W. L. Barnes, T. S. Pagano, V. V. Salomonson, “Prelaunch characteristics of the moderate resolution imaging spectroradiometer (MODIS) on EOS-AM1,” IEEE Trans. Geosci. Remote Sens. 36, 1088–1100 (1998).
[CrossRef]

Palm, S.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Patti, A. J.

A. J. Patti, M. I. Sezan, A. M. Teklap, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1075 (1997).
[CrossRef]

Peleg, S.

M. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graphi. Models Image Process. 53, 231–239 (1991).
[CrossRef]

Pivnik, I.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Porat, Y.

Raiter, S.

S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
[CrossRef]

Reiner, T.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Salomonson, V. V.

W. L. Barnes, T. S. Pagano, V. V. Salomonson, “Prelaunch characteristics of the moderate resolution imaging spectroradiometer (MODIS) on EOS-AM1,” IEEE Trans. Geosci. Remote Sens. 36, 1088–1100 (1998).
[CrossRef]

Sankaran, V.

V. Sankaran, C. M. Weber, K. W. Tobin, “Inspection in Semiconductor manufacturing,” in Webster’s Encyclopedia of Electrical and Electronic Engineering (Wiley, New York, 1999), Vol. 10, pp. 242–262.

Sapir, N.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Sapiro, I.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Schlig, E. S.

H. S. Wong, Y. L. Yao, E. S. Schlig, “TDI charge coupled devices: design and applications,” IBM J. Res. Dev. 36, 83–104 (1992).
[CrossRef]

Schunck, B. G.

K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Scott, V. S.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Sezan, M. I.

A. J. Patti, M. I. Sezan, A. M. Teklap, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1075 (1997).
[CrossRef]

Spinhirne, J. D.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

Stark, H.

H. Stark, P. Oskoui, “High-resolution image recovery from image-plane arrays, using convex projections,” J. Opt. Soc. Am. A 6, 1715–1726 (1989).
[CrossRef] [PubMed]

H. Stark, Y. Yang, Vector Space Projections: A Numerical Approach to Signal and Image Processing Neural Nets and Optics (Wiley, New York, 1998).

Stern, A.

S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
[CrossRef]

Y. Yitzhaky, A. Stern, “Restoration of interlaced images degraded by variable velocity motion,” Opt. Eng. 42, 3557–3565 (2003).
[CrossRef]

A. Stern, Y. Porat, A. Ben-Dor, N. S. Kopeika, “Enhanced-resolution image restoration from a sequence of low-frequency vibrated images by use of convex projections,” Appl. Opt. 40, 4706–4715 (2001).
[CrossRef]

Stiller, C.

C. Stiller, J. Konrad, “Estimating motions in image sequences—a tutorial on modeling and computation of 2D motion,” IEEE Signal Process. Mag. 16, 70–91 (1999).
[CrossRef]

Teklap, A. M.

A. J. Patti, M. I. Sezan, A. M. Teklap, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1075 (1997).
[CrossRef]

Teo, P.

M. Elad, P. Teo, Y. Hel-Or, “Optimal filters for gradient based motion estimation,” in Proceedings of the International Conference on Computer Vision, Corfu, Greece, 1999 (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 559–565.

Tobin, K. W.

V. Sankaran, C. M. Weber, K. W. Tobin, “Inspection in Semiconductor manufacturing,” in Webster’s Encyclopedia of Electrical and Electronic Engineering (Wiley, New York, 1999), Vol. 10, pp. 242–262.

Topaz, J. M.

M. Berger, Y. Lauber, M. Citroen, J. M. Topaz, “Cos4 compensation and optimal TDI operation in multielement linear TDI IR detectors,” in Infrared Technology and Applications XXVIII, B. F. Andresen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 830–839 (2002).
[CrossRef]

van Vliet, L. J.

C. L. Luengo Hendriks, L. J. van Vliet, “Improving resolution to reduce aliasing in an under sampled image sequence,” in Sensors, Cameras, and Systems, for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, G. M. Williams, N. Sampat, T. Yeh, eds., Proc. SPIE3965, 214–222 (2000).
[CrossRef]

Webb, H.

D. C. Youla, H. Webb, “Image restoration by method of convex projections: Part 1— theory,” IEEE Trans. Med. Imaging MI-1, 81–94 (1982).
[CrossRef]

Weber, C. M.

V. Sankaran, C. M. Weber, K. W. Tobin, “Inspection in Semiconductor manufacturing,” in Webster’s Encyclopedia of Electrical and Electronic Engineering (Wiley, New York, 1999), Vol. 10, pp. 242–262.

Wolberg, G.

G. Wolberg, R. P. Loce, “Restoration of images scanned in the presence of vibrations,” J. Electron. Imaging 5, 50–61 (1996).
[CrossRef]

Wong, H. S.

H. S. Wong, Y. L. Yao, E. S. Schlig, “TDI charge coupled devices: design and applications,” IBM J. Res. Dev. 36, 83–104 (1992).
[CrossRef]

Yang, Y.

H. Stark, Y. Yang, Vector Space Projections: A Numerical Approach to Signal and Image Processing Neural Nets and Optics (Wiley, New York, 1998).

Yao, Y. L.

H. S. Wong, Y. L. Yao, E. S. Schlig, “TDI charge coupled devices: design and applications,” IBM J. Res. Dev. 36, 83–104 (1992).
[CrossRef]

Yassen, M.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

Yitzhaky, Y.

Y. Yitzhaky, A. Stern, “Restoration of interlaced images degraded by variable velocity motion,” Opt. Eng. 42, 3557–3565 (2003).
[CrossRef]

Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
[CrossRef]

Youla, D. C.

D. C. Youla, H. Webb, “Image restoration by method of convex projections: Part 1— theory,” IEEE Trans. Med. Imaging MI-1, 81–94 (1982).
[CrossRef]

Zucker, M.

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

ACM Comput. Surv. (CSUR) (1)

G. L. Brown, “A survey on image registration techniques,” ACM Comput. Surv. (CSUR) 24, 325–376 (1992).
[CrossRef]

Appl. Opt. (1)

Artif. Intell. (1)

K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

CVGIP: Graphi. Models Image Process (1)

M. Irani, S. Peleg, “Improving resolution by image registration,” CVGIP: Graphi. Models Image Process. 53, 231–239 (1991).
[CrossRef]

IBM J. Res. Dev. (1)

H. S. Wong, Y. L. Yao, E. S. Schlig, “TDI charge coupled devices: design and applications,” IBM J. Res. Dev. 36, 83–104 (1992).
[CrossRef]

IEEE Signal Process. Mag. (1)

C. Stiller, J. Konrad, “Estimating motions in image sequences—a tutorial on modeling and computation of 2D motion,” IEEE Signal Process. Mag. 16, 70–91 (1999).
[CrossRef]

IEEE Trans. Geosci. Remote Sens. (1)

W. L. Barnes, T. S. Pagano, V. V. Salomonson, “Prelaunch characteristics of the moderate resolution imaging spectroradiometer (MODIS) on EOS-AM1,” IEEE Trans. Geosci. Remote Sens. 36, 1088–1100 (1998).
[CrossRef]

IEEE Trans. Image Process. (1)

A. J. Patti, M. I. Sezan, A. M. Teklap, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1075 (1997).
[CrossRef]

IEEE Trans. Med. Imaging (1)

D. C. Youla, H. Webb, “Image restoration by method of convex projections: Part 1— theory,” IEEE Trans. Med. Imaging MI-1, 81–94 (1982).
[CrossRef]

J. Electron. Imaging (1)

G. Wolberg, R. P. Loce, “Restoration of images scanned in the presence of vibrations,” J. Electron. Imaging 5, 50–61 (1996).
[CrossRef]

J. Opt. Soc. Am. A (1)

Opt. Eng. (4)

S. Raiter, A. Stern, O. Hadar, N. S. Kopeika, “Image restoration from camera vibration and object motion blur in infrared staggered time-delay and integration systems,” Opt. Eng. 42, 3253–3264 (2003).
[CrossRef]

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibrations. Part IV: Real time numerical calculation of the OTF and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Y. Yitzhaky, G. Boshusha, Y. Levi, N. S. Kopeika, “Restoration of an image degraded by vibrations using only a single frame,” Opt. Eng. 38, 2083–2091 (2000).
[CrossRef]

Y. Yitzhaky, A. Stern, “Restoration of interlaced images degraded by variable velocity motion,” Opt. Eng. 42, 3557–3565 (2003).
[CrossRef]

Opt. Lasers Eng. (1)

J. Miettinen, H. Ailisto, “Using a time delay and integration camera at nonzero viewing angle under vibration condition,” Opt. Lasers Eng. 31, 223–238 (1999).
[CrossRef]

Other (9)

M. Berger, Y. Lauber, M. Citroen, J. M. Topaz, “Cos4 compensation and optimal TDI operation in multielement linear TDI IR detectors,” in Infrared Technology and Applications XXVIII, B. F. Andresen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 830–839 (2002).
[CrossRef]

M. Zucker, I. Pivnik, E. Malkinson, J. Haski, T. Reiner, D. Admon, M. Keinan, M. Yassen, I. Sapiro, N. Sapir, A. Fraenkel, “Long mid-wave infrared detector with time delayed integration,” in Infrared Technology and applications XXVIII, B. F. Andersen, G. F. Fulop, M. Strojnik, eds., Proc. SPIE4820, 580–592 (2002).
[CrossRef]

D. F. Barbe, “Charge-coupled imaging devices,” in Charge-Coupled Devices: Technology and Applications, R. Melen, D. Buss, eds. (Institute of Electrical and Electronics Engineers, New York, 1977), pp. 144–147.

C. L. Luengo Hendriks, L. J. van Vliet, “Improving resolution to reduce aliasing in an under sampled image sequence,” in Sensors, Cameras, and Systems, for Scientific, Industrial, and Digital Photography Applications, M. M. Blouke, G. M. Williams, N. Sampat, T. Yeh, eds., Proc. SPIE3965, 214–222 (2000).
[CrossRef]

M. Elad, P. Teo, Y. Hel-Or, “Optimal filters for gradient based motion estimation,” in Proceedings of the International Conference on Computer Vision, Corfu, Greece, 1999 (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 559–565.

J. D. Spinhirne, V. S. Scott, J. Cavanaugh, S. Palm, K. Manizade, J. W. Hoffman, R. C. Grush, “Preliminary space flight results from the uncooled infrared spectral imaging radiometer (ISIR) on shuttle mission STS-85,” in Infrared Detectors and Focal Plane Arrays V, E. L. Dereniak, R. E. Sampson, eds., Proc. SPIE3379, 14–21 (1998).
[CrossRef]

A. Fenster, D. W. Holdsworth, M. Drangova, “Three-dimensional imaging using TDI CCD sensors,” in Charge-Coupled Devices and Solid State Optical Sensors II, M. M. Blouea, eds., Proc. SPIE1447, 28–33 (1991).
[CrossRef]

V. Sankaran, C. M. Weber, K. W. Tobin, “Inspection in Semiconductor manufacturing,” in Webster’s Encyclopedia of Electrical and Electronic Engineering (Wiley, New York, 1999), Vol. 10, pp. 242–262.

H. Stark, Y. Yang, Vector Space Projections: A Numerical Approach to Signal and Image Processing Neural Nets and Optics (Wiley, New York, 1998).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Staggered TDI sensor configuration.

Fig. 2
Fig. 2

Effect of vibrations on the column PSF in the TDI imaging process.

Fig. 3
Fig. 3

Target (original high-resolution) images: (a) stripes, (b) port.

Fig. 4
Fig. 4

Effects of two-dimensional vibrations on staggered TDI images for different scan rates: (a) 2200 (pixels/sc) and (b) 350 (pixels/sc). Both images were simulated with N = 16 and margin = 3.

Fig. 5
Fig. 5

(a) Single (odd) field simulated as captured by a nonvibrated TDI sensor with N = 16. (b) Field degraded by two-dimensional vibrations, demonstrating the resulting nonuniform sampling. The distorted locations of the vertical lines result from the horizontal vibrations’ component, and the distorted shapes of the horizontal curves result from the vertical component.

Fig. 6
Fig. 6

Examples of two sections chosen from an FDV for construction of the initial camera displacement guesses: 1, linear region; 2, nonlinear region. Top, camera motion vector. Bottom, error.

Fig. 7
Fig. 7

Influence of region selection (shown in Fig. 6) as the basis for the construction of the camera-motion vector (N = 16 and margin = 3).

Fig. 8
Fig. 8

(a) True versus the estimated FDV in the horizontal (scan) direction; (b) error between the true and estimated vectors in (a); (c) and (d) same as (a) and (b) but for the vertical displacements.

Fig. 9
Fig. 9

(a) True versus the estimated camera displacement vector in the horizontal (scan) direction; (c) error between the true and estimated vectors in (a); (b), (d) same as (a) and (c) but for the vertical displacements.

Fig. 10
Fig. 10

(a) Odd field of Fig. 4(a) interpolated to the high-resolution dimensions for comparison; (b), (c) restored high-resolution images with the true and the estimated motion vectors, respectively.

Fig. 11
Fig. 11

Restoration of a real-degraded image; (a) image captured by a staggered TDI thermal-imaging system with 6 TDI stages and a margin of 29 pixels, (b) image restored by the POCS method.

Equations (22)

Equations on this page are rendered with MathJax. Learn more.

PSF=fx,yx, y,
Aipixels=2PSDiΔFreqi1/2Δαpix,
Xt=iAx,i sin2πfx,it+ϕx,i,
Yt=jAy,j sin2πfy,jt+ϕy,j,
xnominalpixel=Vnominalpixel/stsec,
XxnominaliAx,i sin2πfx,ixnominalVnominal+ϕx,i,
YxnominaljAy,j sin2πfy,jynominalVnominal+ϕy,j.
DXi=Xi-Xi+N+margin,
DYi=Yi-Yi+N+margin,
ErrDX, DY=x,yRfevenx, y-foddx, y+foddxx, yDX+foddyx, yDY2,
x,yRfoddx2x, yx,yRfoddxx, yx, yRfoddyx, yx,yRfoddxx, yx,yRfoddyx, yx,yRfoddy2x, yDXDY=-x,yRfevenx, y-foddx, yfoddxx, yx,yRfevenx, y-foddx, yfoddyx, y.
Xi+N+margin=Xi-DXi,
Yi+N+margin=Yi-DYi.
PSFtotal=PSFvibrations* PSFcamera.
Cs=i=1mCi,
fl+1=PmPm-1P1fl, l=0, 1, 2, .
Ckm1, m2=f˜n1, n2:|rkf˜m1, m2|δ0m1, m2, k,
rkf˜m1, m2=gkm1, m2-g˜km1, m2.
Pkm1, m2f˜ln1, n2=f˜ln1, n2 + rkf˜lm1, m2-δ0m1, m2, khPSF,kn1, n2; m1, m2o1o2hPSF,k2o1, o2; m1, m2;rkf˜lm1, m2>δ0m1, m2, k0;-δ0m1, m2, krkf˜lm1, m20m1, m2, k,rkf˜lm1, m2+δ0m1, m2, k hPSF,kn1, n2; m1, m2o1o2hPSF,k2o1, o2; m1, m2;rkf˜lm1, m2<-δ0m1, m2, k
CA=f˜n1, n2:0f˜n1, n21.
CF=f˜n1, n2:1D,n1f˜n1, n2=0, for ωn1 >ε,
el=12k=12gk-g˜kl21/2Δ,

Metrics