Abstract

Many imaging systems produce pictures by the superimposition of two fields of frames of interlaced sequences. Pictures obtained in this way, which are termed composite frames, are severely degraded if relative motion between the camera and the scene occurs. In the presence of motion the composite frame is affected by two types of distortion: the edge staircase effect that is due to the fact that objects appear at different positions in successive fields and motion blur that is due to scene motion during each field exposure. Motion-deinterlacing methods previously proposed to recover the staircase effect neglect motion blur. However, motion blur may be significant, especially in systems designed for low-intensity radiometric imaging that use long exposures or even in short-exposure systems that happen to be in moving vehicles such as tanks, planes, ships, etc. We introduce an algorithm for the restoration of the two types of distortion in a composite frame degraded by linear uniform motion.

© 1999 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. W. K. Pratt, Digital Image Processing (Wiley, New York, 1978).
  2. N. S. Kopeika, A System Engineering Approach to Imaging, (SPIE Optical Engineering, Bellingham, Wash., 1998), Chap. 14, pp. 411–440.
  3. B. Girod, “Motion compensation: visual aspects, accuracy, and fundamental limits,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 5, pp. 135–139.
  4. R. A. F. Belfor, R. L. Lagendijk, J. Biemond, “Subsampling of digital image sequences using motion information,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 9.
    [CrossRef]
  5. V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
    [CrossRef]
  6. R. Manduchi, G. M. Cortelazzo, “Spectral characteristics and motion-compensated restoration of composite frames,” IEEE Trans. Image Process. 4, 95–99 (1995).
    [CrossRef] [PubMed]
  7. L. Levi, Applied Optics (Wiley, New York, 1980), Vol. 2, Chap. 18, pp. 722–728.
  8. N. S. Kopeika, A System Engineering Approach to Imaging (SPIE Optical Engineering, Bellingham, Wash., 1998), Chap. 18, pp. 517–524.
  9. M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis and Machine Vision (Chapman & Hall, London, 1993), Chaps. 7 and 14.
    [CrossRef]
  10. O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibration. Part IV: real time numerical calculation of optical transfer functions and experimental verification,” Opt. Eng. 33, 566–578 (1994).
    [CrossRef]
  11. T. Reuter, “Standards conversion using motion compensation,” Signal Process. 16, 73–82 (1989).
    [CrossRef]
  12. S. Tubaro, F. Rocca, “Motion field estimators and their application to image interpolation,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 6, pp. 160–165.
  13. E. Dubois, “The sampling and reconstruction of time-varying imagery with application in video systems,” Proc. IEEE 73, 502–522 (1985).
    [CrossRef]
  14. A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments and its implementation in image restoration,” in Digital Image Recovery and Synthesis III, P. S. Idell, T. J. Schultz, eds., Proc. SPIE2827, 191–202 (1996).
    [CrossRef]
  15. O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
    [CrossRef]
  16. A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments,” J. Opt. Soc. Am. A 14, 388–396 (1997).
    [CrossRef]

1997

O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
[CrossRef]

A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments,” J. Opt. Soc. Am. A 14, 388–396 (1997).
[CrossRef]

1995

R. Manduchi, G. M. Cortelazzo, “Spectral characteristics and motion-compensated restoration of composite frames,” IEEE Trans. Image Process. 4, 95–99 (1995).
[CrossRef] [PubMed]

1994

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibration. Part IV: real time numerical calculation of optical transfer functions and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
[CrossRef]

1989

T. Reuter, “Standards conversion using motion compensation,” Signal Process. 16, 73–82 (1989).
[CrossRef]

1985

E. Dubois, “The sampling and reconstruction of time-varying imagery with application in video systems,” Proc. IEEE 73, 502–522 (1985).
[CrossRef]

Adar, Z.

O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
[CrossRef]

Belfor, R. A. F.

R. A. F. Belfor, R. L. Lagendijk, J. Biemond, “Subsampling of digital image sequences using motion information,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 9.
[CrossRef]

Biemond, J.

R. A. F. Belfor, R. L. Lagendijk, J. Biemond, “Subsampling of digital image sequences using motion information,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 9.
[CrossRef]

Boyle, R.

M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis and Machine Vision (Chapman & Hall, London, 1993), Chaps. 7 and 14.
[CrossRef]

Clatanoff, T.

V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
[CrossRef]

Cortelazzo, G. M.

R. Manduchi, G. M. Cortelazzo, “Spectral characteristics and motion-compensated restoration of composite frames,” IEEE Trans. Image Process. 4, 95–99 (1995).
[CrossRef] [PubMed]

Cotter, A.

O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
[CrossRef]

Dror, I.

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibration. Part IV: real time numerical calculation of optical transfer functions and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Dubois, E.

E. Dubois, “The sampling and reconstruction of time-varying imagery with application in video systems,” Proc. IEEE 73, 502–522 (1985).
[CrossRef]

Girod, B.

B. Girod, “Motion compensation: visual aspects, accuracy, and fundamental limits,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 5, pp. 135–139.

Gove, R.

V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
[CrossRef]

Hadar, O.

O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
[CrossRef]

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibration. Part IV: real time numerical calculation of optical transfer functions and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Hlavac, V.

M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis and Machine Vision (Chapman & Hall, London, 1993), Chaps. 7 and 14.
[CrossRef]

Kopeika, N. S.

A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments,” J. Opt. Soc. Am. A 14, 388–396 (1997).
[CrossRef]

O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
[CrossRef]

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibration. Part IV: real time numerical calculation of optical transfer functions and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments and its implementation in image restoration,” in Digital Image Recovery and Synthesis III, P. S. Idell, T. J. Schultz, eds., Proc. SPIE2827, 191–202 (1996).
[CrossRef]

N. S. Kopeika, A System Engineering Approach to Imaging (SPIE Optical Engineering, Bellingham, Wash., 1998), Chap. 18, pp. 517–524.

N. S. Kopeika, A System Engineering Approach to Imaging, (SPIE Optical Engineering, Bellingham, Wash., 1998), Chap. 14, pp. 411–440.

Lagendijk, R. L.

R. A. F. Belfor, R. L. Lagendijk, J. Biemond, “Subsampling of digital image sequences using motion information,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 9.
[CrossRef]

Levi, L.

L. Levi, Applied Optics (Wiley, New York, 1980), Vol. 2, Chap. 18, pp. 722–728.

Manduchi, R.

R. Manduchi, G. M. Cortelazzo, “Spectral characteristics and motion-compensated restoration of composite frames,” IEEE Trans. Image Process. 4, 95–99 (1995).
[CrossRef] [PubMed]

Markandey, V.

V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
[CrossRef]

Ohara, K.

V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
[CrossRef]

Pratt, W. K.

W. K. Pratt, Digital Image Processing (Wiley, New York, 1978).

Reuter, T.

T. Reuter, “Standards conversion using motion compensation,” Signal Process. 16, 73–82 (1989).
[CrossRef]

Rocca, F.

S. Tubaro, F. Rocca, “Motion field estimators and their application to image interpolation,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 6, pp. 160–165.

Sonka, M.

M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis and Machine Vision (Chapman & Hall, London, 1993), Chaps. 7 and 14.
[CrossRef]

Stern, A.

A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments,” J. Opt. Soc. Am. A 14, 388–396 (1997).
[CrossRef]

A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments and its implementation in image restoration,” in Digital Image Recovery and Synthesis III, P. S. Idell, T. J. Schultz, eds., Proc. SPIE2827, 191–202 (1996).
[CrossRef]

Tubaro, S.

S. Tubaro, F. Rocca, “Motion field estimators and their application to image interpolation,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 6, pp. 160–165.

IEEE Trans. Consumer Electron.

V. Markandey, T. Clatanoff, R. Gove, K. Ohara, “Motion adaptive deinterlacer for DMD (digital micromirror device) based digital television,” IEEE Trans. Consumer Electron. 40, 735–741 (1994).
[CrossRef]

IEEE Trans. Image Process.

R. Manduchi, G. M. Cortelazzo, “Spectral characteristics and motion-compensated restoration of composite frames,” IEEE Trans. Image Process. 4, 95–99 (1995).
[CrossRef] [PubMed]

J. Opt. Soc. Am. A

Opt. Eng.

O. Hadar, I. Dror, N. S. Kopeika, “Image resolution limits resulting from mechanical vibration. Part IV: real time numerical calculation of optical transfer functions and experimental verification,” Opt. Eng. 33, 566–578 (1994).
[CrossRef]

Opt. Laser Technol.

O. Hadar, Z. Adar, A. Cotter, N. S. Kopeika, “Restoration of images degraded by extreme mechanical vibrations,” Opt. Laser Technol. 29, 171–177 (1997).
[CrossRef]

Proc. IEEE

E. Dubois, “The sampling and reconstruction of time-varying imagery with application in video systems,” Proc. IEEE 73, 502–522 (1985).
[CrossRef]

Signal Process.

T. Reuter, “Standards conversion using motion compensation,” Signal Process. 16, 73–82 (1989).
[CrossRef]

Other

S. Tubaro, F. Rocca, “Motion field estimators and their application to image interpolation,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 6, pp. 160–165.

A. Stern, N. S. Kopeika, “Analytical method to calculate optical transfer function for image motion and vibration using moments and its implementation in image restoration,” in Digital Image Recovery and Synthesis III, P. S. Idell, T. J. Schultz, eds., Proc. SPIE2827, 191–202 (1996).
[CrossRef]

L. Levi, Applied Optics (Wiley, New York, 1980), Vol. 2, Chap. 18, pp. 722–728.

N. S. Kopeika, A System Engineering Approach to Imaging (SPIE Optical Engineering, Bellingham, Wash., 1998), Chap. 18, pp. 517–524.

M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis and Machine Vision (Chapman & Hall, London, 1993), Chaps. 7 and 14.
[CrossRef]

W. K. Pratt, Digital Image Processing (Wiley, New York, 1978).

N. S. Kopeika, A System Engineering Approach to Imaging, (SPIE Optical Engineering, Bellingham, Wash., 1998), Chap. 14, pp. 411–440.

B. Girod, “Motion compensation: visual aspects, accuracy, and fundamental limits,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 5, pp. 135–139.

R. A. F. Belfor, R. L. Lagendijk, J. Biemond, “Subsampling of digital image sequences using motion information,” in Motion Analysis and Image Sequence Processing, M. I. Sezan, R. L. Lagendijk, eds. (Kluwer, Boston, 1993), Chap. 9.
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

Composite frame distorted by horizontal motion. The object is a picture of a circle.

Fig. 2
Fig. 2

Motion-distorted composite-frame formation: (a) FOV and object location at the beginnings of exposure instants in two successive fields. A uniform horizontal motion with a speed of v = 8 pixels/frame (4 pixels/field) is assumed, yielding a displacement vector length of ‖s‖ = 4 pixels. (b) The odd and (c) the even fields. At an exposure time equal to 3/4 of the sampling time (t e /t s = 0.75) the blur extent is ‖b‖ = 3 pixels. (d) The distorted composite frame.

Fig. 3
Fig. 3

Restoration schematic algorithm demonstrated on the composite frame of Fig. 2.

Fig. 4
Fig. 4

Power spectrum of a typical composite image [of Fig. 9(a)] (the dense white region denotes a region of high spectral values). Note the low-pass components at the four corners of the elementary cell ν and the high-pass components at f y = 1/2 pixel.

Fig. 5
Fig. 5

Filter mask applicable to the composite-frame spectrum of non-band-limited signals.

Fig. 6
Fig. 6

Spatial function (15) appropriate to the composite frame of Fig. 8. The function has a maximum at point (s x , s y ) = (-5, -6) pixels.

Fig. 7
Fig. 7

(a) Block from a composite frame of a circle moved horizontally, (b) the frame after field realignment, (c) the estimated motion point-spread function, (d) the restored frame.

Fig. 8
Fig. 8

(a) Composite frame distorted by nonhorizontal motion, (b) the restored frame.

Fig. 9
Fig. 9

(a) Composite frame distorted by nonhorizontal motion and (b) the restored frame.

Fig. 10
Fig. 10

(a) Composite frame distorted by nonhorizontal motion and (b) the restored frame.

Equations (23)

Equations on this page are rendered with MathJax. Learn more.

ux, y, t=usx-vxt, y-vyt.
uclΔx, mΔy=u1lΔx, mΔyeven ml, mZ,u2lΔx, mΔyodd m
u1lΔx, mΔy=0te uslΔx, mΔy, tdt,u2lΔx, mΔy=0te uslΔx, mΔy, t+tsdt,
U1fx, fy=l,m=- Usfx-lΔx, fy-mΔysincπbxfx-lΔx+byfy-mΔyexp-jπbxfx-lΔx+byfy-mΔy,
U2fx, fy=l,m=- Usfx-lΔx, fy-mΔysincπbxfx-lΔx+byfy-mΔyexp-jπbxfx-lΔx+byfy-mΔyexp-jπsxfx-lΔx+syfy-mΔy,
Vl,mfx, fy; bx, by=Usfx-lΔx, fy-mΔysincπbxfx-lΔx+byfy-mΔy×exp-jπbxfx-lΔx+byfy-mΔy,
U1fx, fy=l,m=- VL,mfx, fy; bx, by,
U2fx, fy=l,m=- Vl,mfx, fy; bx, by×exp-j2πsxfx-lΔx+syfy-mΔy.
uclΔx, mΔy=u1lΔx, mΔy1+expjmπ2+u2lΔx, mΔy1-expjmπ2,
Ucfx, fy=12U1fx, fy+U1fx, fy-12Δy+U2fx, fy-U2fx, fy-12Δy=12l,m=- Vl,mfx, fy; bx, by×1+exp-j2πsxfx-lΔx+syfy-mΔy+12l,m=- Vl,mfx, fy-12Δy; bx, by×1-exp-j2πsxfx-lΔx+syfy-mΔy-12Δy.
Ucfx, fy+Ucfx, fy+12Δy=l,m=-Vl,mfx, fy+Vl,mfx, fy+12Δy,
Ucfx, fy-Ucfx, fy+12Δy=l,m=-exp-j2πsxfx-lΔx+syfy-mΔy×Vl,mfx, fy+exp-jπsy/2ΔyVl,m×fx, fy+12Δy.
|Usfx, fy|  Usfx, fy+12Δy,|fx|<12Δx, |fy|<12Δy,
|Vl,mfx, fy|  Vl,mfx, fy+12Δy,|fx|<12Δx, |fy|<12Δy.
F-1Ucfx, fy-Ucfx, fy+12ΔyUcfx, fy+Ucfx, fy+12ΔyF-1expj2πsxfx+syfy=δx-sx, y-sy,
Ucfx, fy+Ucfx, fy+12Δy=Usfx, fy+Usfx, fy+12Δy,
Ucfx, fy-Ucfx, fy+12Δy=exp-j2πsxfx-lΔx+syfy-mΔyUsfx, fy+Usfx, fy+12Δy,
Mfx, fy=H*fx, fy|Hfx, fy|2+γfx, fy,
Hfx, fy=sincπbxfx+byfy,
b=νte=s/tste=sxte/ts, syte/tsT,
Ucfx, fy=12l,m=-exp-jπsxfx-lΔx+syfy-mΔyVl,mfx, fy; bx, by×expjπsxfx-lΔx+syfy-mΔy+exp-jπsxfx-lΔx+syfy-mΔy+12l,m=-exp-jπsxfx-lΔx+syfy-mΔy-12ΔyVl,mfx, fy-12Δy; bx, by×expjπsxfx-lΔx+syfy-mΔy-12Δy-exp-jπsxfx-lΔx+syfy-mΔy-12Δy=l,m=-exp-jπsxfx-lΔx+syfy-mΔy×Vl,mfx, fycosπsxfx-lΔx+syfy-mΔy+j exp-jπ sy2ΔyVl,mfx, fy-12Δysinπsxfx-lΔx+syfy-lΔy-m2Δy.
Vl,mfx, fy; bx, by=Usfx-lΔx, fs-mΔy.
Ucfx, fy=l,m=-exp-jπsxfx-lΔx+syfy-mΔyUsfx, fycosπsxfx-lΔx+syfy-mΔy+j exp-jπ sy2ΔyUsfx, fy-12Δysinπsxfx-lΔx+syfy-mΔy-12Δy.

Metrics