Abstract

This paper presents a robust true motion estimation algorithm, designated as MPMVP (Multi-pass and Motion Vector Propagation), to enhance the accuracy of the motion vector fields in frame rate up-conversion applications. The MPMVP uses a multi-pass scheme to progressively refine approximate motion vectors to true motion vectors based upon the motion information acquired in previous passes. The multi-pass motion estimation process uses a large block size to detect the motion vectors within the objects themselves and small block sizes to detect the motion vectors along the object boundaries. Actually, the block size is progressively reduced during the search process. When the motion vector of a block is considered to be sufficiently accurate for motion estimation purposes, the block is said to be converged and the local motion vector search process terminates. A novel technique, referred to as motion vector propagation, is then applied to propagate the motion vector of the converged block to its neighboring blocks. This technique not only ensures that neighboring motion vectors within the same object have a high degree of spatial correlation, but also accelerates the convergence of the motion vectors in the neighboring blocks and therefore reduces the overall computational time and expense of the multi-pass motion vector search procedure. A novel distortion criterion is proposed to enhance the tolerance of the traditional sum-of-absolute-difference measurement technique applied in the motion estimation scheme to noise and shadow effects. The experimental results demonstrate that the proposed true motion estimation algorithm outperforms the traditional full search, 3DRS and TCSBP algorithms in terms of both the smoothness of the generated motion vector fields and the visual quality of the up-converted frames.

© 2008 IEEE

PDF Article

References

  • View by:
  • |
  • |

  1. B.-T. Choi, S.-H. Lee, S.-J. Ko, "New frame rate up-conversion using bi-directional motion estimation," IEEE Trans. Consumer Electron. 46, 603-609 (2000).
  2. J. Wang, D. Wang, W. Zhang, "Temporal compensated motion estimation with simple block-based prediction," IEEE Trans. Broadcast. 49, 241-248 (2003).
  3. H. Taehyeun, L. Seongjoo, K. Jaeseok, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer Electron. 50, 752-759 (2004).
  4. G. de Haan, P. W. A. C. Biezen, H. Huijgen, O. A. Ojo, "True-motion estimation with 3-D recursive search block matching," IEEE Trans. Circuits Syst. Video Technol. 3, 368-379 (1993).
  5. R. Braspenning, G. de Haan, "True-motion estimation using feature correspondence," SPIE, Proc. VCIP 396-407 (2004).
  6. G. de Haan, P. W. A. C. Biezen, "An efficient true-motion estimator using candidate vectors from a parametric motion model," IEEE Trans. Circuits Syst. Video Technol. 8, 85-91 (1998).
  7. A. Pelagotti, G. de Haan, "A new algorithm for high quality video format conversion," 2001 Proc. Int. Conf. on Image Process. (2001) pp. 375-378.
  8. S.-H. Lee, Y.-C. Shin, Y. Seungjoon, H.-H. Moon, R.-H. Park, "Adaptive motion-compensated interpolation for frame rate up-conversion," IEEE Trans. Consumer Electron. 48, 444-450 (2002).
  9. S.-H. Lee, S. Yang, "Adaptive motion-compensated frame rate up-conversion," Electron. Lett. 38, 451-452 (2002).
  10. B.-W. Jeon, G.-I. Lee, S.-H. Lee, R.-H. Park, "Coarse-to-fine frame interpolation for frame rate up-conversion using pyramid structure," IEEE Trans. Consumer Electron. 49, 499-508 (2003).
  11. D. Wang, D. Lauzon, "Hybrid algorithm for estimating true motion fields," Opt. Eng. 39, 2876-2881 (2000).
  12. K. Hilman, H.-W. Park, Y.-M. Kim, "Using motion compensated frame-rate conversion for the correction of 3:2 pulldown artifacts in video sequences," IEEE Trans. Circuits Syst. Video Technol. 10, 869-877 (2000).
  13. P. Ishwar, P. Moulin, "On spatial adaptation of motion-field smoothness in video coding," IEEE Trans. Circuits Syst. Video Technol. 10, 980-989 (2000).
  14. S. Fujiwara, A. Taguchi, "Motion-compensated frame rate up-conversion based on block matching algorithm with multi-size blocks," Proc. 2005 Int. Symp. on Intell. Signal Process. Commun. Syst. (2005) pp. 353-356.
  15. B.-D. Choi, J.-W. Han, C.-S. Kim, S.-J. Ko, "Frame rate up-conversion using perspective transform," IEEE Trans. Consumer Electron. 52, (2006).
  16. O. EePing, H. Wang, P. Xue, "Video coding based on true motion estimation," Proc. 2003 IEEE Int. Conf. on Acoust., Speech, and Signal Process., (2003) pp. 409-12.
  17. J. E. Santos Conde, A. Teuner, B. J. Hosticka, "Hierarchical locally adaptive multigrid motion estimation for surveillance applications," Proc. 1999 IEEE Int. Conf. on Acoust., Speech, Signal Process. (1999) pp. 3365-3368.
  18. G. de Haan, "IC for motion-compensated de-interlacing, noise reduction, and picture-rate conversion," IEEE Trans. Consumer Electron. 45, 617-624 (1999).
  19. J.-W. Suh, Y.-S. Ho, "Error concealment technique based on optical flow," Electron. Lett. 38, 1020-1021 (2002).
  20. J.-Y. Pyun, J.-S. Lee, J.-W. Jeong, J.-H. Jeong, S.-J. Ko, "Robust error concealment for visual communications in burst-packet-loss networks," IEEE Trans. Consumer Electron. 49, 1013-1019 (2003).

2006 (1)

B.-D. Choi, J.-W. Han, C.-S. Kim, S.-J. Ko, "Frame rate up-conversion using perspective transform," IEEE Trans. Consumer Electron. 52, (2006).

2004 (2)

H. Taehyeun, L. Seongjoo, K. Jaeseok, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer Electron. 50, 752-759 (2004).

R. Braspenning, G. de Haan, "True-motion estimation using feature correspondence," SPIE, Proc. VCIP 396-407 (2004).

2003 (3)

J. Wang, D. Wang, W. Zhang, "Temporal compensated motion estimation with simple block-based prediction," IEEE Trans. Broadcast. 49, 241-248 (2003).

J.-Y. Pyun, J.-S. Lee, J.-W. Jeong, J.-H. Jeong, S.-J. Ko, "Robust error concealment for visual communications in burst-packet-loss networks," IEEE Trans. Consumer Electron. 49, 1013-1019 (2003).

B.-W. Jeon, G.-I. Lee, S.-H. Lee, R.-H. Park, "Coarse-to-fine frame interpolation for frame rate up-conversion using pyramid structure," IEEE Trans. Consumer Electron. 49, 499-508 (2003).

2002 (3)

J.-W. Suh, Y.-S. Ho, "Error concealment technique based on optical flow," Electron. Lett. 38, 1020-1021 (2002).

S.-H. Lee, Y.-C. Shin, Y. Seungjoon, H.-H. Moon, R.-H. Park, "Adaptive motion-compensated interpolation for frame rate up-conversion," IEEE Trans. Consumer Electron. 48, 444-450 (2002).

S.-H. Lee, S. Yang, "Adaptive motion-compensated frame rate up-conversion," Electron. Lett. 38, 451-452 (2002).

2000 (4)

B.-T. Choi, S.-H. Lee, S.-J. Ko, "New frame rate up-conversion using bi-directional motion estimation," IEEE Trans. Consumer Electron. 46, 603-609 (2000).

D. Wang, D. Lauzon, "Hybrid algorithm for estimating true motion fields," Opt. Eng. 39, 2876-2881 (2000).

K. Hilman, H.-W. Park, Y.-M. Kim, "Using motion compensated frame-rate conversion for the correction of 3:2 pulldown artifacts in video sequences," IEEE Trans. Circuits Syst. Video Technol. 10, 869-877 (2000).

P. Ishwar, P. Moulin, "On spatial adaptation of motion-field smoothness in video coding," IEEE Trans. Circuits Syst. Video Technol. 10, 980-989 (2000).

1999 (1)

G. de Haan, "IC for motion-compensated de-interlacing, noise reduction, and picture-rate conversion," IEEE Trans. Consumer Electron. 45, 617-624 (1999).

1998 (1)

G. de Haan, P. W. A. C. Biezen, "An efficient true-motion estimator using candidate vectors from a parametric motion model," IEEE Trans. Circuits Syst. Video Technol. 8, 85-91 (1998).

1993 (1)

G. de Haan, P. W. A. C. Biezen, H. Huijgen, O. A. Ojo, "True-motion estimation with 3-D recursive search block matching," IEEE Trans. Circuits Syst. Video Technol. 3, 368-379 (1993).

Electron. Lett. (2)

S.-H. Lee, S. Yang, "Adaptive motion-compensated frame rate up-conversion," Electron. Lett. 38, 451-452 (2002).

J.-W. Suh, Y.-S. Ho, "Error concealment technique based on optical flow," Electron. Lett. 38, 1020-1021 (2002).

IEEE Trans. Broadcast. (1)

J. Wang, D. Wang, W. Zhang, "Temporal compensated motion estimation with simple block-based prediction," IEEE Trans. Broadcast. 49, 241-248 (2003).

IEEE Trans. Circuits Syst. Video Technol. (1)

K. Hilman, H.-W. Park, Y.-M. Kim, "Using motion compensated frame-rate conversion for the correction of 3:2 pulldown artifacts in video sequences," IEEE Trans. Circuits Syst. Video Technol. 10, 869-877 (2000).

IEEE Trans. Circuits Syst. Video Technol. (1)

G. de Haan, P. W. A. C. Biezen, "An efficient true-motion estimator using candidate vectors from a parametric motion model," IEEE Trans. Circuits Syst. Video Technol. 8, 85-91 (1998).

IEEE Trans. Circuits Syst. Video Technol. (2)

G. de Haan, P. W. A. C. Biezen, H. Huijgen, O. A. Ojo, "True-motion estimation with 3-D recursive search block matching," IEEE Trans. Circuits Syst. Video Technol. 3, 368-379 (1993).

P. Ishwar, P. Moulin, "On spatial adaptation of motion-field smoothness in video coding," IEEE Trans. Circuits Syst. Video Technol. 10, 980-989 (2000).

IEEE Trans. Consumer Electron. (3)

B.-W. Jeon, G.-I. Lee, S.-H. Lee, R.-H. Park, "Coarse-to-fine frame interpolation for frame rate up-conversion using pyramid structure," IEEE Trans. Consumer Electron. 49, 499-508 (2003).

H. Taehyeun, L. Seongjoo, K. Jaeseok, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer Electron. 50, 752-759 (2004).

G. de Haan, "IC for motion-compensated de-interlacing, noise reduction, and picture-rate conversion," IEEE Trans. Consumer Electron. 45, 617-624 (1999).

IEEE Trans. Consumer Electron. (4)

B.-T. Choi, S.-H. Lee, S.-J. Ko, "New frame rate up-conversion using bi-directional motion estimation," IEEE Trans. Consumer Electron. 46, 603-609 (2000).

S.-H. Lee, Y.-C. Shin, Y. Seungjoon, H.-H. Moon, R.-H. Park, "Adaptive motion-compensated interpolation for frame rate up-conversion," IEEE Trans. Consumer Electron. 48, 444-450 (2002).

J.-Y. Pyun, J.-S. Lee, J.-W. Jeong, J.-H. Jeong, S.-J. Ko, "Robust error concealment for visual communications in burst-packet-loss networks," IEEE Trans. Consumer Electron. 49, 1013-1019 (2003).

B.-D. Choi, J.-W. Han, C.-S. Kim, S.-J. Ko, "Frame rate up-conversion using perspective transform," IEEE Trans. Consumer Electron. 52, (2006).

Opt. Eng. (1)

D. Wang, D. Lauzon, "Hybrid algorithm for estimating true motion fields," Opt. Eng. 39, 2876-2881 (2000).

SPIE, Proc. VCIP (1)

R. Braspenning, G. de Haan, "True-motion estimation using feature correspondence," SPIE, Proc. VCIP 396-407 (2004).

Other (4)

A. Pelagotti, G. de Haan, "A new algorithm for high quality video format conversion," 2001 Proc. Int. Conf. on Image Process. (2001) pp. 375-378.

S. Fujiwara, A. Taguchi, "Motion-compensated frame rate up-conversion based on block matching algorithm with multi-size blocks," Proc. 2005 Int. Symp. on Intell. Signal Process. Commun. Syst. (2005) pp. 353-356.

O. EePing, H. Wang, P. Xue, "Video coding based on true motion estimation," Proc. 2003 IEEE Int. Conf. on Acoust., Speech, and Signal Process., (2003) pp. 409-12.

J. E. Santos Conde, A. Teuner, B. J. Hosticka, "Hierarchical locally adaptive multigrid motion estimation for surveillance applications," Proc. 1999 IEEE Int. Conf. on Acoust., Speech, Signal Process. (1999) pp. 3365-3368.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.