Abstract

A multiscale algorithm with complexity O(N) (where N is the number of pixels in one image) using wavelet filters is proposed to estimate dense optical flow from two frames. Hierarchical image representation by wavelet decomposition is integrated with differential techniques in a new multiscale framework. It is shown that if a compactly supported wavelet basis with one vanishing moment is carefully selected, hierarchical image, first-order derivative, and corner representations can be obtained from the wavelet decomposition. On the basis of this result, three of the four components of the wavelet decomposition are employed to estimate dense optical flow with use of only two frames. This overcomes the “flattening-out” problem in traditional pyramid methods, which produce large errors when low-texture regions become flat at coarse levels as a result of blurring. A two-dimensional affine motion model is used to formulate the optical flow problem as a linear system, with all resolutions simultaneously (i.e., coarse-and-fine) rather than the traditional coarse-to-fine approach, which unavoidably propagates errors from the coarse level. This not only helps to improve the accuracy but also makes the hardware implementation of our algorithm simple. Experiments on different types of image sequences, together with quantitative and qualitative comparisons with several other optical flow methods, are given to demonstrate the effectiveness and the robustness of our algorithm.

© 2003 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Gibson, The Senses Considered as Perceptual Systems (Houghton Mifflin, Boston, 1966).
  2. J. Oliensis, “A critique of structure-from-motion algorithms,” Comput. Vis. Image Underst. 80, 172–214 (2000).
    [CrossRef]
  3. G. J. Young, R. Chellappa, “3-D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness results,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 995–1013 (1992).
    [CrossRef]
  4. J. Duncan, T. Chou, “On the detection of motion and the computation of optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 346–352 (1992).
    [CrossRef]
  5. K. Prazdny, “Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer,” Comput. Graph. Image Process. 17, 82–87 (1981).
    [CrossRef]
  6. Y. Aloimonos, Z. Duric, “Estimating the heading direction using normal flow,” Int. J. Comput. Vision 13, 33–56 (1994).
    [CrossRef]
  7. M. Hashimoto, J. Sklansky, “Multiple-order derivatives for detecting local image characteristics,” Comput. Vision Image Understand. 39, 28–55 (1987).
    [CrossRef]
  8. W. Thompson, K. Mutch, V. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 374–383 (1985).
    [CrossRef] [PubMed]
  9. W.-S. Chou, Y.-C. Chen, “Estimation of the velocity field of two-dimensional deformable motion,” Pattern Recogn. 26, 351–364 (1993).
    [CrossRef]
  10. J. Gibson, “Optical motions and transformations as stimuli for visual perception,” Psychol. Rev. 64, 288–295 (1957).
    [CrossRef] [PubMed]
  11. S. Ullman, “The interpretation of structure from motion,” Proc. R. Soc. London Ser. B 203, 405–426 (1979).
    [CrossRef]
  12. J. L. Barron, D. J. Fleet, S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vision 12, 43–77 (1994).
    [CrossRef]
  13. P. J. Sobey, M. V. Srinivasan, “Measurement of complex optical flow with use of an augmented generalized gradient scheme,” J. Opt. Soc. Am. A 11, 2787–2798 (1994).
    [CrossRef]
  14. P. J. Sobey, M. G. Nagle, Y. V. Venkatesh, M. V. Srinivasan, “Measurement of optical flow by a generalized gradient scheme,” J. Opt. Soc. Am. A 8, 1488–1498 (1991).
    [CrossRef]
  15. A. G. Borş, I. Pitas, “Optical flow estimation and moving object segmentation based on median radial basis function network,” IEEE Trans. Image Process. 7, 693–702 (1998).
    [CrossRef]
  16. E. Meminémin, P. Pérez, “Dense estimation and object-based segmentation of the optical flow with robust techniques,” IEEE Trans. Image Process. 7, 703–719 (1998).
    [CrossRef]
  17. J. N. Pan, Y. Q. Shi, C. Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Trans. Image Process. 7, 1061–1067 (1998).
    [CrossRef]
  18. Y. T. Wu, T. Kanade, J. Cohn, C. C. Li, “Optical flow estimation using wavelet motion model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1998), pp. 992–998.
  19. M. Hemmendorff, M. T. Andersson, H. Knutsson, “Phase-based image motion estimation and registration,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 3345–3348.
  20. J. Magarey, N. Kingsbury, “Motion estimation using a complex-value wavelet transform,” IEEE Trans. Signal Process. 46, 1069–1084 (1998).
    [CrossRef]
  21. F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
    [CrossRef]
  22. C. P. Bernard, “Discrete wavelet analysis: a new framework for fast optical flow computation,” in Proceedings of the European Conference on Computer Vision (Springer, Freiburg, Germany, 1998), pp. 354–368.
  23. A. D. Bimbo, P. Nesi, J. L. C. Sanz, “Optical flow computation using extended constraints,” IEEE Trans. Image Process. 5, 720–739 (1996).
    [CrossRef] [PubMed]
  24. S. Ghosal, P. Vaněk, “A fast scalable algorithm for discontinuous optical flow estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 181–194 (1996).
    [CrossRef]
  25. H. W. Haussecker, D. J. Fleet, “Computing optical flow with physical models of brightness variation,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 661–673 (2001).
    [CrossRef]
  26. A. Kumar, A. R. Tannenbaum, G. J. Balas, “Optical flow: a curve evolution approach,” IEEE Trans. Image Process. 5, 598–610 (1996).
    [CrossRef] [PubMed]
  27. H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
    [CrossRef]
  28. S. Rakshit, C. H. Anderson, “Computation of optical flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997).
    [CrossRef] [PubMed]
  29. H. Spies, H. Scharr, “Acurate optical flow in noisy image sequences,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 2001), pp. 587–592.
  30. S. Srinivasan, R. Chellappa, “Noise-resilient estimation of optical flow by use of overlapped basis functions,” J. Opt. Soc. Am. A 16, 493–507 (1999).
    [CrossRef]
  31. M. Tistarelli, “Multiple constraints to compute optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1243–1250 (1996).
    [CrossRef]
  32. M. Yeasin, “Optical flow in log-mapped image plane—a new approach,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 125–131 (2002).
    [CrossRef]
  33. J. Weber, J. Malik, “Robust computation of optical flow in a multi-scale differential framework,” Int. J. Comput. Vision 14, 67–81 (1995).
    [CrossRef]
  34. J. Mendelsohn, E. Simoncelli, R. Bajcsy, “Discrete-time rigidity-constrained optical flow,” in Proceedings of the International Conference on Computer Analysis of Images and Patterns (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 255–262.
  35. H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
    [CrossRef]
  36. H. Haussecker, H. Spies, Handbook of Computer Vision and Applications Volume 2: Signal Processing and Pattern Recognition (New York, Academic Press, 1999), chap. 13: Motion, pp. 309–396.
  37. I. Daubechies, Ten Lectures on Wavelets (Society for Industrial and Applied Mathematics, Philadelphia, Pa., 1992).
  38. S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 674–693 (1989).
    [CrossRef]
  39. G. H. Golub, C. F. V. Loan, Matrix Computations (Johns Hopkins U. Press, Baltimore, Md., 1996).
  40. D. J. Fleet, A. D. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vision 5, 77–104 (1990).
    [CrossRef]

2002

M. Yeasin, “Optical flow in log-mapped image plane—a new approach,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 125–131 (2002).
[CrossRef]

2001

H. W. Haussecker, D. J. Fleet, “Computing optical flow with physical models of brightness variation,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 661–673 (2001).
[CrossRef]

2000

J. Oliensis, “A critique of structure-from-motion algorithms,” Comput. Vis. Image Underst. 80, 172–214 (2000).
[CrossRef]

F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
[CrossRef]

1999

1998

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

A. G. Borş, I. Pitas, “Optical flow estimation and moving object segmentation based on median radial basis function network,” IEEE Trans. Image Process. 7, 693–702 (1998).
[CrossRef]

E. Meminémin, P. Pérez, “Dense estimation and object-based segmentation of the optical flow with robust techniques,” IEEE Trans. Image Process. 7, 703–719 (1998).
[CrossRef]

J. N. Pan, Y. Q. Shi, C. Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Trans. Image Process. 7, 1061–1067 (1998).
[CrossRef]

J. Magarey, N. Kingsbury, “Motion estimation using a complex-value wavelet transform,” IEEE Trans. Signal Process. 46, 1069–1084 (1998).
[CrossRef]

1997

H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
[CrossRef]

S. Rakshit, C. H. Anderson, “Computation of optical flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997).
[CrossRef] [PubMed]

1996

A. Kumar, A. R. Tannenbaum, G. J. Balas, “Optical flow: a curve evolution approach,” IEEE Trans. Image Process. 5, 598–610 (1996).
[CrossRef] [PubMed]

M. Tistarelli, “Multiple constraints to compute optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1243–1250 (1996).
[CrossRef]

A. D. Bimbo, P. Nesi, J. L. C. Sanz, “Optical flow computation using extended constraints,” IEEE Trans. Image Process. 5, 720–739 (1996).
[CrossRef] [PubMed]

S. Ghosal, P. Vaněk, “A fast scalable algorithm for discontinuous optical flow estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 181–194 (1996).
[CrossRef]

1995

J. Weber, J. Malik, “Robust computation of optical flow in a multi-scale differential framework,” Int. J. Comput. Vision 14, 67–81 (1995).
[CrossRef]

1994

J. L. Barron, D. J. Fleet, S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vision 12, 43–77 (1994).
[CrossRef]

P. J. Sobey, M. V. Srinivasan, “Measurement of complex optical flow with use of an augmented generalized gradient scheme,” J. Opt. Soc. Am. A 11, 2787–2798 (1994).
[CrossRef]

Y. Aloimonos, Z. Duric, “Estimating the heading direction using normal flow,” Int. J. Comput. Vision 13, 33–56 (1994).
[CrossRef]

1993

W.-S. Chou, Y.-C. Chen, “Estimation of the velocity field of two-dimensional deformable motion,” Pattern Recogn. 26, 351–364 (1993).
[CrossRef]

1992

G. J. Young, R. Chellappa, “3-D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness results,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 995–1013 (1992).
[CrossRef]

J. Duncan, T. Chou, “On the detection of motion and the computation of optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 346–352 (1992).
[CrossRef]

1991

1990

D. J. Fleet, A. D. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vision 5, 77–104 (1990).
[CrossRef]

1989

S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 674–693 (1989).
[CrossRef]

1987

M. Hashimoto, J. Sklansky, “Multiple-order derivatives for detecting local image characteristics,” Comput. Vision Image Understand. 39, 28–55 (1987).
[CrossRef]

1985

W. Thompson, K. Mutch, V. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 374–383 (1985).
[CrossRef] [PubMed]

1981

K. Prazdny, “Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer,” Comput. Graph. Image Process. 17, 82–87 (1981).
[CrossRef]

1979

S. Ullman, “The interpretation of structure from motion,” Proc. R. Soc. London Ser. B 203, 405–426 (1979).
[CrossRef]

1957

J. Gibson, “Optical motions and transformations as stimuli for visual perception,” Psychol. Rev. 64, 288–295 (1957).
[CrossRef] [PubMed]

Aloimonos, Y.

Y. Aloimonos, Z. Duric, “Estimating the heading direction using normal flow,” Int. J. Comput. Vision 13, 33–56 (1994).
[CrossRef]

Anderson, C. H.

S. Rakshit, C. H. Anderson, “Computation of optical flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997).
[CrossRef] [PubMed]

Andersson, M. T.

M. Hemmendorff, M. T. Andersson, H. Knutsson, “Phase-based image motion estimation and registration,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 3345–3348.

Bajcsy, R.

J. Mendelsohn, E. Simoncelli, R. Bajcsy, “Discrete-time rigidity-constrained optical flow,” in Proceedings of the International Conference on Computer Analysis of Images and Patterns (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 255–262.

Balas, G. J.

A. Kumar, A. R. Tannenbaum, G. J. Balas, “Optical flow: a curve evolution approach,” IEEE Trans. Image Process. 5, 598–610 (1996).
[CrossRef] [PubMed]

Barron, J. L.

J. L. Barron, D. J. Fleet, S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vision 12, 43–77 (1994).
[CrossRef]

Beauchemin, S. S.

J. L. Barron, D. J. Fleet, S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vision 12, 43–77 (1994).
[CrossRef]

Bernard, C. P.

C. P. Bernard, “Discrete wavelet analysis: a new framework for fast optical flow computation,” in Proceedings of the European Conference on Computer Vision (Springer, Freiburg, Germany, 1998), pp. 354–368.

Berzins, V.

W. Thompson, K. Mutch, V. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 374–383 (1985).
[CrossRef] [PubMed]

Bimbo, A. D.

A. D. Bimbo, P. Nesi, J. L. C. Sanz, “Optical flow computation using extended constraints,” IEEE Trans. Image Process. 5, 720–739 (1996).
[CrossRef] [PubMed]

Bors, A. G.

A. G. Borş, I. Pitas, “Optical flow estimation and moving object segmentation based on median radial basis function network,” IEEE Trans. Image Process. 7, 693–702 (1998).
[CrossRef]

Camus, T.

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

Chellapa, R.

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

Chellappa, R.

S. Srinivasan, R. Chellappa, “Noise-resilient estimation of optical flow by use of overlapped basis functions,” J. Opt. Soc. Am. A 16, 493–507 (1999).
[CrossRef]

H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
[CrossRef]

G. J. Young, R. Chellappa, “3-D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness results,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 995–1013 (1992).
[CrossRef]

Chen, Y.-C.

W.-S. Chou, Y.-C. Chen, “Estimation of the velocity field of two-dimensional deformable motion,” Pattern Recogn. 26, 351–364 (1993).
[CrossRef]

Chou, T.

J. Duncan, T. Chou, “On the detection of motion and the computation of optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 346–352 (1992).
[CrossRef]

Chou, W.-S.

W.-S. Chou, Y.-C. Chen, “Estimation of the velocity field of two-dimensional deformable motion,” Pattern Recogn. 26, 351–364 (1993).
[CrossRef]

Cohn, J.

Y. T. Wu, T. Kanade, J. Cohn, C. C. Li, “Optical flow estimation using wavelet motion model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1998), pp. 992–998.

Daubechies, I.

I. Daubechies, Ten Lectures on Wavelets (Society for Industrial and Applied Mathematics, Philadelphia, Pa., 1992).

Duncan, J.

J. Duncan, T. Chou, “On the detection of motion and the computation of optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 346–352 (1992).
[CrossRef]

Duric, Z.

Y. Aloimonos, Z. Duric, “Estimating the heading direction using normal flow,” Int. J. Comput. Vision 13, 33–56 (1994).
[CrossRef]

Fleet, D. J.

H. W. Haussecker, D. J. Fleet, “Computing optical flow with physical models of brightness variation,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 661–673 (2001).
[CrossRef]

J. L. Barron, D. J. Fleet, S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vision 12, 43–77 (1994).
[CrossRef]

D. J. Fleet, A. D. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vision 5, 77–104 (1990).
[CrossRef]

Ghosal, S.

S. Ghosal, P. Vaněk, “A fast scalable algorithm for discontinuous optical flow estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 181–194 (1996).
[CrossRef]

Gibson, J.

J. Gibson, “Optical motions and transformations as stimuli for visual perception,” Psychol. Rev. 64, 288–295 (1957).
[CrossRef] [PubMed]

J. Gibson, The Senses Considered as Perceptual Systems (Houghton Mifflin, Boston, 1966).

Golub, G. H.

G. H. Golub, C. F. V. Loan, Matrix Computations (Johns Hopkins U. Press, Baltimore, Md., 1996).

Hashimoto, M.

M. Hashimoto, J. Sklansky, “Multiple-order derivatives for detecting local image characteristics,” Comput. Vision Image Understand. 39, 28–55 (1987).
[CrossRef]

Haussecker, H.

H. Haussecker, H. Spies, Handbook of Computer Vision and Applications Volume 2: Signal Processing and Pattern Recognition (New York, Academic Press, 1999), chap. 13: Motion, pp. 309–396.

Haussecker, H. W.

H. W. Haussecker, D. J. Fleet, “Computing optical flow with physical models of brightness variation,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 661–673 (2001).
[CrossRef]

Hemmendorff, M.

M. Hemmendorff, M. T. Andersson, H. Knutsson, “Phase-based image motion estimation and registration,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 3345–3348.

Herman, M.

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
[CrossRef]

Hong, T.

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

Hong, T. H.

H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
[CrossRef]

Jepson, A. D.

D. J. Fleet, A. D. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vision 5, 77–104 (1990).
[CrossRef]

Kanade, T.

Y. T. Wu, T. Kanade, J. Cohn, C. C. Li, “Optical flow estimation using wavelet motion model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1998), pp. 992–998.

Kingsbury, N.

J. Magarey, N. Kingsbury, “Motion estimation using a complex-value wavelet transform,” IEEE Trans. Signal Process. 46, 1069–1084 (1998).
[CrossRef]

Knutsson, H.

M. Hemmendorff, M. T. Andersson, H. Knutsson, “Phase-based image motion estimation and registration,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 3345–3348.

Kumar, A.

A. Kumar, A. R. Tannenbaum, G. J. Balas, “Optical flow: a curve evolution approach,” IEEE Trans. Image Process. 5, 598–610 (1996).
[CrossRef] [PubMed]

Leduc, J. P.

F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
[CrossRef]

Li, C. C.

Y. T. Wu, T. Kanade, J. Cohn, C. C. Li, “Optical flow estimation using wavelet motion model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1998), pp. 992–998.

Liu, H.

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
[CrossRef]

Loan, C. F. V.

G. H. Golub, C. F. V. Loan, Matrix Computations (Johns Hopkins U. Press, Baltimore, Md., 1996).

Magarey, J.

J. Magarey, N. Kingsbury, “Motion estimation using a complex-value wavelet transform,” IEEE Trans. Signal Process. 46, 1069–1084 (1998).
[CrossRef]

Malik, J.

J. Weber, J. Malik, “Robust computation of optical flow in a multi-scale differential framework,” Int. J. Comput. Vision 14, 67–81 (1995).
[CrossRef]

Mallat, S. G.

S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 674–693 (1989).
[CrossRef]

Meminémin, E.

E. Meminémin, P. Pérez, “Dense estimation and object-based segmentation of the optical flow with robust techniques,” IEEE Trans. Image Process. 7, 703–719 (1998).
[CrossRef]

Mendelsohn, J.

J. Mendelsohn, E. Simoncelli, R. Bajcsy, “Discrete-time rigidity-constrained optical flow,” in Proceedings of the International Conference on Computer Analysis of Images and Patterns (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 255–262.

Mujica, F. A.

F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
[CrossRef]

Murenzi, R.

F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
[CrossRef]

Mutch, K.

W. Thompson, K. Mutch, V. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 374–383 (1985).
[CrossRef] [PubMed]

Nagle, M. G.

Nesi, P.

A. D. Bimbo, P. Nesi, J. L. C. Sanz, “Optical flow computation using extended constraints,” IEEE Trans. Image Process. 5, 720–739 (1996).
[CrossRef] [PubMed]

Oliensis, J.

J. Oliensis, “A critique of structure-from-motion algorithms,” Comput. Vis. Image Underst. 80, 172–214 (2000).
[CrossRef]

Pan, J. N.

J. N. Pan, Y. Q. Shi, C. Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Trans. Image Process. 7, 1061–1067 (1998).
[CrossRef]

Pérez, P.

E. Meminémin, P. Pérez, “Dense estimation and object-based segmentation of the optical flow with robust techniques,” IEEE Trans. Image Process. 7, 703–719 (1998).
[CrossRef]

Pitas, I.

A. G. Borş, I. Pitas, “Optical flow estimation and moving object segmentation based on median radial basis function network,” IEEE Trans. Image Process. 7, 693–702 (1998).
[CrossRef]

Prazdny, K.

K. Prazdny, “Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer,” Comput. Graph. Image Process. 17, 82–87 (1981).
[CrossRef]

Rakshit, S.

S. Rakshit, C. H. Anderson, “Computation of optical flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997).
[CrossRef] [PubMed]

Sanz, J. L. C.

A. D. Bimbo, P. Nesi, J. L. C. Sanz, “Optical flow computation using extended constraints,” IEEE Trans. Image Process. 5, 720–739 (1996).
[CrossRef] [PubMed]

Scharr, H.

H. Spies, H. Scharr, “Acurate optical flow in noisy image sequences,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 2001), pp. 587–592.

Shi, Y. Q.

J. N. Pan, Y. Q. Shi, C. Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Trans. Image Process. 7, 1061–1067 (1998).
[CrossRef]

Shu, C. Q.

J. N. Pan, Y. Q. Shi, C. Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Trans. Image Process. 7, 1061–1067 (1998).
[CrossRef]

Simoncelli, E.

J. Mendelsohn, E. Simoncelli, R. Bajcsy, “Discrete-time rigidity-constrained optical flow,” in Proceedings of the International Conference on Computer Analysis of Images and Patterns (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 255–262.

Sklansky, J.

M. Hashimoto, J. Sklansky, “Multiple-order derivatives for detecting local image characteristics,” Comput. Vision Image Understand. 39, 28–55 (1987).
[CrossRef]

Smith, M. J. T.

F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
[CrossRef]

Sobey, P. J.

Spies, H.

H. Spies, H. Scharr, “Acurate optical flow in noisy image sequences,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 2001), pp. 587–592.

H. Haussecker, H. Spies, Handbook of Computer Vision and Applications Volume 2: Signal Processing and Pattern Recognition (New York, Academic Press, 1999), chap. 13: Motion, pp. 309–396.

Srinivasan, M. V.

Srinivasan, S.

Tannenbaum, A. R.

A. Kumar, A. R. Tannenbaum, G. J. Balas, “Optical flow: a curve evolution approach,” IEEE Trans. Image Process. 5, 598–610 (1996).
[CrossRef] [PubMed]

Thompson, W.

W. Thompson, K. Mutch, V. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 374–383 (1985).
[CrossRef] [PubMed]

Tistarelli, M.

M. Tistarelli, “Multiple constraints to compute optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1243–1250 (1996).
[CrossRef]

Ullman, S.

S. Ullman, “The interpretation of structure from motion,” Proc. R. Soc. London Ser. B 203, 405–426 (1979).
[CrossRef]

Vanek, P.

S. Ghosal, P. Vaněk, “A fast scalable algorithm for discontinuous optical flow estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 181–194 (1996).
[CrossRef]

Venkatesh, Y. V.

Weber, J.

J. Weber, J. Malik, “Robust computation of optical flow in a multi-scale differential framework,” Int. J. Comput. Vision 14, 67–81 (1995).
[CrossRef]

Wu, Y. T.

Y. T. Wu, T. Kanade, J. Cohn, C. C. Li, “Optical flow estimation using wavelet motion model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1998), pp. 992–998.

Yeasin, M.

M. Yeasin, “Optical flow in log-mapped image plane—a new approach,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 125–131 (2002).
[CrossRef]

Young, G. J.

G. J. Young, R. Chellappa, “3-D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness results,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 995–1013 (1992).
[CrossRef]

Comput. Graph. Image Process.

K. Prazdny, “Determining the instantaneous direction of motion from optical flow generated by a curvilinearly moving observer,” Comput. Graph. Image Process. 17, 82–87 (1981).
[CrossRef]

Comput. Vis. Image Underst.

J. Oliensis, “A critique of structure-from-motion algorithms,” Comput. Vis. Image Underst. 80, 172–214 (2000).
[CrossRef]

Comput. Vision Image Understand.

M. Hashimoto, J. Sklansky, “Multiple-order derivatives for detecting local image characteristics,” Comput. Vision Image Understand. 39, 28–55 (1987).
[CrossRef]

H. Liu, T. Hong, M. Herman, T. Camus, R. Chellapa, “Accuracy vs. efficiency trade-off in optical flow algorithms,” Comput. Vision Image Understand. 72, 271–286 (1998).
[CrossRef]

IEEE Trans. Image Process.

F. A. Mujica, J. P. Leduc, R. Murenzi, M. J. T. Smith, “A new motion parameter estimation algorithm based on the continuous wavelet transform,” IEEE Trans. Image Process. 9, 873–888 (2000).
[CrossRef]

A. D. Bimbo, P. Nesi, J. L. C. Sanz, “Optical flow computation using extended constraints,” IEEE Trans. Image Process. 5, 720–739 (1996).
[CrossRef] [PubMed]

A. Kumar, A. R. Tannenbaum, G. J. Balas, “Optical flow: a curve evolution approach,” IEEE Trans. Image Process. 5, 598–610 (1996).
[CrossRef] [PubMed]

S. Rakshit, C. H. Anderson, “Computation of optical flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997).
[CrossRef] [PubMed]

A. G. Borş, I. Pitas, “Optical flow estimation and moving object segmentation based on median radial basis function network,” IEEE Trans. Image Process. 7, 693–702 (1998).
[CrossRef]

E. Meminémin, P. Pérez, “Dense estimation and object-based segmentation of the optical flow with robust techniques,” IEEE Trans. Image Process. 7, 703–719 (1998).
[CrossRef]

J. N. Pan, Y. Q. Shi, C. Q. Shu, “Correlation-feedback technique in optical flow determination,” IEEE Trans. Image Process. 7, 1061–1067 (1998).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

W. Thompson, K. Mutch, V. Berzins, “Dynamic occlusion analysis in optical flow fields,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 374–383 (1985).
[CrossRef] [PubMed]

G. J. Young, R. Chellappa, “3-D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness results,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 995–1013 (1992).
[CrossRef]

J. Duncan, T. Chou, “On the detection of motion and the computation of optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 346–352 (1992).
[CrossRef]

S. Ghosal, P. Vaněk, “A fast scalable algorithm for discontinuous optical flow estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 181–194 (1996).
[CrossRef]

H. W. Haussecker, D. J. Fleet, “Computing optical flow with physical models of brightness variation,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 661–673 (2001).
[CrossRef]

M. Tistarelli, “Multiple constraints to compute optical flow,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1243–1250 (1996).
[CrossRef]

M. Yeasin, “Optical flow in log-mapped image plane—a new approach,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 125–131 (2002).
[CrossRef]

S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 674–693 (1989).
[CrossRef]

IEEE Trans. Signal Process.

J. Magarey, N. Kingsbury, “Motion estimation using a complex-value wavelet transform,” IEEE Trans. Signal Process. 46, 1069–1084 (1998).
[CrossRef]

Int. J. Comput. Vis.

H. Liu, T. H. Hong, M. Herman, R. Chellappa, “A general motion model and spatio-temporal filters for computing optical flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
[CrossRef]

Int. J. Comput. Vision

J. Weber, J. Malik, “Robust computation of optical flow in a multi-scale differential framework,” Int. J. Comput. Vision 14, 67–81 (1995).
[CrossRef]

Y. Aloimonos, Z. Duric, “Estimating the heading direction using normal flow,” Int. J. Comput. Vision 13, 33–56 (1994).
[CrossRef]

J. L. Barron, D. J. Fleet, S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vision 12, 43–77 (1994).
[CrossRef]

D. J. Fleet, A. D. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vision 5, 77–104 (1990).
[CrossRef]

J. Opt. Soc. Am. A

Pattern Recogn.

W.-S. Chou, Y.-C. Chen, “Estimation of the velocity field of two-dimensional deformable motion,” Pattern Recogn. 26, 351–364 (1993).
[CrossRef]

Proc. R. Soc. London Ser. B

S. Ullman, “The interpretation of structure from motion,” Proc. R. Soc. London Ser. B 203, 405–426 (1979).
[CrossRef]

Psychol. Rev.

J. Gibson, “Optical motions and transformations as stimuli for visual perception,” Psychol. Rev. 64, 288–295 (1957).
[CrossRef] [PubMed]

Other

J. Gibson, The Senses Considered as Perceptual Systems (Houghton Mifflin, Boston, 1966).

Y. T. Wu, T. Kanade, J. Cohn, C. C. Li, “Optical flow estimation using wavelet motion model,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1998), pp. 992–998.

M. Hemmendorff, M. T. Andersson, H. Knutsson, “Phase-based image motion estimation and registration,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1999), pp. 3345–3348.

H. Haussecker, H. Spies, Handbook of Computer Vision and Applications Volume 2: Signal Processing and Pattern Recognition (New York, Academic Press, 1999), chap. 13: Motion, pp. 309–396.

I. Daubechies, Ten Lectures on Wavelets (Society for Industrial and Applied Mathematics, Philadelphia, Pa., 1992).

J. Mendelsohn, E. Simoncelli, R. Bajcsy, “Discrete-time rigidity-constrained optical flow,” in Proceedings of the International Conference on Computer Analysis of Images and Patterns (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 255–262.

H. Spies, H. Scharr, “Acurate optical flow in noisy image sequences,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 2001), pp. 587–592.

C. P. Bernard, “Discrete wavelet analysis: a new framework for fast optical flow computation,” in Proceedings of the European Conference on Computer Vision (Springer, Freiburg, Germany, 1998), pp. 354–368.

G. H. Golub, C. F. V. Loan, Matrix Computations (Johns Hopkins U. Press, Baltimore, Md., 1996).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

Multiscale strategy: (a) one frame of the Yosemite sequence, (b) traditional Gaussian smoothing (σ=1.5) and dyadic downsampling at level 2, (c) wavelet decomposition at level 2 using biorthogonal wavelets (bior1.3).

Fig. 2
Fig. 2

Wavelet decomposition: a, approximation channel; h, v, horizontal and vertical channels; d, diagonal channel.

Fig. 3
Fig. 3

Wavelet filter unit, where Lo_Dx(y) is the decomposition low-pass filter and Hi_Dx(y) is the decomposition high-pass filter. See the text for details.

Fig. 4
Fig. 4

Hierarchical coarse-and-fine gradient constraint.

Fig. 5
Fig. 5

Hierarchical coarse-and-fine optical flow estimation.  

Fig. 6
Fig. 6

Synthetic image sequence: row 1, one frame from each sequence; row 2, ground-truth optical flow; row 3, optical flow estimated by proposed method; row 4, error distribution.

Fig. 7
Fig. 7

Real sequences: (1) SRI Trees, (2) NASA, (3) Rubik Cube, (4) Hamburg Taxi.

Tables (4)

Tables Icon

Table 1 Algorithm Efficiency with bior1.3 (in Flops per Pixel)

Tables Icon

Table 2 Translating Tree Results (Frame 20)

Tables Icon

Table 3 Diverging Tree Results (Frame 20)

Tables Icon

Table 4 Yosemite Results (Frame 9)

Equations (37)

Equations on this page are rendered with MathJax. Learn more.

I(x, t)=I(x-v, 0),
(Ix(x, t), Iy(x, t))  v+It(x, t)=0,
u=p1x+p2y+p3,v=p4x+p5y+p6.
(Twavf)(a, b)=f, ψa,b=1|a|-+f(x)ψx-ba¯dx,
Cψ=-+|ψˆ(ω)|2|ω|dω<.
-+ψ(x)dx=0.
limt-ha0(t)=0,limt+ha0(t)=-+ψxadx=0.
ha1(t)=-tha0(x)dx=xha0(x)|-t--txddx ha0(x)dx.
limt-ha1(t)=0,
limt+ha1(t)=--+xψxadx=-a2-+xψ(x)dx.
-+han-1(x)dx=(-1)nan+1n!-+xnψ(x)dx0.
h(t)=c-t-t1-tn-2ψ(x)dxdtn-2dt1dt,
-+h(x)dx=1.
limt-h(t)=limt+h(t)=0.
(Twavf)(a, b)=f, ψa,b=1|a|-+f(x)ψx-badx=(-1)nc|a|-+f(x)dndxn hx-badx=(-1)nc|a|dndxn-+f(x)hx-badx.
ψa(x, y)=ϕ(x)ϕ(y),ψh(x, y)=ϕ(x)ψ(y),
ψv(x, y)=ψ(x)ϕ(y),ψd(x, y)=ψ(x)ψ(y),
u=f(X, Y)p,
f(X, Y)=xiyi1000000xiyi1T,
Ix  f(X, Y)p=-It,
Ix=Ix1Iy10000000000IxMIyM,
It=(It1, It2,, ItM)T.
Alp=-Itl,
Al=Ixl  f(Xl, Yl).
Xl=D(W  Xl-1),X0=X,
Yl=D(W  Yl-1),Y0=Y.
Ap=b,
A=[A0, A1,, AL]T,b=-(It0, It1,, ItL)T.
FAST_OF_ESTIMATION(I1, I2)
I1:Imageframet.
I2:Imageframet+1.
L=[log2 Vmax],
flops=4D(2-12L-1)waveletdecomposition+4constructmatrixA+3×4L+1+15solvelinearequations+8computeaffineopticalflow,
flow=216 (-11881-1),
fhigh=22 (00-1100).
12520 (-225-150600-210002100-600150-252).
error=arccosvcvc2  veve2.

Metrics