Abstract

A nonlinear correlation algorithm is proposed for estimating the motion of objects from an image pair. This algorithm requires no a priori information on the number, size, or shape of the moving objects and does not require feature extraction or segmentation of either image. The algorithm directly yields information on the number of moving objects, the motion of the objects, and the size of the objects. Additional processing can be performed to yield the centroid of the objects in either frame. The utility of the resulting algorithm is demonstrated by application to a pair of example image sequences.

© 2001 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. G. Schunk, “The image flow constraint equation,” Comput. Vision Graph. Image Process. 35, 20–46 (1986).
    [CrossRef]
  2. B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
    [CrossRef]
  3. B. G. Schunk, “Image flow segmentation and estimation by constraint line clustering,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 1010–1024 (1989).
    [CrossRef]
  4. M. R. Luettgen, W. C. Karl, A. S. Willsky, “Efficient multiscale regularization with applications to the computation of optical flow,” IEEE Trans. Image Process. 3, 41–63 (1994).
    [CrossRef] [PubMed]
  5. R. Jain, W. N. Martin, J. K. Aggarwal, “Segmentation through the detection of changes due to motion,” Comput. Graph. Image Process. 11, 13–34 (1979).
    [CrossRef]
  6. J. K. Aggarwal, R. O. Duda, “Computer analysis of moving polygonal images,” IEEE Trans. Comput. 24, 966–976 (1975).
    [CrossRef]
  7. J. W. Roach, J. K. Aggarwal, “Determining the movement of objects from a sequence of images,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 554–562 (1980).
    [CrossRef]
  8. R. Y. Tsai, T. S. Huang, “Estimating 3-D motion parameters of a rigid planar patch. I,” IEEE Trans. Acoust. Speech Signal Process. 29, 1147–1152 (1981).
    [CrossRef]
  9. B. L. Yen, T. S. Huang, “Determining 3-D motion and structure of a rigid body using the spherical projection,” Comput. Vision Graph. Image Process. 21, 21–32 (1983).
    [CrossRef]
  10. R. Jain, H.-H. Nagel, “On the analysis of accumulative difference pictures from image sequences of real world scenes,” IEEE Trans. Pattern Analy. Mach. Intell. 1, 206–214 (1979).
    [CrossRef]
  11. J. B. Burl, “A reduced order extended Kalman filter for sequential images containing a moving object,” IEEE Trans. Image Process. 2, 285–385 (1993).
    [CrossRef] [PubMed]
  12. J. B. Burl, “A nonlinear correlation algorithm for estimating the motion of multiple objects in an image sequence,” in Conference Record of the Twenty-Ninth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1995), Vol. 2, 1413–1418.
  13. J. B. Burl, S. S. Karampuri, “Nonlinear correlation for motion estimation in sequences of Markov modeled images,” in Conference Record of the Thirtieth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1996), Vol. 2, 1036–1040.
  14. H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I (Wiley, New York, 1968).

1994 (1)

M. R. Luettgen, W. C. Karl, A. S. Willsky, “Efficient multiscale regularization with applications to the computation of optical flow,” IEEE Trans. Image Process. 3, 41–63 (1994).
[CrossRef] [PubMed]

1993 (1)

J. B. Burl, “A reduced order extended Kalman filter for sequential images containing a moving object,” IEEE Trans. Image Process. 2, 285–385 (1993).
[CrossRef] [PubMed]

1989 (1)

B. G. Schunk, “Image flow segmentation and estimation by constraint line clustering,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 1010–1024 (1989).
[CrossRef]

1986 (1)

B. G. Schunk, “The image flow constraint equation,” Comput. Vision Graph. Image Process. 35, 20–46 (1986).
[CrossRef]

1983 (1)

B. L. Yen, T. S. Huang, “Determining 3-D motion and structure of a rigid body using the spherical projection,” Comput. Vision Graph. Image Process. 21, 21–32 (1983).
[CrossRef]

1981 (2)

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

R. Y. Tsai, T. S. Huang, “Estimating 3-D motion parameters of a rigid planar patch. I,” IEEE Trans. Acoust. Speech Signal Process. 29, 1147–1152 (1981).
[CrossRef]

1980 (1)

J. W. Roach, J. K. Aggarwal, “Determining the movement of objects from a sequence of images,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 554–562 (1980).
[CrossRef]

1979 (2)

R. Jain, H.-H. Nagel, “On the analysis of accumulative difference pictures from image sequences of real world scenes,” IEEE Trans. Pattern Analy. Mach. Intell. 1, 206–214 (1979).
[CrossRef]

R. Jain, W. N. Martin, J. K. Aggarwal, “Segmentation through the detection of changes due to motion,” Comput. Graph. Image Process. 11, 13–34 (1979).
[CrossRef]

1975 (1)

J. K. Aggarwal, R. O. Duda, “Computer analysis of moving polygonal images,” IEEE Trans. Comput. 24, 966–976 (1975).
[CrossRef]

Aggarwal, J. K.

J. W. Roach, J. K. Aggarwal, “Determining the movement of objects from a sequence of images,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 554–562 (1980).
[CrossRef]

R. Jain, W. N. Martin, J. K. Aggarwal, “Segmentation through the detection of changes due to motion,” Comput. Graph. Image Process. 11, 13–34 (1979).
[CrossRef]

J. K. Aggarwal, R. O. Duda, “Computer analysis of moving polygonal images,” IEEE Trans. Comput. 24, 966–976 (1975).
[CrossRef]

Burl, J. B.

J. B. Burl, “A reduced order extended Kalman filter for sequential images containing a moving object,” IEEE Trans. Image Process. 2, 285–385 (1993).
[CrossRef] [PubMed]

J. B. Burl, S. S. Karampuri, “Nonlinear correlation for motion estimation in sequences of Markov modeled images,” in Conference Record of the Thirtieth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1996), Vol. 2, 1036–1040.

J. B. Burl, “A nonlinear correlation algorithm for estimating the motion of multiple objects in an image sequence,” in Conference Record of the Twenty-Ninth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1995), Vol. 2, 1413–1418.

Duda, R. O.

J. K. Aggarwal, R. O. Duda, “Computer analysis of moving polygonal images,” IEEE Trans. Comput. 24, 966–976 (1975).
[CrossRef]

Horn, B. K. P.

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Huang, T. S.

B. L. Yen, T. S. Huang, “Determining 3-D motion and structure of a rigid body using the spherical projection,” Comput. Vision Graph. Image Process. 21, 21–32 (1983).
[CrossRef]

R. Y. Tsai, T. S. Huang, “Estimating 3-D motion parameters of a rigid planar patch. I,” IEEE Trans. Acoust. Speech Signal Process. 29, 1147–1152 (1981).
[CrossRef]

Jain, R.

R. Jain, H.-H. Nagel, “On the analysis of accumulative difference pictures from image sequences of real world scenes,” IEEE Trans. Pattern Analy. Mach. Intell. 1, 206–214 (1979).
[CrossRef]

R. Jain, W. N. Martin, J. K. Aggarwal, “Segmentation through the detection of changes due to motion,” Comput. Graph. Image Process. 11, 13–34 (1979).
[CrossRef]

Karampuri, S. S.

J. B. Burl, S. S. Karampuri, “Nonlinear correlation for motion estimation in sequences of Markov modeled images,” in Conference Record of the Thirtieth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1996), Vol. 2, 1036–1040.

Karl, W. C.

M. R. Luettgen, W. C. Karl, A. S. Willsky, “Efficient multiscale regularization with applications to the computation of optical flow,” IEEE Trans. Image Process. 3, 41–63 (1994).
[CrossRef] [PubMed]

Luettgen, M. R.

M. R. Luettgen, W. C. Karl, A. S. Willsky, “Efficient multiscale regularization with applications to the computation of optical flow,” IEEE Trans. Image Process. 3, 41–63 (1994).
[CrossRef] [PubMed]

Martin, W. N.

R. Jain, W. N. Martin, J. K. Aggarwal, “Segmentation through the detection of changes due to motion,” Comput. Graph. Image Process. 11, 13–34 (1979).
[CrossRef]

Nagel, H.-H.

R. Jain, H.-H. Nagel, “On the analysis of accumulative difference pictures from image sequences of real world scenes,” IEEE Trans. Pattern Analy. Mach. Intell. 1, 206–214 (1979).
[CrossRef]

Roach, J. W.

J. W. Roach, J. K. Aggarwal, “Determining the movement of objects from a sequence of images,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 554–562 (1980).
[CrossRef]

Schunck, B. G.

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Schunk, B. G.

B. G. Schunk, “Image flow segmentation and estimation by constraint line clustering,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 1010–1024 (1989).
[CrossRef]

B. G. Schunk, “The image flow constraint equation,” Comput. Vision Graph. Image Process. 35, 20–46 (1986).
[CrossRef]

Tsai, R. Y.

R. Y. Tsai, T. S. Huang, “Estimating 3-D motion parameters of a rigid planar patch. I,” IEEE Trans. Acoust. Speech Signal Process. 29, 1147–1152 (1981).
[CrossRef]

Van Trees, H. L.

H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I (Wiley, New York, 1968).

Willsky, A. S.

M. R. Luettgen, W. C. Karl, A. S. Willsky, “Efficient multiscale regularization with applications to the computation of optical flow,” IEEE Trans. Image Process. 3, 41–63 (1994).
[CrossRef] [PubMed]

Yen, B. L.

B. L. Yen, T. S. Huang, “Determining 3-D motion and structure of a rigid body using the spherical projection,” Comput. Vision Graph. Image Process. 21, 21–32 (1983).
[CrossRef]

Artif. Intell. (1)

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Comput. Graph. Image Process. (1)

R. Jain, W. N. Martin, J. K. Aggarwal, “Segmentation through the detection of changes due to motion,” Comput. Graph. Image Process. 11, 13–34 (1979).
[CrossRef]

Comput. Vision Graph. Image Process. (2)

B. L. Yen, T. S. Huang, “Determining 3-D motion and structure of a rigid body using the spherical projection,” Comput. Vision Graph. Image Process. 21, 21–32 (1983).
[CrossRef]

B. G. Schunk, “The image flow constraint equation,” Comput. Vision Graph. Image Process. 35, 20–46 (1986).
[CrossRef]

IEEE Trans. Acoust. Speech Signal Process. (1)

R. Y. Tsai, T. S. Huang, “Estimating 3-D motion parameters of a rigid planar patch. I,” IEEE Trans. Acoust. Speech Signal Process. 29, 1147–1152 (1981).
[CrossRef]

IEEE Trans. Comput. (1)

J. K. Aggarwal, R. O. Duda, “Computer analysis of moving polygonal images,” IEEE Trans. Comput. 24, 966–976 (1975).
[CrossRef]

IEEE Trans. Image Process. (2)

M. R. Luettgen, W. C. Karl, A. S. Willsky, “Efficient multiscale regularization with applications to the computation of optical flow,” IEEE Trans. Image Process. 3, 41–63 (1994).
[CrossRef] [PubMed]

J. B. Burl, “A reduced order extended Kalman filter for sequential images containing a moving object,” IEEE Trans. Image Process. 2, 285–385 (1993).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

B. G. Schunk, “Image flow segmentation and estimation by constraint line clustering,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 1010–1024 (1989).
[CrossRef]

J. W. Roach, J. K. Aggarwal, “Determining the movement of objects from a sequence of images,” IEEE Trans. Pattern Anal. Mach. Intell. 2, 554–562 (1980).
[CrossRef]

IEEE Trans. Pattern Analy. Mach. Intell. (1)

R. Jain, H.-H. Nagel, “On the analysis of accumulative difference pictures from image sequences of real world scenes,” IEEE Trans. Pattern Analy. Mach. Intell. 1, 206–214 (1979).
[CrossRef]

Other (3)

J. B. Burl, “A nonlinear correlation algorithm for estimating the motion of multiple objects in an image sequence,” in Conference Record of the Twenty-Ninth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1995), Vol. 2, 1413–1418.

J. B. Burl, S. S. Karampuri, “Nonlinear correlation for motion estimation in sequences of Markov modeled images,” in Conference Record of the Thirtieth Annual Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society, Los Alamitos, Calif., 1996), Vol. 2, 1036–1040.

H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I (Wiley, New York, 1968).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

Image motion estimator.

Fig. 2
Fig. 2

Image sequence. (a) Image 1. (b) Image 2.

Fig. 3
Fig. 3

Accumulator for the example.

Fig. 4
Fig. 4

Image sequence. (a) Image 1. (b) Image 2.

Fig. 5
Fig. 5

Accumulator.

Fig. 6
Fig. 6

Accumulator after thresholding.

Fig. 7
Fig. 7

Object size estimates verses the change in pixel intensity between frames. (a) Object with displacement [0, 7]. (b) Object with displacement [7, -11].

Equations (31)

Equations on this page are rendered with MathJax. Learn more.

EIm1, n1, iIm2, n2, i=σI2ρx|m1-m2|ρy|n1-n2|.
|Im2, n2, 2-Im1, n1, 1|Δ,
Au, νηu, ν,
PAu, ν=k|S.
PAou, ν=lPABu, ν=k-l,
PAu, ν=k|S=l=0S PAou, ν=l×PABu, ν=k-l,
x¯ˆ=k=1Au,ν mk-xcE[ABu, ν]Sˆ;y¯ˆ=k=1Au,ν nk-ycE[ABu, ν]Sˆ,
Pfa=u,νV PAu, νηGMu, ν|H0,
Pfa=NBPAu, νηGMu, ν|H0.
Dm, n, u, ν=Im+u, n+ν, 2-Im, n, 1  -Δ, Δ
Im+u, n+ν, 2=Im+u, n+ν, 1+ΔIm+u, n+ν.
Dm, n, u, ν=Im+u, n+ν, 1-Im, n, 1+ΔIm+u, n+ν.
μDu, ν=EIm+u, n+ν, 1-Im, n, 1+ΔIm+u, n+ν=0.
σD2u, ν=E[D2m, n, u, ν]=EI2m+u, n+ν, 1+EI2m, n, 1-2EIm+u, n+ν, 1Im, n, 1+EΔI2m+u, n+ν=2σI21-ρx|u|ρy|ν|+σΔI2.
Psmu, ν=2 erfΔσDu, ν,
erfx=12π0xexp-y22dy.
μAu, ν=Nu, νPsmu, ν;σA2u, ν=Nu, νPsmu, ν1-Psmu, ν,
ηu, ν=μAu, ν+βσAu, ν.
Pfa=2NB erfcβ,
erfcx=12πxexp-y22dy.
PAu, ν=k|S=l=0S PAou, ν=l×PABu, ν=k-l.
PABu, ν=j=Nu, ν-S!j!Nu, ν-S-j!Psmju, ν×1-Psmu, νNu,ν-S-j.
PAou, ν=l=S!l!S-l! Pml1-PmS-l,
Pm=2 erfΔ/σΔI.
PAu, ν=k|S=l=0SS!Pml1-PmS-ll!S-l!×Nu, ν-S!Psmk-lu, ν1-Psmu, νNu, ν-S-k+lk-l!Nu, ν-S-k+l!.
PAu, ν=k|Sl=0Sexp-l-SPm22SPm1-Pm(2πSPm1-Pm)1/2×exp-k-l-Nu, ν-SPsmu, ν22Nu, ν-SPsmu, ν1-Psmu, ν(2πNu, ν-SPsmu, ν1-Psmu, ν)1/2.
ESˆk-S2=kSˆk-S2PAu, ν=k|S.
u,νB Nu, νNBNxNy
SPm+u,νBNu, ν-SPsmu, νNBNxNy
2SPm+u,νBNu, ν-SPsmu, ν2NBNxNy.
ESˆk-S2=1.7 pixels2.

Metrics