Abstract

We propose a model of how humans sense the velocity of moving images. The model exploits constraints provided by human psychophysics, notably that motion-sensing elements appear tuned for two-dimensional spatial frequency, and by the frequency spectrum of a moving image, namely, that its support lies in the plane in which the temporal frequency equals the dot product of the spatial frequency and the image velocity. The first stage of the model is a set of spatial-frequency-tuned, direction-selective linear sensors. The temporal frequency of the response of each sensor is shown to encode the component of the image velocity in the sensor direction. At the second stage, these components are resolved in order to measure the velocity of image motion at each of a number of spatial locations and spatial frequencies. The model has been applied to several illustrative examples, including apparent motion, coherent gratings, and natural image sequences. The model agrees qualitatively with human perception.

© 1985 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Spatiotemporal energy models for the perception of motion

Edward H. Adelson and James R. Bergen
J. Opt. Soc. Am. A 2(2) 284-299 (1985)

Temporal covariance model of human motion perception

Jan P. H. van Santen and George Sperling
J. Opt. Soc. Am. A 1(5) 451-473 (1984)

Three-systems theory of human visual motion perception: review and update

Zhong-Lin Lu and George Sperling
J. Opt. Soc. Am. A 18(9) 2331-2370 (2001)

References

  • View by:
  • |
  • |
  • |

  1. J. J. Gibson, The Perception of the Visual World (Houghton Mifflin, Boston, Mass., 1950).
  2. S. Ullman, The Interpretation of Visual Motion (MIT U. Press, Cambridge, Mass., 1979).
  3. H. C. Longuet-Higgins and K. Prazdny, “The interpretation of moving retinal images,” Proc. R. Soc. London Ser. B 208, 385–387 (1980).
    [Crossref]
  4. E. C. Hildreth, The Measurement of Visual Motion (MIT U. Press, Cambridge, Mass., 1983).
  5. A. B. Watson and A. J. Ahumada, “A look at motion in the frequency domain,”NASA Tech. Mem. 84352 (1983).
  6. E. H. Adelson and J. A. Movshon, “Phenomenal coherence of moving visual patterns,” Nature 300, 523–525 (1982).
    [Crossref] [PubMed]
  7. A. B. Watson and A. J. Ahumada, “A linear motion sensor,” Perception 12, A17 (1983).
  8. F. W. Campbell and J. G. Robson, “Application of Fourier analysis to the visibility of gratings,”J. Phys. London 197, 551–566 (1968).
  9. N. Graham and J. Nachmias, “Detection of grating patterns containing two spatial frequencies: a comparison of single-channel and multiple-channel models,” Vision Res. 11, 251–259 (1971).
    [Crossref] [PubMed]
  10. A. B. Watson, “Summation of grating patches indicates many types of detector at one retinal location,” Vision Res. 22, 17–25 (1982).
    [Crossref] [PubMed]
  11. N. Graham, “Spatial-frequency channels in the human visual system: effects of luminance and pattern drift rate,” Vision Res. 12, 53–68 (1972).
    [Crossref] [PubMed]
  12. A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
    [Crossref] [PubMed]
  13. P. Thompson, “The coding of velocity of movement in the human visual system,” Vision Res. 24, 41–45 (1984).
    [Crossref] [PubMed]
  14. S. P. McKee, “A local mechanism for differential velocity detection,” Vision Res. 21, 491–500 (1981).
    [Crossref] [PubMed]
  15. A. Pantle, “Motion aftereffect magnitude as a measure of the spatiotemporal response properties of direction-sensitive analyzers,” Vision Res. 14, 1229–1236 (1974).
    [Crossref] [PubMed]
  16. E. Levinson and R. Sekuler, “Adaptation alters perceived direction of motion,” Vision Res. 16, 779–781 (1976).
    [Crossref] [PubMed]
  17. D. J. Tolhurst, “Separate channels for the analysis of the shape and the movement of a moving visual stimulus,”J. Physiol. London 231, 385–402 (1973).
  18. E. Levinson and R. Sekuler, “The independence of channels in human vision selective for direction of movement,”J. Physiol. London 250, 347–366 (1975).
    [PubMed]
  19. C. F. Stromeyer, J. C. Madsen, S. Klein, and Y. Y. Zeevi, “Movement-selective mechanisms in human vision sensitive to high spatial frequencies,”J. Opt. Soc. Am. 68, 1002–1005 (1978).
    [Crossref] [PubMed]
  20. R. J. W. Mansfield and J. Nachmias, “Perceived direction of motion under retinal image stabilization,” Vision Res. 21, 1423–1425 (1981).
    [Crossref] [PubMed]
  21. J. G. Robson, “Spatial and temporal contrast-sensitivity functions of the visual system,”J. Opt. Soc. Am. 56, 1141–1142 (1966).
    [Crossref]
  22. J. J. Koenderink and A. J. van Doorn, “Spatiotemporal contrast detection threshold surface is bimodal,” Opt. Lett. 4, 32–34 (1979).
    [Crossref] [PubMed]
  23. D. H. Kelly, “Adaptation effects on spatio-temporal sine-wave thresholds,” Vision Res. 12, 89–101 (1972).
    [Crossref] [PubMed]
  24. D. H. Kelly, “Motion and vision. II. Stabilized spatio-temporal theshold surface,”J. Opt. Soc. Am. 69, 1340–1349 (1979).
    [Crossref] [PubMed]
  25. D. C. Burr and J. Ross, “Contrast sensitivity at high velocities,” Vision Res. 22, 479–484 (1982).
    [Crossref] [PubMed]
  26. P. A. Kolers, Aspects of Apparent Motion (Pergamon, New York, 1972).
  27. G. Sperling, “Movement perception in computer-driven visual displays,” Behav. Res. Methods Instrum. 8, 144–151 (1976).
    [Crossref]
  28. A. B. Watson and A. J. Ahumada, “Sampling, filtering, and apparent motion,” Perception 11, A15 (1982).
  29. A. B. Watson, A. J. Ahumada, and J. Farrell, “The window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays,”NASA Tech. Pap. 2211, (1983).
  30. M. Fable and T. Poggio, “Visual hyperacuity: spatiotemporal interpolation in human vision,” Proc. R. Soc. London Ser. B 213, 451–477 (1981).
    [Crossref]
  31. A. B. Watson and J. G. Robson, “Discrimination at threshold: labelled detectors in human vision,” Vision Res. 21, 1115–1122 (1981).
    [Crossref]
  32. E. Levinson and R. Sekuler, “Inhibition and disinhibition of direction-specific mechanisms in human vision,” Nature 254, 692–694 (1975).
    [Crossref] [PubMed]
  33. C. F. Stromeyer, R. E. Kronauer, J. C. Madsen, and S. A. Klein, “Opponent-movement mechanisms in human vision,” J. Opt. Soc. Am. A 1, 876–884 (1984).
    [Crossref] [PubMed]
  34. P. Thompson, “Discrimination of moving gratings at and above detection threshold,” Vision Res. 23, 1533–1538 (1983).
    [Crossref] [PubMed]
  35. F. W. Campbell, J. Nachmias, and J. Jukes, “Spatial-frequency discrimination in human vision,”J. Opt. Soc. Am. 60, 555–559 (1970).
    [Crossref] [PubMed]
  36. F. H. C. Crick, D. C. Marr, and T. Poggio, “An information processing approach to understanding the visual cortex,” in The Organization of the Cerebral Cortex, S. G. Dennis, ed. (MIT U. Press, Cambridge, Mass., 1981).
  37. J. G. Moik, “Digital processing of remotely sensed images,”NASA Doc. SP-431 (1980).
  38. The orientation of a 2D frequency component is the angle of a normal to the wavefront.
  39. D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” Proc. R. Soc. London Ser. B 211, 151–180 (1981).
    [Crossref]
  40. F. L. van Nes, J. J. Koenderink, H. Nas, and M. A. Bouman, “Spatiotemporal modulation transfer in the human eye,”J. Opt. Soc. Am. 57, 1082–1088 (1967).
    [Crossref] [PubMed]
  41. A. B. Watson, “Temporal Sensitivity,” in Handbook of Perception and Human Performance, J. Thomas, ed. (Wiley, New York, to be published).
  42. M. G. F. Fourtes and A. L. Hodgkin, “Changes in the time scale and sensitivity in the omatidia of limulus,”J. Physiol. London 172, 239–263 (1964).
  43. A. B. Watson, “Derivation of the impulse response: comments on the method of Roufs and Blommaert,” Vision Res. 22, 1335–1337 (1982).
    [Crossref] [PubMed]
  44. S. Marcelja, “Mathematical description of the responses of simple cortical cells,”J. Opt. Soc. Am. 70, 1297–1300 (1980).
    [Crossref] [PubMed]
  45. B. Sakitt and H. B. Barlow, “A model for the economical encoding of the visual image in cerebral cortex,” Biol. Cybern. 43, 97–108 (1982).
    [Crossref] [PubMed]
  46. A. B. Watson, “Detection and recognition of simple spatial forms,” in Physical and Biological Processing of Images, A. C. Slade, ed. (Springer-Verlag, Berlin, 1983).
    [Crossref]
  47. S. Tanimoto and T. Pavlidis, “A hierarchical data structure for picture processing,” Comput. Graphics Image Process. 4, 104–119 (1975).
    [Crossref]
  48. P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,”IEEE Trans. Commun. COM-31, 532–540 (1983).
    [Crossref]
  49. J. L. Crowley and R. M. Stern, “Fast computation of the difference of low-pass transform,”IEEE Trans. Pattern Anal. Mach. Intelligence PAMI-6, 212–222 (1984).
    [Crossref]
  50. G. Sperling and J. P. H. van Santen, “Temporal covariance model of human motion perception,” J. Opt. Soc. Am. A 1, 451–473 (1984).
    [Crossref] [PubMed]
  51. E. H. Adelson and J. R. Bergen, “Motion channels based on spatiotemporal energy,” Invest. Ophthalmol. Vis. Sci. Suppl. 25, 14 (A) (1984).

1984 (5)

P. Thompson, “The coding of velocity of movement in the human visual system,” Vision Res. 24, 41–45 (1984).
[Crossref] [PubMed]

J. L. Crowley and R. M. Stern, “Fast computation of the difference of low-pass transform,”IEEE Trans. Pattern Anal. Mach. Intelligence PAMI-6, 212–222 (1984).
[Crossref]

E. H. Adelson and J. R. Bergen, “Motion channels based on spatiotemporal energy,” Invest. Ophthalmol. Vis. Sci. Suppl. 25, 14 (A) (1984).

G. Sperling and J. P. H. van Santen, “Temporal covariance model of human motion perception,” J. Opt. Soc. Am. A 1, 451–473 (1984).
[Crossref] [PubMed]

C. F. Stromeyer, R. E. Kronauer, J. C. Madsen, and S. A. Klein, “Opponent-movement mechanisms in human vision,” J. Opt. Soc. Am. A 1, 876–884 (1984).
[Crossref] [PubMed]

1983 (3)

P. Thompson, “Discrimination of moving gratings at and above detection threshold,” Vision Res. 23, 1533–1538 (1983).
[Crossref] [PubMed]

A. B. Watson and A. J. Ahumada, “A linear motion sensor,” Perception 12, A17 (1983).

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,”IEEE Trans. Commun. COM-31, 532–540 (1983).
[Crossref]

1982 (6)

A. B. Watson, “Derivation of the impulse response: comments on the method of Roufs and Blommaert,” Vision Res. 22, 1335–1337 (1982).
[Crossref] [PubMed]

E. H. Adelson and J. A. Movshon, “Phenomenal coherence of moving visual patterns,” Nature 300, 523–525 (1982).
[Crossref] [PubMed]

A. B. Watson, “Summation of grating patches indicates many types of detector at one retinal location,” Vision Res. 22, 17–25 (1982).
[Crossref] [PubMed]

D. C. Burr and J. Ross, “Contrast sensitivity at high velocities,” Vision Res. 22, 479–484 (1982).
[Crossref] [PubMed]

A. B. Watson and A. J. Ahumada, “Sampling, filtering, and apparent motion,” Perception 11, A15 (1982).

B. Sakitt and H. B. Barlow, “A model for the economical encoding of the visual image in cerebral cortex,” Biol. Cybern. 43, 97–108 (1982).
[Crossref] [PubMed]

1981 (5)

M. Fable and T. Poggio, “Visual hyperacuity: spatiotemporal interpolation in human vision,” Proc. R. Soc. London Ser. B 213, 451–477 (1981).
[Crossref]

A. B. Watson and J. G. Robson, “Discrimination at threshold: labelled detectors in human vision,” Vision Res. 21, 1115–1122 (1981).
[Crossref]

D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” Proc. R. Soc. London Ser. B 211, 151–180 (1981).
[Crossref]

R. J. W. Mansfield and J. Nachmias, “Perceived direction of motion under retinal image stabilization,” Vision Res. 21, 1423–1425 (1981).
[Crossref] [PubMed]

S. P. McKee, “A local mechanism for differential velocity detection,” Vision Res. 21, 491–500 (1981).
[Crossref] [PubMed]

1980 (4)

A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
[Crossref] [PubMed]

H. C. Longuet-Higgins and K. Prazdny, “The interpretation of moving retinal images,” Proc. R. Soc. London Ser. B 208, 385–387 (1980).
[Crossref]

J. G. Moik, “Digital processing of remotely sensed images,”NASA Doc. SP-431 (1980).

S. Marcelja, “Mathematical description of the responses of simple cortical cells,”J. Opt. Soc. Am. 70, 1297–1300 (1980).
[Crossref] [PubMed]

1979 (2)

1978 (1)

1976 (2)

E. Levinson and R. Sekuler, “Adaptation alters perceived direction of motion,” Vision Res. 16, 779–781 (1976).
[Crossref] [PubMed]

G. Sperling, “Movement perception in computer-driven visual displays,” Behav. Res. Methods Instrum. 8, 144–151 (1976).
[Crossref]

1975 (3)

S. Tanimoto and T. Pavlidis, “A hierarchical data structure for picture processing,” Comput. Graphics Image Process. 4, 104–119 (1975).
[Crossref]

E. Levinson and R. Sekuler, “The independence of channels in human vision selective for direction of movement,”J. Physiol. London 250, 347–366 (1975).
[PubMed]

E. Levinson and R. Sekuler, “Inhibition and disinhibition of direction-specific mechanisms in human vision,” Nature 254, 692–694 (1975).
[Crossref] [PubMed]

1974 (1)

A. Pantle, “Motion aftereffect magnitude as a measure of the spatiotemporal response properties of direction-sensitive analyzers,” Vision Res. 14, 1229–1236 (1974).
[Crossref] [PubMed]

1973 (1)

D. J. Tolhurst, “Separate channels for the analysis of the shape and the movement of a moving visual stimulus,”J. Physiol. London 231, 385–402 (1973).

1972 (2)

D. H. Kelly, “Adaptation effects on spatio-temporal sine-wave thresholds,” Vision Res. 12, 89–101 (1972).
[Crossref] [PubMed]

N. Graham, “Spatial-frequency channels in the human visual system: effects of luminance and pattern drift rate,” Vision Res. 12, 53–68 (1972).
[Crossref] [PubMed]

1971 (1)

N. Graham and J. Nachmias, “Detection of grating patterns containing two spatial frequencies: a comparison of single-channel and multiple-channel models,” Vision Res. 11, 251–259 (1971).
[Crossref] [PubMed]

1970 (1)

1968 (1)

F. W. Campbell and J. G. Robson, “Application of Fourier analysis to the visibility of gratings,”J. Phys. London 197, 551–566 (1968).

1967 (1)

1966 (1)

1964 (1)

M. G. F. Fourtes and A. L. Hodgkin, “Changes in the time scale and sensitivity in the omatidia of limulus,”J. Physiol. London 172, 239–263 (1964).

Adelson, E. H.

E. H. Adelson and J. R. Bergen, “Motion channels based on spatiotemporal energy,” Invest. Ophthalmol. Vis. Sci. Suppl. 25, 14 (A) (1984).

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,”IEEE Trans. Commun. COM-31, 532–540 (1983).
[Crossref]

E. H. Adelson and J. A. Movshon, “Phenomenal coherence of moving visual patterns,” Nature 300, 523–525 (1982).
[Crossref] [PubMed]

Ahumada, A. J.

A. B. Watson and A. J. Ahumada, “A linear motion sensor,” Perception 12, A17 (1983).

A. B. Watson and A. J. Ahumada, “Sampling, filtering, and apparent motion,” Perception 11, A15 (1982).

A. B. Watson, A. J. Ahumada, and J. Farrell, “The window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays,”NASA Tech. Pap. 2211, (1983).

A. B. Watson and A. J. Ahumada, “A look at motion in the frequency domain,”NASA Tech. Mem. 84352 (1983).

Barlow, H. B.

B. Sakitt and H. B. Barlow, “A model for the economical encoding of the visual image in cerebral cortex,” Biol. Cybern. 43, 97–108 (1982).
[Crossref] [PubMed]

Bergen, J. R.

E. H. Adelson and J. R. Bergen, “Motion channels based on spatiotemporal energy,” Invest. Ophthalmol. Vis. Sci. Suppl. 25, 14 (A) (1984).

Bouman, M. A.

Burr, D. C.

D. C. Burr and J. Ross, “Contrast sensitivity at high velocities,” Vision Res. 22, 479–484 (1982).
[Crossref] [PubMed]

Burt, P. J.

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,”IEEE Trans. Commun. COM-31, 532–540 (1983).
[Crossref]

Campbell, F. W.

F. W. Campbell, J. Nachmias, and J. Jukes, “Spatial-frequency discrimination in human vision,”J. Opt. Soc. Am. 60, 555–559 (1970).
[Crossref] [PubMed]

F. W. Campbell and J. G. Robson, “Application of Fourier analysis to the visibility of gratings,”J. Phys. London 197, 551–566 (1968).

Crick, F. H. C.

F. H. C. Crick, D. C. Marr, and T. Poggio, “An information processing approach to understanding the visual cortex,” in The Organization of the Cerebral Cortex, S. G. Dennis, ed. (MIT U. Press, Cambridge, Mass., 1981).

Crowley, J. L.

J. L. Crowley and R. M. Stern, “Fast computation of the difference of low-pass transform,”IEEE Trans. Pattern Anal. Mach. Intelligence PAMI-6, 212–222 (1984).
[Crossref]

Fable, M.

M. Fable and T. Poggio, “Visual hyperacuity: spatiotemporal interpolation in human vision,” Proc. R. Soc. London Ser. B 213, 451–477 (1981).
[Crossref]

Farrell, J.

A. B. Watson, A. J. Ahumada, and J. Farrell, “The window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays,”NASA Tech. Pap. 2211, (1983).

Fourtes, M. G. F.

M. G. F. Fourtes and A. L. Hodgkin, “Changes in the time scale and sensitivity in the omatidia of limulus,”J. Physiol. London 172, 239–263 (1964).

Gibson, J. J.

J. J. Gibson, The Perception of the Visual World (Houghton Mifflin, Boston, Mass., 1950).

Graham, N.

N. Graham, “Spatial-frequency channels in the human visual system: effects of luminance and pattern drift rate,” Vision Res. 12, 53–68 (1972).
[Crossref] [PubMed]

N. Graham and J. Nachmias, “Detection of grating patterns containing two spatial frequencies: a comparison of single-channel and multiple-channel models,” Vision Res. 11, 251–259 (1971).
[Crossref] [PubMed]

Hildreth, E. C.

E. C. Hildreth, The Measurement of Visual Motion (MIT U. Press, Cambridge, Mass., 1983).

Hodgkin, A. L.

M. G. F. Fourtes and A. L. Hodgkin, “Changes in the time scale and sensitivity in the omatidia of limulus,”J. Physiol. London 172, 239–263 (1964).

Jukes, J.

Kelly, D. H.

D. H. Kelly, “Motion and vision. II. Stabilized spatio-temporal theshold surface,”J. Opt. Soc. Am. 69, 1340–1349 (1979).
[Crossref] [PubMed]

D. H. Kelly, “Adaptation effects on spatio-temporal sine-wave thresholds,” Vision Res. 12, 89–101 (1972).
[Crossref] [PubMed]

Klein, S.

Klein, S. A.

Koenderink, J. J.

Kolers, P. A.

P. A. Kolers, Aspects of Apparent Motion (Pergamon, New York, 1972).

Kronauer, R. E.

Levinson, E.

E. Levinson and R. Sekuler, “Adaptation alters perceived direction of motion,” Vision Res. 16, 779–781 (1976).
[Crossref] [PubMed]

E. Levinson and R. Sekuler, “Inhibition and disinhibition of direction-specific mechanisms in human vision,” Nature 254, 692–694 (1975).
[Crossref] [PubMed]

E. Levinson and R. Sekuler, “The independence of channels in human vision selective for direction of movement,”J. Physiol. London 250, 347–366 (1975).
[PubMed]

Longuet-Higgins, H. C.

H. C. Longuet-Higgins and K. Prazdny, “The interpretation of moving retinal images,” Proc. R. Soc. London Ser. B 208, 385–387 (1980).
[Crossref]

Madsen, J. C.

Mansfield, R. J. W.

R. J. W. Mansfield and J. Nachmias, “Perceived direction of motion under retinal image stabilization,” Vision Res. 21, 1423–1425 (1981).
[Crossref] [PubMed]

Marcelja, S.

Marr, D.

D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” Proc. R. Soc. London Ser. B 211, 151–180 (1981).
[Crossref]

Marr, D. C.

F. H. C. Crick, D. C. Marr, and T. Poggio, “An information processing approach to understanding the visual cortex,” in The Organization of the Cerebral Cortex, S. G. Dennis, ed. (MIT U. Press, Cambridge, Mass., 1981).

McKee, S. P.

S. P. McKee, “A local mechanism for differential velocity detection,” Vision Res. 21, 491–500 (1981).
[Crossref] [PubMed]

Moik, J. G.

J. G. Moik, “Digital processing of remotely sensed images,”NASA Doc. SP-431 (1980).

Movshon, J. A.

E. H. Adelson and J. A. Movshon, “Phenomenal coherence of moving visual patterns,” Nature 300, 523–525 (1982).
[Crossref] [PubMed]

Murphy, B. J.

A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
[Crossref] [PubMed]

Nachmias, J.

R. J. W. Mansfield and J. Nachmias, “Perceived direction of motion under retinal image stabilization,” Vision Res. 21, 1423–1425 (1981).
[Crossref] [PubMed]

A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
[Crossref] [PubMed]

N. Graham and J. Nachmias, “Detection of grating patterns containing two spatial frequencies: a comparison of single-channel and multiple-channel models,” Vision Res. 11, 251–259 (1971).
[Crossref] [PubMed]

F. W. Campbell, J. Nachmias, and J. Jukes, “Spatial-frequency discrimination in human vision,”J. Opt. Soc. Am. 60, 555–559 (1970).
[Crossref] [PubMed]

Nas, H.

Pantle, A.

A. Pantle, “Motion aftereffect magnitude as a measure of the spatiotemporal response properties of direction-sensitive analyzers,” Vision Res. 14, 1229–1236 (1974).
[Crossref] [PubMed]

Pavlidis, T.

S. Tanimoto and T. Pavlidis, “A hierarchical data structure for picture processing,” Comput. Graphics Image Process. 4, 104–119 (1975).
[Crossref]

Poggio, T.

M. Fable and T. Poggio, “Visual hyperacuity: spatiotemporal interpolation in human vision,” Proc. R. Soc. London Ser. B 213, 451–477 (1981).
[Crossref]

F. H. C. Crick, D. C. Marr, and T. Poggio, “An information processing approach to understanding the visual cortex,” in The Organization of the Cerebral Cortex, S. G. Dennis, ed. (MIT U. Press, Cambridge, Mass., 1981).

Prazdny, K.

H. C. Longuet-Higgins and K. Prazdny, “The interpretation of moving retinal images,” Proc. R. Soc. London Ser. B 208, 385–387 (1980).
[Crossref]

Robson, J. G.

A. B. Watson and J. G. Robson, “Discrimination at threshold: labelled detectors in human vision,” Vision Res. 21, 1115–1122 (1981).
[Crossref]

F. W. Campbell and J. G. Robson, “Application of Fourier analysis to the visibility of gratings,”J. Phys. London 197, 551–566 (1968).

J. G. Robson, “Spatial and temporal contrast-sensitivity functions of the visual system,”J. Opt. Soc. Am. 56, 1141–1142 (1966).
[Crossref]

Ross, J.

D. C. Burr and J. Ross, “Contrast sensitivity at high velocities,” Vision Res. 22, 479–484 (1982).
[Crossref] [PubMed]

Sakitt, B.

B. Sakitt and H. B. Barlow, “A model for the economical encoding of the visual image in cerebral cortex,” Biol. Cybern. 43, 97–108 (1982).
[Crossref] [PubMed]

Sekuler, R.

E. Levinson and R. Sekuler, “Adaptation alters perceived direction of motion,” Vision Res. 16, 779–781 (1976).
[Crossref] [PubMed]

E. Levinson and R. Sekuler, “Inhibition and disinhibition of direction-specific mechanisms in human vision,” Nature 254, 692–694 (1975).
[Crossref] [PubMed]

E. Levinson and R. Sekuler, “The independence of channels in human vision selective for direction of movement,”J. Physiol. London 250, 347–366 (1975).
[PubMed]

Sperling, G.

G. Sperling and J. P. H. van Santen, “Temporal covariance model of human motion perception,” J. Opt. Soc. Am. A 1, 451–473 (1984).
[Crossref] [PubMed]

G. Sperling, “Movement perception in computer-driven visual displays,” Behav. Res. Methods Instrum. 8, 144–151 (1976).
[Crossref]

Stern, R. M.

J. L. Crowley and R. M. Stern, “Fast computation of the difference of low-pass transform,”IEEE Trans. Pattern Anal. Mach. Intelligence PAMI-6, 212–222 (1984).
[Crossref]

Stromeyer, C. F.

Tanimoto, S.

S. Tanimoto and T. Pavlidis, “A hierarchical data structure for picture processing,” Comput. Graphics Image Process. 4, 104–119 (1975).
[Crossref]

Thompson, P.

P. Thompson, “The coding of velocity of movement in the human visual system,” Vision Res. 24, 41–45 (1984).
[Crossref] [PubMed]

P. Thompson, “Discrimination of moving gratings at and above detection threshold,” Vision Res. 23, 1533–1538 (1983).
[Crossref] [PubMed]

Thompson, P. G.

A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
[Crossref] [PubMed]

Tolhurst, D. J.

D. J. Tolhurst, “Separate channels for the analysis of the shape and the movement of a moving visual stimulus,”J. Physiol. London 231, 385–402 (1973).

Ullman, S.

D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” Proc. R. Soc. London Ser. B 211, 151–180 (1981).
[Crossref]

S. Ullman, The Interpretation of Visual Motion (MIT U. Press, Cambridge, Mass., 1979).

van Doorn, A. J.

van Nes, F. L.

van Santen, J. P. H.

Watson, A. B.

A. B. Watson and A. J. Ahumada, “A linear motion sensor,” Perception 12, A17 (1983).

A. B. Watson, “Derivation of the impulse response: comments on the method of Roufs and Blommaert,” Vision Res. 22, 1335–1337 (1982).
[Crossref] [PubMed]

A. B. Watson and A. J. Ahumada, “Sampling, filtering, and apparent motion,” Perception 11, A15 (1982).

A. B. Watson, “Summation of grating patches indicates many types of detector at one retinal location,” Vision Res. 22, 17–25 (1982).
[Crossref] [PubMed]

A. B. Watson and J. G. Robson, “Discrimination at threshold: labelled detectors in human vision,” Vision Res. 21, 1115–1122 (1981).
[Crossref]

A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
[Crossref] [PubMed]

A. B. Watson, A. J. Ahumada, and J. Farrell, “The window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays,”NASA Tech. Pap. 2211, (1983).

A. B. Watson, “Detection and recognition of simple spatial forms,” in Physical and Biological Processing of Images, A. C. Slade, ed. (Springer-Verlag, Berlin, 1983).
[Crossref]

A. B. Watson and A. J. Ahumada, “A look at motion in the frequency domain,”NASA Tech. Mem. 84352 (1983).

A. B. Watson, “Temporal Sensitivity,” in Handbook of Perception and Human Performance, J. Thomas, ed. (Wiley, New York, to be published).

Zeevi, Y. Y.

Behav. Res. Methods Instrum. (1)

G. Sperling, “Movement perception in computer-driven visual displays,” Behav. Res. Methods Instrum. 8, 144–151 (1976).
[Crossref]

Biol. Cybern. (1)

B. Sakitt and H. B. Barlow, “A model for the economical encoding of the visual image in cerebral cortex,” Biol. Cybern. 43, 97–108 (1982).
[Crossref] [PubMed]

Comput. Graphics Image Process. (1)

S. Tanimoto and T. Pavlidis, “A hierarchical data structure for picture processing,” Comput. Graphics Image Process. 4, 104–119 (1975).
[Crossref]

IEEE Trans. Commun. (1)

P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,”IEEE Trans. Commun. COM-31, 532–540 (1983).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intelligence (1)

J. L. Crowley and R. M. Stern, “Fast computation of the difference of low-pass transform,”IEEE Trans. Pattern Anal. Mach. Intelligence PAMI-6, 212–222 (1984).
[Crossref]

Invest. Ophthalmol. Vis. Sci. Suppl. (1)

E. H. Adelson and J. R. Bergen, “Motion channels based on spatiotemporal energy,” Invest. Ophthalmol. Vis. Sci. Suppl. 25, 14 (A) (1984).

J. Opt. Soc. Am. (6)

J. Opt. Soc. Am. A (2)

J. Phys. London (1)

F. W. Campbell and J. G. Robson, “Application of Fourier analysis to the visibility of gratings,”J. Phys. London 197, 551–566 (1968).

J. Physiol. London (3)

D. J. Tolhurst, “Separate channels for the analysis of the shape and the movement of a moving visual stimulus,”J. Physiol. London 231, 385–402 (1973).

E. Levinson and R. Sekuler, “The independence of channels in human vision selective for direction of movement,”J. Physiol. London 250, 347–366 (1975).
[PubMed]

M. G. F. Fourtes and A. L. Hodgkin, “Changes in the time scale and sensitivity in the omatidia of limulus,”J. Physiol. London 172, 239–263 (1964).

NASA Doc. SP-431 (1)

J. G. Moik, “Digital processing of remotely sensed images,”NASA Doc. SP-431 (1980).

Nature (2)

E. H. Adelson and J. A. Movshon, “Phenomenal coherence of moving visual patterns,” Nature 300, 523–525 (1982).
[Crossref] [PubMed]

E. Levinson and R. Sekuler, “Inhibition and disinhibition of direction-specific mechanisms in human vision,” Nature 254, 692–694 (1975).
[Crossref] [PubMed]

Opt. Lett. (1)

Perception (2)

A. B. Watson and A. J. Ahumada, “A linear motion sensor,” Perception 12, A17 (1983).

A. B. Watson and A. J. Ahumada, “Sampling, filtering, and apparent motion,” Perception 11, A15 (1982).

Proc. R. Soc. London Ser. B (3)

M. Fable and T. Poggio, “Visual hyperacuity: spatiotemporal interpolation in human vision,” Proc. R. Soc. London Ser. B 213, 451–477 (1981).
[Crossref]

D. Marr and S. Ullman, “Directional selectivity and its use in early visual processing,” Proc. R. Soc. London Ser. B 211, 151–180 (1981).
[Crossref]

H. C. Longuet-Higgins and K. Prazdny, “The interpretation of moving retinal images,” Proc. R. Soc. London Ser. B 208, 385–387 (1980).
[Crossref]

Vision Res. (14)

D. C. Burr and J. Ross, “Contrast sensitivity at high velocities,” Vision Res. 22, 479–484 (1982).
[Crossref] [PubMed]

N. Graham and J. Nachmias, “Detection of grating patterns containing two spatial frequencies: a comparison of single-channel and multiple-channel models,” Vision Res. 11, 251–259 (1971).
[Crossref] [PubMed]

A. B. Watson, “Summation of grating patches indicates many types of detector at one retinal location,” Vision Res. 22, 17–25 (1982).
[Crossref] [PubMed]

N. Graham, “Spatial-frequency channels in the human visual system: effects of luminance and pattern drift rate,” Vision Res. 12, 53–68 (1972).
[Crossref] [PubMed]

A. B. Watson, P. G. Thompson, B. J. Murphy, and J. Nachmias, “Summation and discrimination of gratings moving in opposite directions,” Vision Res. 20, 341–347 (1980).
[Crossref] [PubMed]

P. Thompson, “The coding of velocity of movement in the human visual system,” Vision Res. 24, 41–45 (1984).
[Crossref] [PubMed]

S. P. McKee, “A local mechanism for differential velocity detection,” Vision Res. 21, 491–500 (1981).
[Crossref] [PubMed]

A. Pantle, “Motion aftereffect magnitude as a measure of the spatiotemporal response properties of direction-sensitive analyzers,” Vision Res. 14, 1229–1236 (1974).
[Crossref] [PubMed]

E. Levinson and R. Sekuler, “Adaptation alters perceived direction of motion,” Vision Res. 16, 779–781 (1976).
[Crossref] [PubMed]

A. B. Watson, “Derivation of the impulse response: comments on the method of Roufs and Blommaert,” Vision Res. 22, 1335–1337 (1982).
[Crossref] [PubMed]

A. B. Watson and J. G. Robson, “Discrimination at threshold: labelled detectors in human vision,” Vision Res. 21, 1115–1122 (1981).
[Crossref]

R. J. W. Mansfield and J. Nachmias, “Perceived direction of motion under retinal image stabilization,” Vision Res. 21, 1423–1425 (1981).
[Crossref] [PubMed]

D. H. Kelly, “Adaptation effects on spatio-temporal sine-wave thresholds,” Vision Res. 12, 89–101 (1972).
[Crossref] [PubMed]

P. Thompson, “Discrimination of moving gratings at and above detection threshold,” Vision Res. 23, 1533–1538 (1983).
[Crossref] [PubMed]

Other (10)

A. B. Watson, “Detection and recognition of simple spatial forms,” in Physical and Biological Processing of Images, A. C. Slade, ed. (Springer-Verlag, Berlin, 1983).
[Crossref]

F. H. C. Crick, D. C. Marr, and T. Poggio, “An information processing approach to understanding the visual cortex,” in The Organization of the Cerebral Cortex, S. G. Dennis, ed. (MIT U. Press, Cambridge, Mass., 1981).

A. B. Watson, A. J. Ahumada, and J. Farrell, “The window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays,”NASA Tech. Pap. 2211, (1983).

The orientation of a 2D frequency component is the angle of a normal to the wavefront.

A. B. Watson, “Temporal Sensitivity,” in Handbook of Perception and Human Performance, J. Thomas, ed. (Wiley, New York, to be published).

P. A. Kolers, Aspects of Apparent Motion (Pergamon, New York, 1972).

E. C. Hildreth, The Measurement of Visual Motion (MIT U. Press, Cambridge, Mass., 1983).

A. B. Watson and A. J. Ahumada, “A look at motion in the frequency domain,”NASA Tech. Mem. 84352 (1983).

J. J. Gibson, The Perception of the Visual World (Houghton Mifflin, Boston, Mass., 1950).

S. Ullman, The Interpretation of Visual Motion (MIT U. Press, Cambridge, Mass., 1979).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (26)

Fig. 1
Fig. 1

The effect of motion on the Fourier transform of a 2D space–time image. The effect is shown for a single representative component of spatial frequency u0. The open circles show the location of the components of the static image. The dotted circles show the locations when the image moves with speed r. Each transform point is sheared in the w dimension by an amount −ru, as indicated by the arrows.

Fig. 2
Fig. 2

The effect of motion on the transform of a 3D space–time image. The effect is shown both for a plane representing the full spectrum and for a single representative component of spatial frequency f and orientation α. The solid plane and filled circles show the location of the spectrum of the static image defined by w = 0. Motion at velocity r shears the spectrum into the plane w = r · b, as shown by the dashed–dotted plane and dotted circles. The arrows indicate the displacement of a single spatial-frequency component.

Fig. 3
Fig. 3

An example of direction ambiguity. The motion of the contour seen through the aperture is consistent with any of the velocities a or b or c (after Marr and Ullman). Note that all possible velocities have the same velocity component orthogonal to the contour.39

Fig. 4
Fig. 4

Mathematical structure of the scalar motion sensor.

Fig. 5
Fig. 5

A, Impulse response; B, amplitude response; C, phase response of the basic temporal filter. The symbols in the center panel are contrast-sensitivity measurements made by Robson21 for a 0.5-cycle/deg grating.

Fig. 6
Fig. 6

Spatial impulse responses of a, main and b, quadrature paths. The spatial impulse response of thee basic spatial filter is equivalent to that of the main path, a. The gray background indicates a value of 0.

Fig. 7
Fig. 7

The Hilbert filter. a, Impulse response. b, Transfer function. The dotted line indicates an imaginary value.

Fig. 8
Fig. 8

Temporal impulse responses of main (solid line) and quadrature (dotted line) paths of the scalar motion sensor.

Fig. 9
Fig. 9

a, Impulse response and b, amplitude response of a scalar sensor for leftward motion (direction = 0). In a, the axes are x (horizontal) and t (vertical). In b, they are u and w.

Fig. 10
Fig. 10

a, Impulse response and b amplitude response of a scalar sensor for motion in direction = 3240. In a, frames show successive time samples of 12.5 msec. In b, frames show successive temporal frequency samples of 5 Hz, with the origin in frame 9.

Fig. 11
Fig. 11

Sensor-response amplitude as a function of the spatial frequency of a moving sinusoidal grating. a, Constant temporal frequency. b, Constant velocity. The shape of the curve is determined by the product of speed and sensor spatial frequency. The three curves are for values of 1 (solid), 16 (dotted), and 32 Hz (dashed).

Fig. 12
Fig. 12

Normalized response amplitude of the scalar motion sensor as a function of the direction of a moving sinusoid [Eq. (38)] with ρ = 0.795.

Fig. 13
Fig. 13

The structure of the vector motion sensor.

Fig. 14
Fig. 14

The shrink algorithm. The original image of width W is denoted C0. Application of a fft results in C ˜ 0. It contains frequencies up to 2−1W. A square core, which contains frequencies up to 2−2W, is inverse transformed to yield a movie C1 that is half as large in each spatial dimension. The procedure can be repeated to yield C2, and so on. The algorithm can be generalized to other size ratios and to the time dimension. This procedure is similar to other pyramid schemes4749 but has the advantage of precisely band limiting the signal before subsampling.

Fig. 15
Fig. 15

Computation of Rk,l, the sensor response at scale k, direction l. Shaded objects are Fourier transforms.

Fig. 16
Fig. 16

Illustration of the frequency-subsampling operator Su. The array at each successive scale is obtained by sampling every other element in u and v dimensions.

Fig. 17
Fig. 17

3D Gaussian blob. The dimensions are W = 32, H = 32, L = 16. The speed is 2 pixels/frame, and the direction is 315°. The spatial spread is two pixels; the temporal spread is eight frames.

Fig. 18
Fig. 18

Amplitude spectrum of the Gaussian blob. The axes are as in Fig. 11b.

Fig. 19
Fig. 19

Response of the scalar sensors of frequency 4 cycles/width and direction −36°.

Fig. 20
Fig. 20

Response of the vector sensors at scale 0 to the Gaussian blob. Each arrow is an estimate of image velocity at the corresponding spatial frequency and location. The contrast of the arrow indicates the strength of the response.

Fig. 21
Fig. 21

Vector-sensor responses to the Gaussian blob at scale 1.

Fig. 22
Fig. 22

Simulated responses to the sum of two gratings of different spatial frequencies that move in different directions. The frequencies were 2 and 8 cycles/width, the directions were 90° and 180°, and the speeds were both 1 pixel/frame. a, Response at the scale of the lower frequency. b, Response at the scale of the higher frequency.

Fig. 23
Fig. 23

Simulation of apparent motion. The input was as shown in Fig. 17 with all but frames 6 and 8 blank. The output at a scale of 4 cycles/width is shown.

Fig. 24
Fig. 24

A sequence of natural images in which two, objects (the hands) move in different directions. The width is 32 pixels.

Fig. 25
Fig. 25

The scalar-sensor responses Rk,l to the input in Fig. 24. Scale, 8 cycles/width, direction, 108°. The first four frames are wrap-arounds from the end of the response.

Fig. 26
Fig. 26

Vector-sensor responses to the hand-waving sequence, a, Output at a scale of 8 cycles/width, b, Output at a scale of 4 cycles/width.

Equations (47)

Equations on this page are rendered with MathJax. Learn more.

c 0 ( x , y , t ) = c 0 ( x , y , 0 )             for all t .
c r ( x , y , t ) = c ( x - r x t , y - r y t , t ) .
a = ( x t ) ,             b = ( u w ) ,
c ( a ) 2 c ˜ ( b ) ,
a = ( x - r t t ) = A a ,             A = [ 1 - r 0 1 ] .
c ( a ) 2 c ˜ [ ( A - 1 ) T b ] ,
( A - 1 ) T = [ 1 0 r 1 ] ,
c ( x - r t , t ) 2 c ˜ ( u , w + r u ) .
c ( x - r x t , y - r y t , t ) 3 c ˜ ( u , v , w + r x u + r y v ) .
w = - r · f = - ( r x u + r y v ) = - r f cos ( θ - α ) ,
r θ = r α cos ( θ - α ) .
f ( t ) = ξ [ f 1 ( t ) - ζ f 2 ( t ) ] ,
f i ( t ) = u ( t ) τ i ( n i - 1 ) ! ( t / τ i ) n i - 1 e - t / τ i
f ˜ ( w ) = ξ [ f 1 ( w ) - ζ f ˜ 2 ( w ) ] ,
f ˜ i ( w ) = ( i 2 π w τ i + 1 ) - n i .
g ( x , y ) = a ( x ) b ( y ) ,
a ( x ) = exp [ - ( x / λ ) 2 ] cos ( 2 π u s x ) ,
b ( y ) = exp [ - ( y / λ ) 2 ] .
g ˜ ( u , v ) = a ˜ ( u ) b ˜ ( v ) ,
a ˜ ( u ) = π λ / 2 ( exp { - [ π λ ( u - u s ) ] 2 } + exp { - [ π λ ( u + u s ) ] 2 } ) ,
b ˜ ( v ) = π λ exp [ - ( π λ v ) 2 ] .
λ = ρ / f .
h ( x ) = - 1 / π x .
h ˜ ( u ) = i sgn ( u ) .
[ h ( x ) * a ( x ) ] b ( y )
i sign ( u ) a ˜ ( u ) b ˜ ( v )
m ˜ m ( u , v , w ) = a ˜ ( u ) b ˜ ( v ) f ˜ ( w ) exp ( - i 2 π w τ ) ,
m ˜ q ( u , v , w ) = - m ˜ m ( u , v , w ) sign ( u ) sign ( w ) .
m ˜ r ( u , v , w ) = m ˜ m + m ˜ q = a ˜ ( u ) b ˜ ( v ) f ˜ ( w ) × exp ( - i 2 π w τ ) [ 1 - sign ( u ) sign ( w ) ] .
u = u cos θ s + v sin θ s ,             v = - u sin θ s + v cos θ s .
m ˜ s ( f , w ) = G { exp [ - ( π λ s - f ) 2 ] + exp [ - ( π λ s + f ) 2 ] } × f ˜ ( w ) exp ( - i 2 π τ w ) [ 1 - sgn ( s · f ) sgn ( w ) ] .
a ( x ) b ( y ) c ( t ) * [ δ ( x , y , t ) + δ ( y ) h ( x ) h ( t ) ] ,
r ( x , y , t ) = c ( x , y , t ) * m ( x , y , t )
r ( x , y , t ) 3 c ˜ ( u , v , w ) m ˜ ( u , v , w ) .
w s = s · r .
G exp [ - ( π λ s - f ) 2 ] f ˜ ( s f ) [ 1 - sign ( s · r ) ] .
G exp { - [ π λ ( f - f s ) ] 2 } f ˜ ( r f ) .
G exp ( - π 2 s 2 [ f s 2 + f 2 - 2 f s f cos ( θ - θ s ) ] ) f ˜ ( r f ) .
exp { - [ 2 π ρ sin ( θ - θ s 2 ) ] 2 } .
exp { - [ π ρ ( θ - θ s ) ] 2 } .
w = r f s cos ( θ - θ s ) ,
C k S h C k + 1 .
M 0 , l fft M ˜ 0 , l ,
M k , l S u M k + 1 , l .
C k fft C ˜ k ,
M ˜ k , l C ˜ k = R ˜ k , l ,
R ˜ k , l fft - 1 R k , l .

Metrics