Abstract

A number of models of depth–cue combination suggest that the final depth percept results from a weighted average of independent depth estimates based on the different cues available. The weight of each cue in such an average is thought to depend on the reliability of each cue. In principle, such a depth estimation could be statistically optimal in the sense of producing the minimum-variance unbiased estimator that can be constructed from the available information. Here we test such models by using visual and haptic depth information. Different texture types produce differences in slant-discrimination performance, thus providing a means for testing a reliability-sensitive cue-combination model with texture as one of the cues to slant. Our results show that the weights for the cues were generally sensitive to their reliability but fell short of statistically optimal combination—we find reliability-based reweighting but not statistically optimal cue combination.

© 2005 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. J. Clark, A. L. Yuille, Data Fusion for Sensory Information Processing Systems (Kluwer, Boston, Mass., 1990).
    [CrossRef]
  2. A. J. Parker, B. G. Cumming, E. B. Johnston, A. C. Hurlbert, “Multiple cues for three-dimensional shape,” in The Cognitive Neurosciences, M. S. Gazzaniga, ed. (MIT, Cambridge, Mass., 1995), pp. 351–364.
  3. L. T. Maloney, M. S. Landy, “A statistical framework for robust fusion of depth information,” in Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. SPIE1199, 1154–1163 (1989).
  4. M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
    [CrossRef] [PubMed]
  5. M. O. Ernst, M. S. Banks, “Humans integrate visual and haptic information in a statistically optimal fashion,” Nature (London) 415, 429–433 (2002).
    [CrossRef]
  6. M. J. Young, M. S. Landy, L. T. Maloney, “A perturbation analysis of depth perception from combinations of texture and motion cues,” Vision Res. 33, 2685–2696 (1993).
    [CrossRef] [PubMed]
  7. M. A. Goodale, C. G. Ellard, L. Booth, “The role of image size and retinal motion in the computation of absolute distance by the Mongolian gerbil (Meriones unguiculatus),” Vision Res. 30, 399–413 (1990).
    [CrossRef] [PubMed]
  8. P. Rosas, F. A. Wichmann, J. Wagemans, “Some observations on the effects of slant and texture type on slant-from-texture,” Vision Res. 44, 1511–1535 (2004).
    [CrossRef] [PubMed]
  9. G. Turk, “Generating textures on arbitrary surfaces using reaction-diffusion,” in Proceedings of the Computer Graphics Conference (SIGGRAPH ’91), T. W. Sederberg, ed. (1991), Vol. 25, No. 4, pp. 289–298 (www.SIGGRAPH.org).
  10. K. Perlin, “An image synthesizer,” Comput. Graph. 19, 287–296 (1985).
    [CrossRef]
  11. A. B. Watson, D. G. Pelli, “QUEST: a Bayesian adaptive psychometric method,” Percept. Psychophys. 33, 113–20 (1983).
    [CrossRef] [PubMed]
  12. F. A. Wichmann, N. J. Hill, “The psychometric function: I. Fitting, sampling, and goodness of fit,” Percept. Psychophys. 63, 1293–1313 (2001).
    [CrossRef]
  13. F. A. Wichmann, N. J. Hill, “The pychometric function: II. Bootstrap-based confidence intervals and sampling,” Percept. Psychophys. 63, 1314–1329 (2001).
    [CrossRef]
  14. F. A. Wichmann, “Some aspects of modelling human spatial vision: Contrast discrimination,” Ph.D. thesis (University of Oxford, Oxford UK, 1999).
  15. D. C. Knill, “Discrimination of planar surface slant from texture: human and ideal observers compared,” Vision Res. 38, 1683–711 (1998).
    [CrossRef] [PubMed]
  16. D. C. Knill, J. A. Saunders, “Do humans optimally integrate stereo and texture information for judgments of surface slant?” Vision Res. 43, 2539–2558 (2003).
    [CrossRef] [PubMed]
  17. I. Oruç, L. T. Maloney, M. S. Landy, “Weighted linear cue combination with possibly correlated error,” Vision Res. 43, 2451–2468 (2003).
    [CrossRef] [PubMed]

2004 (1)

P. Rosas, F. A. Wichmann, J. Wagemans, “Some observations on the effects of slant and texture type on slant-from-texture,” Vision Res. 44, 1511–1535 (2004).
[CrossRef] [PubMed]

2003 (2)

D. C. Knill, J. A. Saunders, “Do humans optimally integrate stereo and texture information for judgments of surface slant?” Vision Res. 43, 2539–2558 (2003).
[CrossRef] [PubMed]

I. Oruç, L. T. Maloney, M. S. Landy, “Weighted linear cue combination with possibly correlated error,” Vision Res. 43, 2451–2468 (2003).
[CrossRef] [PubMed]

2002 (1)

M. O. Ernst, M. S. Banks, “Humans integrate visual and haptic information in a statistically optimal fashion,” Nature (London) 415, 429–433 (2002).
[CrossRef]

2001 (2)

F. A. Wichmann, N. J. Hill, “The psychometric function: I. Fitting, sampling, and goodness of fit,” Percept. Psychophys. 63, 1293–1313 (2001).
[CrossRef]

F. A. Wichmann, N. J. Hill, “The pychometric function: II. Bootstrap-based confidence intervals and sampling,” Percept. Psychophys. 63, 1314–1329 (2001).
[CrossRef]

1998 (1)

D. C. Knill, “Discrimination of planar surface slant from texture: human and ideal observers compared,” Vision Res. 38, 1683–711 (1998).
[CrossRef] [PubMed]

1995 (1)

M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
[CrossRef] [PubMed]

1993 (1)

M. J. Young, M. S. Landy, L. T. Maloney, “A perturbation analysis of depth perception from combinations of texture and motion cues,” Vision Res. 33, 2685–2696 (1993).
[CrossRef] [PubMed]

1990 (1)

M. A. Goodale, C. G. Ellard, L. Booth, “The role of image size and retinal motion in the computation of absolute distance by the Mongolian gerbil (Meriones unguiculatus),” Vision Res. 30, 399–413 (1990).
[CrossRef] [PubMed]

1985 (1)

K. Perlin, “An image synthesizer,” Comput. Graph. 19, 287–296 (1985).
[CrossRef]

1983 (1)

A. B. Watson, D. G. Pelli, “QUEST: a Bayesian adaptive psychometric method,” Percept. Psychophys. 33, 113–20 (1983).
[CrossRef] [PubMed]

Banks, M. S.

M. O. Ernst, M. S. Banks, “Humans integrate visual and haptic information in a statistically optimal fashion,” Nature (London) 415, 429–433 (2002).
[CrossRef]

Booth, L.

M. A. Goodale, C. G. Ellard, L. Booth, “The role of image size and retinal motion in the computation of absolute distance by the Mongolian gerbil (Meriones unguiculatus),” Vision Res. 30, 399–413 (1990).
[CrossRef] [PubMed]

Clark, J. J.

J. J. Clark, A. L. Yuille, Data Fusion for Sensory Information Processing Systems (Kluwer, Boston, Mass., 1990).
[CrossRef]

Cumming, B. G.

A. J. Parker, B. G. Cumming, E. B. Johnston, A. C. Hurlbert, “Multiple cues for three-dimensional shape,” in The Cognitive Neurosciences, M. S. Gazzaniga, ed. (MIT, Cambridge, Mass., 1995), pp. 351–364.

Ellard, C. G.

M. A. Goodale, C. G. Ellard, L. Booth, “The role of image size and retinal motion in the computation of absolute distance by the Mongolian gerbil (Meriones unguiculatus),” Vision Res. 30, 399–413 (1990).
[CrossRef] [PubMed]

Ernst, M. O.

M. O. Ernst, M. S. Banks, “Humans integrate visual and haptic information in a statistically optimal fashion,” Nature (London) 415, 429–433 (2002).
[CrossRef]

Goodale, M. A.

M. A. Goodale, C. G. Ellard, L. Booth, “The role of image size and retinal motion in the computation of absolute distance by the Mongolian gerbil (Meriones unguiculatus),” Vision Res. 30, 399–413 (1990).
[CrossRef] [PubMed]

Hill, N. J.

F. A. Wichmann, N. J. Hill, “The pychometric function: II. Bootstrap-based confidence intervals and sampling,” Percept. Psychophys. 63, 1314–1329 (2001).
[CrossRef]

F. A. Wichmann, N. J. Hill, “The psychometric function: I. Fitting, sampling, and goodness of fit,” Percept. Psychophys. 63, 1293–1313 (2001).
[CrossRef]

Hurlbert, A. C.

A. J. Parker, B. G. Cumming, E. B. Johnston, A. C. Hurlbert, “Multiple cues for three-dimensional shape,” in The Cognitive Neurosciences, M. S. Gazzaniga, ed. (MIT, Cambridge, Mass., 1995), pp. 351–364.

Johnston, E. B.

M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
[CrossRef] [PubMed]

A. J. Parker, B. G. Cumming, E. B. Johnston, A. C. Hurlbert, “Multiple cues for three-dimensional shape,” in The Cognitive Neurosciences, M. S. Gazzaniga, ed. (MIT, Cambridge, Mass., 1995), pp. 351–364.

Knill, D. C.

D. C. Knill, J. A. Saunders, “Do humans optimally integrate stereo and texture information for judgments of surface slant?” Vision Res. 43, 2539–2558 (2003).
[CrossRef] [PubMed]

D. C. Knill, “Discrimination of planar surface slant from texture: human and ideal observers compared,” Vision Res. 38, 1683–711 (1998).
[CrossRef] [PubMed]

Landy, M. S.

I. Oruç, L. T. Maloney, M. S. Landy, “Weighted linear cue combination with possibly correlated error,” Vision Res. 43, 2451–2468 (2003).
[CrossRef] [PubMed]

M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
[CrossRef] [PubMed]

M. J. Young, M. S. Landy, L. T. Maloney, “A perturbation analysis of depth perception from combinations of texture and motion cues,” Vision Res. 33, 2685–2696 (1993).
[CrossRef] [PubMed]

L. T. Maloney, M. S. Landy, “A statistical framework for robust fusion of depth information,” in Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. SPIE1199, 1154–1163 (1989).

Maloney, L. T.

I. Oruç, L. T. Maloney, M. S. Landy, “Weighted linear cue combination with possibly correlated error,” Vision Res. 43, 2451–2468 (2003).
[CrossRef] [PubMed]

M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
[CrossRef] [PubMed]

M. J. Young, M. S. Landy, L. T. Maloney, “A perturbation analysis of depth perception from combinations of texture and motion cues,” Vision Res. 33, 2685–2696 (1993).
[CrossRef] [PubMed]

L. T. Maloney, M. S. Landy, “A statistical framework for robust fusion of depth information,” in Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. SPIE1199, 1154–1163 (1989).

Oruç, I.

I. Oruç, L. T. Maloney, M. S. Landy, “Weighted linear cue combination with possibly correlated error,” Vision Res. 43, 2451–2468 (2003).
[CrossRef] [PubMed]

Parker, A. J.

A. J. Parker, B. G. Cumming, E. B. Johnston, A. C. Hurlbert, “Multiple cues for three-dimensional shape,” in The Cognitive Neurosciences, M. S. Gazzaniga, ed. (MIT, Cambridge, Mass., 1995), pp. 351–364.

Pelli, D. G.

A. B. Watson, D. G. Pelli, “QUEST: a Bayesian adaptive psychometric method,” Percept. Psychophys. 33, 113–20 (1983).
[CrossRef] [PubMed]

Perlin, K.

K. Perlin, “An image synthesizer,” Comput. Graph. 19, 287–296 (1985).
[CrossRef]

Rosas, P.

P. Rosas, F. A. Wichmann, J. Wagemans, “Some observations on the effects of slant and texture type on slant-from-texture,” Vision Res. 44, 1511–1535 (2004).
[CrossRef] [PubMed]

Saunders, J. A.

D. C. Knill, J. A. Saunders, “Do humans optimally integrate stereo and texture information for judgments of surface slant?” Vision Res. 43, 2539–2558 (2003).
[CrossRef] [PubMed]

Turk, G.

G. Turk, “Generating textures on arbitrary surfaces using reaction-diffusion,” in Proceedings of the Computer Graphics Conference (SIGGRAPH ’91), T. W. Sederberg, ed. (1991), Vol. 25, No. 4, pp. 289–298 (www.SIGGRAPH.org).

Wagemans, J.

P. Rosas, F. A. Wichmann, J. Wagemans, “Some observations on the effects of slant and texture type on slant-from-texture,” Vision Res. 44, 1511–1535 (2004).
[CrossRef] [PubMed]

Watson, A. B.

A. B. Watson, D. G. Pelli, “QUEST: a Bayesian adaptive psychometric method,” Percept. Psychophys. 33, 113–20 (1983).
[CrossRef] [PubMed]

Wichmann, F. A.

P. Rosas, F. A. Wichmann, J. Wagemans, “Some observations on the effects of slant and texture type on slant-from-texture,” Vision Res. 44, 1511–1535 (2004).
[CrossRef] [PubMed]

F. A. Wichmann, N. J. Hill, “The psychometric function: I. Fitting, sampling, and goodness of fit,” Percept. Psychophys. 63, 1293–1313 (2001).
[CrossRef]

F. A. Wichmann, N. J. Hill, “The pychometric function: II. Bootstrap-based confidence intervals and sampling,” Percept. Psychophys. 63, 1314–1329 (2001).
[CrossRef]

F. A. Wichmann, “Some aspects of modelling human spatial vision: Contrast discrimination,” Ph.D. thesis (University of Oxford, Oxford UK, 1999).

Young, M.

M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
[CrossRef] [PubMed]

Young, M. J.

M. J. Young, M. S. Landy, L. T. Maloney, “A perturbation analysis of depth perception from combinations of texture and motion cues,” Vision Res. 33, 2685–2696 (1993).
[CrossRef] [PubMed]

Yuille, A. L.

J. J. Clark, A. L. Yuille, Data Fusion for Sensory Information Processing Systems (Kluwer, Boston, Mass., 1990).
[CrossRef]

Comput. Graph. (1)

K. Perlin, “An image synthesizer,” Comput. Graph. 19, 287–296 (1985).
[CrossRef]

Nature (London) (1)

M. O. Ernst, M. S. Banks, “Humans integrate visual and haptic information in a statistically optimal fashion,” Nature (London) 415, 429–433 (2002).
[CrossRef]

Percept. Psychophys. (3)

A. B. Watson, D. G. Pelli, “QUEST: a Bayesian adaptive psychometric method,” Percept. Psychophys. 33, 113–20 (1983).
[CrossRef] [PubMed]

F. A. Wichmann, N. J. Hill, “The psychometric function: I. Fitting, sampling, and goodness of fit,” Percept. Psychophys. 63, 1293–1313 (2001).
[CrossRef]

F. A. Wichmann, N. J. Hill, “The pychometric function: II. Bootstrap-based confidence intervals and sampling,” Percept. Psychophys. 63, 1314–1329 (2001).
[CrossRef]

Vision Res. (7)

M. J. Young, M. S. Landy, L. T. Maloney, “A perturbation analysis of depth perception from combinations of texture and motion cues,” Vision Res. 33, 2685–2696 (1993).
[CrossRef] [PubMed]

M. A. Goodale, C. G. Ellard, L. Booth, “The role of image size and retinal motion in the computation of absolute distance by the Mongolian gerbil (Meriones unguiculatus),” Vision Res. 30, 399–413 (1990).
[CrossRef] [PubMed]

P. Rosas, F. A. Wichmann, J. Wagemans, “Some observations on the effects of slant and texture type on slant-from-texture,” Vision Res. 44, 1511–1535 (2004).
[CrossRef] [PubMed]

D. C. Knill, “Discrimination of planar surface slant from texture: human and ideal observers compared,” Vision Res. 38, 1683–711 (1998).
[CrossRef] [PubMed]

D. C. Knill, J. A. Saunders, “Do humans optimally integrate stereo and texture information for judgments of surface slant?” Vision Res. 43, 2539–2558 (2003).
[CrossRef] [PubMed]

I. Oruç, L. T. Maloney, M. S. Landy, “Weighted linear cue combination with possibly correlated error,” Vision Res. 43, 2451–2468 (2003).
[CrossRef] [PubMed]

M. S. Landy, L. T. Maloney, E. B. Johnston, M. Young, “Measurement and modeling of depth cue combination: in defense of weak fusion,” Vision Res. 35, 389–412 (1995).
[CrossRef] [PubMed]

Other (5)

J. J. Clark, A. L. Yuille, Data Fusion for Sensory Information Processing Systems (Kluwer, Boston, Mass., 1990).
[CrossRef]

A. J. Parker, B. G. Cumming, E. B. Johnston, A. C. Hurlbert, “Multiple cues for three-dimensional shape,” in The Cognitive Neurosciences, M. S. Gazzaniga, ed. (MIT, Cambridge, Mass., 1995), pp. 351–364.

L. T. Maloney, M. S. Landy, “A statistical framework for robust fusion of depth information,” in Visual Communications and Image Processing IV, W. A. Pearlman, ed., Proc. SPIE1199, 1154–1163 (1989).

G. Turk, “Generating textures on arbitrary surfaces using reaction-diffusion,” in Proceedings of the Computer Graphics Conference (SIGGRAPH ’91), T. W. Sederberg, ed. (1991), Vol. 25, No. 4, pp. 289–298 (www.SIGGRAPH.org).

F. A. Wichmann, “Some aspects of modelling human spatial vision: Contrast discrimination,” Ph.D. thesis (University of Oxford, Oxford UK, 1999).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Effect of texture type on slant discrimination. Top, the psychometric functions are significantly different for three texture types (error bars represent 68% confidence intervals). Bottom, texture patterns used to obtain the data. Left to right: circles, leopard skinlike, and Perlin noise textures. For this subject (TV) the task was easier when the circle texture was mapped onto the slanted planes while discriminating slant near 40   deg (the standard, depicted as a solid vertical line), reflected in the steepest psychometric function. Her worst performance was obtained with Perlin noise.

Fig. 2
Fig. 2

Examples of the psychometric functions for slant discrimination obtained for texture only (solid gray curve for the fitted function, gray squares for data), haptic only (solid black curve for fit, black triangles for data) and texture and haptic cues (dashed black curve for fit, black circles for data) for subjects NK, PR, and TV. On each plot, the horizontal axis represents slant in degrees and the vertical axis shows performance as the fraction of the comparison stimuli perceived as more slanted than the standard. Standards are indicated with a vertical solid line. Error bars represent 68% confidence intervals. The size of the data points is proportional to the number of trials recorded.

Fig. 3
Fig. 3

Measured and predicted thresholds for slant discrimination with haptic and texture information provided by three different texture types for all five subjects. The vertical axes contain the thresholds in degrees on a log scale, and the texture types are on the horizontal axes. Error bars represent 68% confidence intervals estimated by bootstrap. Thresholds are defined as the difference between the stimulus judged 84% of the trials as more slanted and the PSE. (Fig. 3 continued next page.)

Fig. 4
Fig. 4

Example of the effect of the perturbation analysis for subject TV. Top left, effect of introducing a discrepancy in the slant depicted by the texture cue while the haptic cue was depicting 40   deg . Top right, reverse, that is, perturbing the haptic cue while the texture cue was fixed at 40   deg slant. Bottom, PSEs obtained from the psychometric functions of the top row. The slope (obtained by weighted-square-root linear fit) represents the weight given to the perturbed cue. Error bars represent 95% confidence intervals. This figure is analogous to, for example, Fig. 7 of Landy et al.[4]

Fig. 5
Fig. 5

Estimated weights as the slope of change in the PSE given a change in the slant depicted by the perturbed cue for subject TV: left column, perturbed texture; right column, perturbed haptic; one texture type per row. Error bars represent 95% confidence intervals obtained by bootstrap.

Tables (1)

Tables Icon

Table 1 Measured and Predicted Weights for the Texture Cue

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

ψ ( x , α , β , λ ) = λ + ( 1 λ ) F ( x , α , β ) ,
τ th 2 = τ t 2 τ h 2 τ t 2 + τ h 2 .
ω t p = τ h 2 τ t 2 + τ h 2 .

Metrics