Abstract

The application of multivariate techniques to neuroimaging and electrophysiological data has greatly enhanced the ability to detect where, when, and how functional neural information is processed during a variety of behavioral tasks. With the extension to single-trial analysis, neuroscientists are able to relate brain states to perceptual, cognitive, and motor processes. Using pattern classification methods, the neuroscientist can extract neural performance measures in a manner analogous to human behavioral performance, allowing for a consistent information content metric across measurement modalities. However, as with behavioral psychophysical performance, pattern classifier performances are a product of both the task-relevant information inherent in the brain and in the task/stimuli. Here, we argue for the use of an ideal observer framework with which the researcher can effectively normalize the observed neural performance given the task’s inherent objective difficulty. We use data from a face versus car discrimination task and compare classifier performance applied to electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data with corresponding human behavior through the absolute and relative efficiency metrics. We show that confounding variables that can lead to erroneous interpretations of information content can be accounted for through comparisons to an ideal observer, allowing for more confident interpretation of the neural mechanisms involved in the task of interest. Finally, we discuss limitations of interpretation due to the transduction of indirect measures of neural activity, underlying assumptions in the optimality of the pattern classifiers, and dependence of efficiency results on signal contrast.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. Further assumptions are well-documented and not the focus of this paper, such as the exact relation between neuronal firing and the observable measurement provided by the specific imaging modality. For our purposes we will assume (somewhat safely, given the large and growing body of evidence) that the imaging observables represent some direct transformation of underlying neural activity (fMRI: below; EEG: below; MEG: below).
  2. J. Haynes and G. Rees, “Predicting the orientation of invisible stimuli from activity in human primary visual cortex,” Nat. Neurosci. 8, 686–691 (2005).
    [CrossRef] [PubMed]
  3. H. Barlow, “Single units and sensation: A neuron doctrine for perceptual psychology?” Perception 1, 371–394 (1972).
    [CrossRef] [PubMed]
  4. K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
    [PubMed]
  5. J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
    [CrossRef] [PubMed]
  6. With spatially coarse measures such as fMRI and ERP the local interactions are pooled together and lost. However, long-range connections can be monitored between the large groups of neurons represented in each voxel or electrode.
  7. R. Duda, P. Hart, and D. Storke, Pattern Classification (Wiley, 2001).
  8. Y. Kamitani and F. Tong, “Decoding the visual and subjective contents of the human brain,” Nat. Neurosci. 8, 679–685 (2005).
    [CrossRef] [PubMed]
  9. J. Haynes and G. Rees, “Decoding mental states from brain activity in humans,” Nat. Rev. Neurosci. 7, 523–534 (2006).
    [CrossRef] [PubMed]
  10. D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
    [CrossRef] [PubMed]
  11. T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
    [CrossRef] [PubMed]
  12. L. Pessoa and S. Padmala, “Quantitative prediction of perceptual decisions during near-threshold fear detection,” Proc. Natl. Acad. Sci. U.S.A. 102, 5612–5617 (2005).
    [CrossRef] [PubMed]
  13. M. Philiastides and P. Sajda, “Temporal characterization of the neural correlates of perceptual decision making in the human brain,” Cereb. Cortex 16, 509–518 (2006).
    [CrossRef]
  14. T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
    [CrossRef] [PubMed]
  15. R. Ratcliff, M. Philiastides, and P. Sajda, “Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG,” Proc. Natl. Acad. Sci. U.S.A. 106, 6539–6544 (2009).
    [CrossRef] [PubMed]
  16. S. Li, S. Mayhew, and Z. Kourtzi, “Learning shapes the representation of behavioral choice in the human brain,” Neuron 62, 441–452 (2009).
    [CrossRef] [PubMed]
  17. D. Green and J. Swets, Signal Detection Theory and Psychophysics (Wiley, 1966).
  18. J. Solomon and D. Pelli, “The visual filter mediating letter identification,” Nature (London) 369, 395–397 (1994).
    [CrossRef]
  19. B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
    [CrossRef] [PubMed]
  20. J. Gold, P. Bennett, and A. Sekuler, “Identification of band-pass filtered letters and faces by human and ideal observers,” Vision Res. 39, 3537–3560 (1999).
    [CrossRef]
  21. W. Geisler, “Ideal observer analysis,” in The Visual Neurosciences, L.Chalupa and J.Werner, eds. (MIT Press, 2003), pp. 825–837.
  22. J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
    [CrossRef] [PubMed]
  23. Z. Shang and T. Sim, “When Fisher meets Fukunaga-Koontz: A new look at linear discriminants,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 323–329.
  24. N. Kanwisher, J. McDermott, and M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci. 17, 4302–4311 (1997).
    [PubMed]
  25. W. Peterson, T. Birdsall, and W. Fox, “The theory of signal detectability,” IEEE Trans. Inf. Theory 4, 171–212 (1954).
    [CrossRef]
  26. R. McDonough and A. Whalen, Detection of Signals in Noise (Academic, 1995).
  27. This amounts to a spatial deconvolution, or, equivalently, multiplying by the inverse of the covariance matrix per Eq. .
  28. H. Barlow, “The absolute efficiency of perceptual decisions,” Philos. Trans. R. Soc. London Ser. B 290, 71–91 (1980).
    [CrossRef]
  29. Cclassifier is equivalent to Chuman as the human’s behavioral response and brain activation are derived from the same displayed stimulus.
  30. M. Eckstein, B. Beutter, and L. Stone, “Quantifying the performance limits of human saccadic targeting during visual search,” Perception 30, 1389–1401 (2001).
    [CrossRef]
  31. In addition, there were also procedural differences: stimulus presentation time, display luminance, image size on the retina (4.57° vs. 5.13°). However, given that the task is limited by external noise, it is likely that these procedural differences might result in a smaller performance difference than one might expect from noiseless displays.
  32. W. Geisler, “Sequential ideal-observer analysis of visual discriminations,” Psychol. Rev. 96, 267–314 (1989).
    [CrossRef] [PubMed]
  33. Y. Zhang, B. Pham, and M. Eckstein, “The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds,” IEEE Trans. Med. Imaging 25, 1348–1362 (2006).
    [CrossRef] [PubMed]
  34. B. Tjan and A. Nandy, “Classification images with uncertainty,” J. Vision 6, 387–413 (2006).
    [CrossRef]
  35. Here, the imaging data can be thought of as coming from two classes, face present and car present. On top of the category-specific activity is the imaging noise which has been shown to be highly Gaussian for fMRI data . Thus, the data themselves can be thought of as a Gaussian noise process with a mean displaced by the category-specific activation.
  36. C. Chen, C. Tyler, and H. Baseler, “Statistical properties of BOLD magnetic resonance activity in the human brain,” Neuroimage 20, 1096–1109 (2003).
    [CrossRef] [PubMed]
  37. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
    [CrossRef]
  38. T. Cover and J. Thomas, Elements of Information Theory (Wiley, 1991).
    [CrossRef]
  39. D. Heeger and D. Ress, “What does fMRI tell us about neuronal activity?” Nat. Rev. Neurosci. 3, 142–151 (2002).
    [CrossRef] [PubMed]
  40. G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
    [PubMed]
  41. S. Zhang and T. Sim, “Discriminant subspace analysis: a Fukunaga-Koontz approach,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 1732–1745 (2007).
    [CrossRef] [PubMed]
  42. V. Vapnik, Statistical Learning Theory (Wiley, 1998).
  43. K. Das and Z. Nenadic, “An efficient discriminant-based solution for small sample size problem,” Pattern Recogn. Lett. 42, 857–866 (2009).
  44. K. Das and Z. Nenadic, “Approximate information discriminant analysis: a computationally simple heteroscedastic feature extraction technique,” Pattern Recogn. Lett. 41, 1565–1574 (2008).
  45. N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
    [CrossRef]
  46. P. Nunez, Electrical Fields of the Brain: the Neurophysics of EEG (Oxford University Press, 1981).
  47. Y. Okada, “Neurogenesis of evoked magnetic fields,” in Biomagnetism: an Interdisciplinary Approach, S.Williamson, ed. (Plenum, 1983), pp. 399–408.

2009 (3)

R. Ratcliff, M. Philiastides, and P. Sajda, “Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG,” Proc. Natl. Acad. Sci. U.S.A. 106, 6539–6544 (2009).
[CrossRef] [PubMed]

S. Li, S. Mayhew, and Z. Kourtzi, “Learning shapes the representation of behavioral choice in the human brain,” Neuron 62, 441–452 (2009).
[CrossRef] [PubMed]

K. Das and Z. Nenadic, “An efficient discriminant-based solution for small sample size problem,” Pattern Recogn. Lett. 42, 857–866 (2009).

2008 (4)

K. Das and Z. Nenadic, “Approximate information discriminant analysis: a computationally simple heteroscedastic feature extraction technique,” Pattern Recogn. Lett. 41, 1565–1574 (2008).

D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
[CrossRef] [PubMed]

T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
[CrossRef] [PubMed]

J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
[CrossRef] [PubMed]

2007 (2)

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

S. Zhang and T. Sim, “Discriminant subspace analysis: a Fukunaga-Koontz approach,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 1732–1745 (2007).
[CrossRef] [PubMed]

2006 (5)

J. Haynes and G. Rees, “Decoding mental states from brain activity in humans,” Nat. Rev. Neurosci. 7, 523–534 (2006).
[CrossRef] [PubMed]

M. Philiastides and P. Sajda, “Temporal characterization of the neural correlates of perceptual decision making in the human brain,” Cereb. Cortex 16, 509–518 (2006).
[CrossRef]

Z. Shang and T. Sim, “When Fisher meets Fukunaga-Koontz: A new look at linear discriminants,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 323–329.

Y. Zhang, B. Pham, and M. Eckstein, “The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds,” IEEE Trans. Med. Imaging 25, 1348–1362 (2006).
[CrossRef] [PubMed]

B. Tjan and A. Nandy, “Classification images with uncertainty,” J. Vision 6, 387–413 (2006).
[CrossRef]

2005 (3)

L. Pessoa and S. Padmala, “Quantitative prediction of perceptual decisions during near-threshold fear detection,” Proc. Natl. Acad. Sci. U.S.A. 102, 5612–5617 (2005).
[CrossRef] [PubMed]

Y. Kamitani and F. Tong, “Decoding the visual and subjective contents of the human brain,” Nat. Neurosci. 8, 679–685 (2005).
[CrossRef] [PubMed]

J. Haynes and G. Rees, “Predicting the orientation of invisible stimuli from activity in human primary visual cortex,” Nat. Neurosci. 8, 686–691 (2005).
[CrossRef] [PubMed]

2003 (2)

C. Chen, C. Tyler, and H. Baseler, “Statistical properties of BOLD magnetic resonance activity in the human brain,” Neuroimage 20, 1096–1109 (2003).
[CrossRef] [PubMed]

W. Geisler, “Ideal observer analysis,” in The Visual Neurosciences, L.Chalupa and J.Werner, eds. (MIT Press, 2003), pp. 825–837.

2002 (1)

D. Heeger and D. Ress, “What does fMRI tell us about neuronal activity?” Nat. Rev. Neurosci. 3, 142–151 (2002).
[CrossRef] [PubMed]

2001 (4)

M. Eckstein, B. Beutter, and L. Stone, “Quantifying the performance limits of human saccadic targeting during visual search,” Perception 30, 1389–1401 (2001).
[CrossRef]

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

R. Duda, P. Hart, and D. Storke, Pattern Classification (Wiley, 2001).

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

1999 (1)

J. Gold, P. Bennett, and A. Sekuler, “Identification of band-pass filtered letters and faces by human and ideal observers,” Vision Res. 39, 3537–3560 (1999).
[CrossRef]

1998 (1)

V. Vapnik, Statistical Learning Theory (Wiley, 1998).

1997 (1)

N. Kanwisher, J. McDermott, and M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci. 17, 4302–4311 (1997).
[PubMed]

1996 (1)

G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
[PubMed]

1995 (2)

R. McDonough and A. Whalen, Detection of Signals in Noise (Academic, 1995).

B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
[CrossRef] [PubMed]

1994 (1)

J. Solomon and D. Pelli, “The visual filter mediating letter identification,” Nature (London) 369, 395–397 (1994).
[CrossRef]

1992 (1)

K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
[PubMed]

1991 (1)

T. Cover and J. Thomas, Elements of Information Theory (Wiley, 1991).
[CrossRef]

1989 (1)

W. Geisler, “Sequential ideal-observer analysis of visual discriminations,” Psychol. Rev. 96, 267–314 (1989).
[CrossRef] [PubMed]

1987 (1)

A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
[CrossRef]

1983 (1)

Y. Okada, “Neurogenesis of evoked magnetic fields,” in Biomagnetism: an Interdisciplinary Approach, S.Williamson, ed. (Plenum, 1983), pp. 399–408.

1981 (1)

P. Nunez, Electrical Fields of the Brain: the Neurophysics of EEG (Oxford University Press, 1981).

1980 (1)

H. Barlow, “The absolute efficiency of perceptual decisions,” Philos. Trans. R. Soc. London Ser. B 290, 71–91 (1980).
[CrossRef]

1972 (1)

H. Barlow, “Single units and sensation: A neuron doctrine for perceptual psychology?” Perception 1, 371–394 (1972).
[CrossRef] [PubMed]

1966 (1)

D. Green and J. Swets, Signal Detection Theory and Psychophysics (Wiley, 1966).

1954 (1)

W. Peterson, T. Birdsall, and W. Fox, “The theory of signal detectability,” IEEE Trans. Inf. Theory 4, 171–212 (1954).
[CrossRef]

Augath, M.

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

Barlow, H.

H. Barlow, “The absolute efficiency of perceptual decisions,” Philos. Trans. R. Soc. London Ser. B 290, 71–91 (1980).
[CrossRef]

H. Barlow, “Single units and sensation: A neuron doctrine for perceptual psychology?” Perception 1, 371–394 (1972).
[CrossRef] [PubMed]

Baseler, H.

C. Chen, C. Tyler, and H. Baseler, “Statistical properties of BOLD magnetic resonance activity in the human brain,” Neuroimage 20, 1096–1109 (2003).
[CrossRef] [PubMed]

Bauer, M.

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

Bennett, P.

J. Gold, P. Bennett, and A. Sekuler, “Identification of band-pass filtered letters and faces by human and ideal observers,” Vision Res. 39, 3537–3560 (1999).
[CrossRef]

Beutter, B.

M. Eckstein, B. Beutter, and L. Stone, “Quantifying the performance limits of human saccadic targeting during visual search,” Perception 30, 1389–1401 (2001).
[CrossRef]

Birdsall, T.

W. Peterson, T. Birdsall, and W. Fox, “The theory of signal detectability,” IEEE Trans. Inf. Theory 4, 171–212 (1954).
[CrossRef]

Blake, R.

J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
[CrossRef] [PubMed]

Blumer, A.

A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
[CrossRef]

Boynton, G.

G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
[PubMed]

Braje, W.

B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
[CrossRef] [PubMed]

Britten, K.

K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
[PubMed]

Chen, C.

C. Chen, C. Tyler, and H. Baseler, “Statistical properties of BOLD magnetic resonance activity in the human brain,” Neuroimage 20, 1096–1109 (2003).
[CrossRef] [PubMed]

Chun, M.

N. Kanwisher, J. McDermott, and M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci. 17, 4302–4311 (1997).
[PubMed]

Cook, S.

J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
[CrossRef] [PubMed]

Cover, T.

T. Cover and J. Thomas, Elements of Information Theory (Wiley, 1991).
[CrossRef]

Das, K.

K. Das and Z. Nenadic, “An efficient discriminant-based solution for small sample size problem,” Pattern Recogn. Lett. 42, 857–866 (2009).

K. Das and Z. Nenadic, “Approximate information discriminant analysis: a computationally simple heteroscedastic feature extraction technique,” Pattern Recogn. Lett. 41, 1565–1574 (2008).

Donner, T.

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

Duda, R.

R. Duda, P. Hart, and D. Storke, Pattern Classification (Wiley, 2001).

Eckstein, M.

Y. Zhang, B. Pham, and M. Eckstein, “The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds,” IEEE Trans. Med. Imaging 25, 1348–1362 (2006).
[CrossRef] [PubMed]

M. Eckstein, B. Beutter, and L. Stone, “Quantifying the performance limits of human saccadic targeting during visual search,” Perception 30, 1389–1401 (2001).
[CrossRef]

Ehrenfeucht, A.

A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
[CrossRef]

Engel, A.

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

Engel, S.

G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
[PubMed]

Fox, W.

W. Peterson, T. Birdsall, and W. Fox, “The theory of signal detectability,” IEEE Trans. Inf. Theory 4, 171–212 (1954).
[CrossRef]

Fries, P.

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

Furey, M.

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Geisler, W.

W. Geisler, “Ideal observer analysis,” in The Visual Neurosciences, L.Chalupa and J.Werner, eds. (MIT Press, 2003), pp. 825–837.

W. Geisler, “Sequential ideal-observer analysis of visual discriminations,” Psychol. Rev. 96, 267–314 (1989).
[CrossRef] [PubMed]

Glover, G.

G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
[PubMed]

Gobbini, M.

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Gold, J.

J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
[CrossRef] [PubMed]

J. Gold, P. Bennett, and A. Sekuler, “Identification of band-pass filtered letters and faces by human and ideal observers,” Vision Res. 39, 3537–3560 (1999).
[CrossRef]

Green, D.

D. Green and J. Swets, Signal Detection Theory and Psychophysics (Wiley, 1966).

Hart, P.

R. Duda, P. Hart, and D. Storke, Pattern Classification (Wiley, 2001).

Haussler, D.

A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
[CrossRef]

Haxby, J.

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Haynes, J.

J. Haynes and G. Rees, “Decoding mental states from brain activity in humans,” Nat. Rev. Neurosci. 7, 523–534 (2006).
[CrossRef] [PubMed]

J. Haynes and G. Rees, “Predicting the orientation of invisible stimuli from activity in human primary visual cortex,” Nat. Neurosci. 8, 686–691 (2005).
[CrossRef] [PubMed]

Heeger, D.

D. Heeger and D. Ress, “What does fMRI tell us about neuronal activity?” Nat. Rev. Neurosci. 3, 142–151 (2002).
[CrossRef] [PubMed]

G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
[PubMed]

Ishai, A.

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Kamitani, Y.

Y. Kamitani and F. Tong, “Decoding the visual and subjective contents of the human brain,” Nat. Neurosci. 8, 679–685 (2005).
[CrossRef] [PubMed]

Kanwisher, N.

N. Kanwisher, J. McDermott, and M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci. 17, 4302–4311 (1997).
[PubMed]

Kersten, D.

B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
[CrossRef] [PubMed]

Kourtzi, Z.

S. Li, S. Mayhew, and Z. Kourtzi, “Learning shapes the representation of behavioral choice in the human brain,” Neuron 62, 441–452 (2009).
[CrossRef] [PubMed]

D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
[CrossRef] [PubMed]

T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
[CrossRef] [PubMed]

Lam, J.

D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
[CrossRef] [PubMed]

Legge, G.

B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
[CrossRef] [PubMed]

Li, S.

S. Li, S. Mayhew, and Z. Kourtzi, “Learning shapes the representation of behavioral choice in the human brain,” Neuron 62, 441–452 (2009).
[CrossRef] [PubMed]

D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
[CrossRef] [PubMed]

T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
[CrossRef] [PubMed]

Logothetis, N.

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

Mayhew, S.

S. Li, S. Mayhew, and Z. Kourtzi, “Learning shapes the representation of behavioral choice in the human brain,” Neuron 62, 441–452 (2009).
[CrossRef] [PubMed]

McDermott, J.

N. Kanwisher, J. McDermott, and M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci. 17, 4302–4311 (1997).
[PubMed]

McDonough, R.

R. McDonough and A. Whalen, Detection of Signals in Noise (Academic, 1995).

Movshon, J.

K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
[PubMed]

Nandy, A.

B. Tjan and A. Nandy, “Classification images with uncertainty,” J. Vision 6, 387–413 (2006).
[CrossRef]

Nenadic, Z.

K. Das and Z. Nenadic, “An efficient discriminant-based solution for small sample size problem,” Pattern Recogn. Lett. 42, 857–866 (2009).

K. Das and Z. Nenadic, “Approximate information discriminant analysis: a computationally simple heteroscedastic feature extraction technique,” Pattern Recogn. Lett. 41, 1565–1574 (2008).

Newsome, W.

K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
[PubMed]

Nunez, P.

P. Nunez, Electrical Fields of the Brain: the Neurophysics of EEG (Oxford University Press, 1981).

Oeltermann, A.

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

Okada, Y.

Y. Okada, “Neurogenesis of evoked magnetic fields,” in Biomagnetism: an Interdisciplinary Approach, S.Williamson, ed. (Plenum, 1983), pp. 399–408.

Oostenveld, R.

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

Ostwald, D.

D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
[CrossRef] [PubMed]

Padmala, S.

L. Pessoa and S. Padmala, “Quantitative prediction of perceptual decisions during near-threshold fear detection,” Proc. Natl. Acad. Sci. U.S.A. 102, 5612–5617 (2005).
[CrossRef] [PubMed]

Pauls, J.

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

Pelli, D.

J. Solomon and D. Pelli, “The visual filter mediating letter identification,” Nature (London) 369, 395–397 (1994).
[CrossRef]

Pessoa, L.

L. Pessoa and S. Padmala, “Quantitative prediction of perceptual decisions during near-threshold fear detection,” Proc. Natl. Acad. Sci. U.S.A. 102, 5612–5617 (2005).
[CrossRef] [PubMed]

Peterson, W.

W. Peterson, T. Birdsall, and W. Fox, “The theory of signal detectability,” IEEE Trans. Inf. Theory 4, 171–212 (1954).
[CrossRef]

Pham, B.

Y. Zhang, B. Pham, and M. Eckstein, “The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds,” IEEE Trans. Med. Imaging 25, 1348–1362 (2006).
[CrossRef] [PubMed]

Philiastides, M.

R. Ratcliff, M. Philiastides, and P. Sajda, “Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG,” Proc. Natl. Acad. Sci. U.S.A. 106, 6539–6544 (2009).
[CrossRef] [PubMed]

M. Philiastides and P. Sajda, “Temporal characterization of the neural correlates of perceptual decision making in the human brain,” Cereb. Cortex 16, 509–518 (2006).
[CrossRef]

Pietrini, P.

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Preston, T.

T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
[CrossRef] [PubMed]

Ratcliff, R.

R. Ratcliff, M. Philiastides, and P. Sajda, “Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG,” Proc. Natl. Acad. Sci. U.S.A. 106, 6539–6544 (2009).
[CrossRef] [PubMed]

Rees, G.

J. Haynes and G. Rees, “Decoding mental states from brain activity in humans,” Nat. Rev. Neurosci. 7, 523–534 (2006).
[CrossRef] [PubMed]

J. Haynes and G. Rees, “Predicting the orientation of invisible stimuli from activity in human primary visual cortex,” Nat. Neurosci. 8, 686–691 (2005).
[CrossRef] [PubMed]

Ress, D.

D. Heeger and D. Ress, “What does fMRI tell us about neuronal activity?” Nat. Rev. Neurosci. 3, 142–151 (2002).
[CrossRef] [PubMed]

Sajda, P.

R. Ratcliff, M. Philiastides, and P. Sajda, “Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG,” Proc. Natl. Acad. Sci. U.S.A. 106, 6539–6544 (2009).
[CrossRef] [PubMed]

M. Philiastides and P. Sajda, “Temporal characterization of the neural correlates of perceptual decision making in the human brain,” Cereb. Cortex 16, 509–518 (2006).
[CrossRef]

Schouten, J.

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Sekuler, A.

J. Gold, P. Bennett, and A. Sekuler, “Identification of band-pass filtered letters and faces by human and ideal observers,” Vision Res. 39, 3537–3560 (1999).
[CrossRef]

Shadlen, M.

K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
[PubMed]

Shang, Z.

Z. Shang and T. Sim, “When Fisher meets Fukunaga-Koontz: A new look at linear discriminants,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 323–329.

Siegel, M.

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

Sim, T.

S. Zhang and T. Sim, “Discriminant subspace analysis: a Fukunaga-Koontz approach,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 1732–1745 (2007).
[CrossRef] [PubMed]

Z. Shang and T. Sim, “When Fisher meets Fukunaga-Koontz: A new look at linear discriminants,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 323–329.

Solomon, J.

J. Solomon and D. Pelli, “The visual filter mediating letter identification,” Nature (London) 369, 395–397 (1994).
[CrossRef]

Stone, L.

M. Eckstein, B. Beutter, and L. Stone, “Quantifying the performance limits of human saccadic targeting during visual search,” Perception 30, 1389–1401 (2001).
[CrossRef]

Storke, D.

R. Duda, P. Hart, and D. Storke, Pattern Classification (Wiley, 2001).

Swets, J.

D. Green and J. Swets, Signal Detection Theory and Psychophysics (Wiley, 1966).

Tadin, D.

J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
[CrossRef] [PubMed]

Thomas, J.

T. Cover and J. Thomas, Elements of Information Theory (Wiley, 1991).
[CrossRef]

Tjan, B.

B. Tjan and A. Nandy, “Classification images with uncertainty,” J. Vision 6, 387–413 (2006).
[CrossRef]

B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
[CrossRef] [PubMed]

Tong, F.

Y. Kamitani and F. Tong, “Decoding the visual and subjective contents of the human brain,” Nat. Neurosci. 8, 679–685 (2005).
[CrossRef] [PubMed]

Trinath, T.

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

Tyler, C.

C. Chen, C. Tyler, and H. Baseler, “Statistical properties of BOLD magnetic resonance activity in the human brain,” Neuroimage 20, 1096–1109 (2003).
[CrossRef] [PubMed]

Vapnik, V.

V. Vapnik, Statistical Learning Theory (Wiley, 1998).

Warmuth, M.

A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
[CrossRef]

Welchman, A.

T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
[CrossRef] [PubMed]

Whalen, A.

R. McDonough and A. Whalen, Detection of Signals in Noise (Academic, 1995).

Zhang, S.

S. Zhang and T. Sim, “Discriminant subspace analysis: a Fukunaga-Koontz approach,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 1732–1745 (2007).
[CrossRef] [PubMed]

Zhang, Y.

Y. Zhang, B. Pham, and M. Eckstein, “The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds,” IEEE Trans. Med. Imaging 25, 1348–1362 (2006).
[CrossRef] [PubMed]

Cereb. Cortex (1)

M. Philiastides and P. Sajda, “Temporal characterization of the neural correlates of perceptual decision making in the human brain,” Cereb. Cortex 16, 509–518 (2006).
[CrossRef]

IEEE Trans. Inf. Theory (1)

W. Peterson, T. Birdsall, and W. Fox, “The theory of signal detectability,” IEEE Trans. Inf. Theory 4, 171–212 (1954).
[CrossRef]

IEEE Trans. Med. Imaging (1)

Y. Zhang, B. Pham, and M. Eckstein, “The effect of nonlinear human visual system components on performance of a channelized Hotelling observer in structured backgrounds,” IEEE Trans. Med. Imaging 25, 1348–1362 (2006).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

S. Zhang and T. Sim, “Discriminant subspace analysis: a Fukunaga-Koontz approach,” IEEE Trans. Pattern Anal. Mach. Intell. 29, 1732–1745 (2007).
[CrossRef] [PubMed]

Inf. Process. Lett. (1)

A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, “Occam’s Razor,” Inf. Process. Lett. 24, 377–380 (1987).
[CrossRef]

J. Neurophysiol. (2)

T. Donner, M. Siegel, R. Oostenveld, P. Fries, M. Bauer, and A. Engel, “Population activity in the human dorsal pathway predicts the accuracy of visual motion detection,” J. Neurophysiol. 98, 345–359 (2007).
[CrossRef] [PubMed]

D. Ostwald, J. Lam, S. Li, and Z. Kourtzi, “Neural coding of global form in the human visual cortex,” J. Neurophysiol. 99, 2456–2469 (2008).
[CrossRef] [PubMed]

J. Neurosci. (4)

T. Preston, S. Li, Z. Kourtzi, and A. Welchman, “Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain,” J. Neurosci. 28, 11315–11327 (2008).
[CrossRef] [PubMed]

K. Britten, M. Shadlen, W. Newsome, and J. Movshon, “The analysis of visual motion: a comparison of neuronal and psychophysical performance,” J. Neurosci. 12, 4745–4765 (1992).
[PubMed]

N. Kanwisher, J. McDermott, and M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci. 17, 4302–4311 (1997).
[PubMed]

G. Boynton, S. Engel, G. Glover, and D. Heeger, “Linear systems analysis of functional magnetic resonance imaging in Human V1,” J. Neurosci. 16, 4207–4221 (1996).
[PubMed]

J. Vision (1)

B. Tjan and A. Nandy, “Classification images with uncertainty,” J. Vision 6, 387–413 (2006).
[CrossRef]

Nat. Neurosci. (2)

J. Haynes and G. Rees, “Predicting the orientation of invisible stimuli from activity in human primary visual cortex,” Nat. Neurosci. 8, 686–691 (2005).
[CrossRef] [PubMed]

Y. Kamitani and F. Tong, “Decoding the visual and subjective contents of the human brain,” Nat. Neurosci. 8, 679–685 (2005).
[CrossRef] [PubMed]

Nat. Rev. Neurosci. (2)

J. Haynes and G. Rees, “Decoding mental states from brain activity in humans,” Nat. Rev. Neurosci. 7, 523–534 (2006).
[CrossRef] [PubMed]

D. Heeger and D. Ress, “What does fMRI tell us about neuronal activity?” Nat. Rev. Neurosci. 3, 142–151 (2002).
[CrossRef] [PubMed]

Nature (London) (2)

N. Logothetis, J. Pauls, M. Augath, T. Trinath, and A. Oeltermann, “Neurophysiological investigation of the basis of the fMRI signal,” Nature (London) 412, 150–157 (2001).
[CrossRef]

J. Solomon and D. Pelli, “The visual filter mediating letter identification,” Nature (London) 369, 395–397 (1994).
[CrossRef]

Neuroimage (1)

C. Chen, C. Tyler, and H. Baseler, “Statistical properties of BOLD magnetic resonance activity in the human brain,” Neuroimage 20, 1096–1109 (2003).
[CrossRef] [PubMed]

Neuron (1)

S. Li, S. Mayhew, and Z. Kourtzi, “Learning shapes the representation of behavioral choice in the human brain,” Neuron 62, 441–452 (2009).
[CrossRef] [PubMed]

Pattern Recogn. Lett. (2)

K. Das and Z. Nenadic, “An efficient discriminant-based solution for small sample size problem,” Pattern Recogn. Lett. 42, 857–866 (2009).

K. Das and Z. Nenadic, “Approximate information discriminant analysis: a computationally simple heteroscedastic feature extraction technique,” Pattern Recogn. Lett. 41, 1565–1574 (2008).

Percept. Psychophys. (1)

J. Gold, D. Tadin, S. Cook, and R. Blake, “The efficiency of biological motion perception,” Percept. Psychophys. 70, 88–95 (2008).
[CrossRef] [PubMed]

Perception (2)

M. Eckstein, B. Beutter, and L. Stone, “Quantifying the performance limits of human saccadic targeting during visual search,” Perception 30, 1389–1401 (2001).
[CrossRef]

H. Barlow, “Single units and sensation: A neuron doctrine for perceptual psychology?” Perception 1, 371–394 (1972).
[CrossRef] [PubMed]

Philos. Trans. R. Soc. London Ser. B (1)

H. Barlow, “The absolute efficiency of perceptual decisions,” Philos. Trans. R. Soc. London Ser. B 290, 71–91 (1980).
[CrossRef]

Proc. Natl. Acad. Sci. U.S.A. (2)

L. Pessoa and S. Padmala, “Quantitative prediction of perceptual decisions during near-threshold fear detection,” Proc. Natl. Acad. Sci. U.S.A. 102, 5612–5617 (2005).
[CrossRef] [PubMed]

R. Ratcliff, M. Philiastides, and P. Sajda, “Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG,” Proc. Natl. Acad. Sci. U.S.A. 106, 6539–6544 (2009).
[CrossRef] [PubMed]

Psychol. Rev. (1)

W. Geisler, “Sequential ideal-observer analysis of visual discriminations,” Psychol. Rev. 96, 267–314 (1989).
[CrossRef] [PubMed]

Science (1)

J. Haxby, M. Gobbini, M. Furey, A. Ishai, J. Schouten, and P. Pietrini, “Distributed and overlapping representations of faces and objects in ventral temporal cortex,” Science 293, 2425–2430 (2001).
[CrossRef] [PubMed]

Vision Res. (2)

B. Tjan, W. Braje, G. Legge, and D. Kersten, “Human efficiency for recognizing 3-D objects in luminance noise,” Vision Res. 35, 3053–3068 (1995).
[CrossRef] [PubMed]

J. Gold, P. Bennett, and A. Sekuler, “Identification of band-pass filtered letters and faces by human and ideal observers,” Vision Res. 39, 3537–3560 (1999).
[CrossRef]

Other (15)

W. Geisler, “Ideal observer analysis,” in The Visual Neurosciences, L.Chalupa and J.Werner, eds. (MIT Press, 2003), pp. 825–837.

With spatially coarse measures such as fMRI and ERP the local interactions are pooled together and lost. However, long-range connections can be monitored between the large groups of neurons represented in each voxel or electrode.

R. Duda, P. Hart, and D. Storke, Pattern Classification (Wiley, 2001).

Here, the imaging data can be thought of as coming from two classes, face present and car present. On top of the category-specific activity is the imaging noise which has been shown to be highly Gaussian for fMRI data . Thus, the data themselves can be thought of as a Gaussian noise process with a mean displaced by the category-specific activation.

Cclassifier is equivalent to Chuman as the human’s behavioral response and brain activation are derived from the same displayed stimulus.

In addition, there were also procedural differences: stimulus presentation time, display luminance, image size on the retina (4.57° vs. 5.13°). However, given that the task is limited by external noise, it is likely that these procedural differences might result in a smaller performance difference than one might expect from noiseless displays.

Z. Shang and T. Sim, “When Fisher meets Fukunaga-Koontz: A new look at linear discriminants,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 323–329.

D. Green and J. Swets, Signal Detection Theory and Psychophysics (Wiley, 1966).

Further assumptions are well-documented and not the focus of this paper, such as the exact relation between neuronal firing and the observable measurement provided by the specific imaging modality. For our purposes we will assume (somewhat safely, given the large and growing body of evidence) that the imaging observables represent some direct transformation of underlying neural activity (fMRI: below; EEG: below; MEG: below).

R. McDonough and A. Whalen, Detection of Signals in Noise (Academic, 1995).

This amounts to a spatial deconvolution, or, equivalently, multiplying by the inverse of the covariance matrix per Eq. .

V. Vapnik, Statistical Learning Theory (Wiley, 1998).

T. Cover and J. Thomas, Elements of Information Theory (Wiley, 1991).
[CrossRef]

P. Nunez, Electrical Fields of the Brain: the Neurophysics of EEG (Oxford University Press, 1981).

Y. Okada, “Neurogenesis of evoked magnetic fields,” in Biomagnetism: an Interdisciplinary Approach, S.Williamson, ed. (Plenum, 1983), pp. 399–408.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1

Outline of an ideal observer framework to analyze human behavior and classifier performance based on neural activity. The visual stimulus impinges on the observer’s retina, where it is sampled by the photoreceptor lattice. This neural signal is then propagated back to the cortex. Imaging techniques such as fMRI and EEG observe indirect measures of neural firing related to the stimulus and task. Pattern classifiers are used to map patterns of activity to specific postulated brain states that can then be used to make inferences about the state of the world. The human brain also uses the pattern of activity to make decisions. The ideal observer performs the visual task in a mathematically optimal way, giving an upper bound on task performance. The human’s behavior and the pattern classifier’s decisions can be compared with those of the ideal observer, allowing for a principled comparison of the quality of information being used by each decision mechanism.

Fig. 2
Fig. 2

Task structures for the EEG and fMRI sessions. The images and task were the same, but acquisition concerns required slight modifications to presentation and response. The C and F in the fMRI protocol show the observer which hand (left side of fixation cross means use left hand and vice versa) to use for a car or a face response, respectively.

Fig. 3
Fig. 3

(a) Averaged EEG signal time-locked to stimulus presentation across the three observers for the time epoch of interest showing a classic large negative deflection for faces around 170 ms . (b) Localized regions of interest for one of the observers in the fMRI session. Here, the FFA was selected for further analysis. (c) Prototypical hemodynamic response functions (HRF) from the FFA from one of the observers in the fMRI sessions showing increased response to faces compared to cars. The two time points of interest are provided for reference.

Fig. 4
Fig. 4

Ideal observer algorithm. Likelihoods are computed by cross-correlating the stimulus with the possible prewhitened templates and then summing across within category products. The maximum summed likelihood is taken as the decision.

Fig. 5
Fig. 5

Absolute and relative efficiencies are calculated by determining the signal contrast that causes the ideal observer to perform at the level of the human or the classifier and then taking the squared ratio of the contrast values.

Fig. 6
Fig. 6

Task performance, in terms of proportion correct, for each observer’s (1, 2, and 3) behavioral and neural classifier decisions. Error bars represent one SEM.

Fig. 7
Fig. 7

Performance transformed into efficiency units using ideal observer analysis. Error bars represent one SEM.

Fig. 8
Fig. 8

Relative efficiencies of the classifier compared to behavioral performance. Of note is the especially poor extraction of neural information from observer 3 in the EEG session compared with the behavioral performance. Error bars represent one SEM.

Fig. 9
Fig. 9

Mean performances across observers. On the left are the behavioral and classifier proportion correct scores for each modality. On the right are the corresponding relative efficiencies. Error bars represent one SEM.

Fig. 10
Fig. 10

Ideal observer simulation results for hypothetical situations. (a) Ideal observer performance can be degraded with either additional noise or intrinsic uncertainty as seen in these simulated psychometric curves. Here, uncertainty was created by adding modified versions of the original car and face templates to the signal sets through shifting the images left, right, up, and down by 0.25° visual angle. Noise was zero-mean white Gaussian with an RMS noise contrast equivalent to the image noise at 14.5%. (b) When converted into efficiency units (by comparing the sub-optimal models with the ideal) it is shown that additional noise lowers efficiency but by a constant amount across signal strengths. Meanwhile, non-linear effects, such as those given by added signal uncertainty, lead to an efficiency that varies monotonically with signal strength.

Fig. 11
Fig. 11

Behavioral performance is shown alongside four different classifiers (including FKT, which has been used throughout the paper). Overall performance levels vary, but relative patterns remain intact. Error bars represent one SEM.

Fig. 12
Fig. 12

Relative efficiency of the four classifiers tested. Once again, patterns remain clear except possibly with the CPCA in the fMRI session. This lends strength to the argument that the results are not a product of the choice of classifier. Error bars represent one SEM.

Fig. 13
Fig. 13

Complete set of amplitude spectrum-normalized stimuli before addition of the filtered white noise.

Tables (1)

Tables Icon

Table 1 Performance Values, in Terms of Proportion Correct, for a Linear Univariate Bayesian Analysis Using the Average BOLD Signal from All Voxels in the ROI as Input and the Four Classifiers Implemented for This Study for the Three fMRI Observers a

Equations (21)

Equations on this page are rendered with MathJax. Learn more.

ρ = i , j I ( s i , j ) 24 ,
s i , j = I 1 ( I ( s i , j ) ρ I ( s i , j ) ) ,
n = I 1 [ I ( n ) ρ ] .
P ( x i | g t ) = P ( x i ) P ( g t | x i ) P ( g t ) P ( g t | x i ) = l i , t .
l i , t = j P ( s i , j ) P ( g t | s i , j ) j P ( g t | s i , j ) = j l i , j , t .
l i , j , t = 1 ( 2 π ) P 2 | | 1 2 exp ( 1 2 ( g t s i , j ) T 1 ( g t s i , j ) ) .
s ̂ i , j = I 1 ( I ( s i , j ) ρ ) ,
l i , j , t = exp ( g t T s ̂ i , j 0.5 E ̂ i , j ) ,
η 0 , human = C I O | human 2 C human 2 or η 0 , classifier = C I O | classifier 2 C classifier 2 .
η rel , cond 1 , cond 2 = η 0 , cond 1 η 0 , cond 2 .
S b = k = 1 C n k m k m k T ,
S w = k = 1 C i ( x i m k ) ( x i m k ) T .
P T ( S b + S w ) P = S ̃ b + S ̃ w = I ,
w = K 1 ( μ f μ c ) ,
where K = 1 N t i = 1 N t ( x i μ ) ( x i μ ) T .
K reg = K + p Diag ( K ) ,
w T x i b + 1 for L i = + 1 ,
w T b 1 for L i = 1.
L i ( w T x i b ) 1.
min ( 2 w ) s.t. L i ( w T x i b ) 1 , i = 1 N t .
x * i = F i T ( x * μ i ) , i = 1 , 2.

Metrics