Abstract

The space of visual signals is high-dimensional and natural visual images have a highly complex statistical structure. While many studies suggest that only a limited number of image statistics are used for perceptual judgments, a full understanding of visual function requires analysis not only of the impact of individual image statistics, but also, how they interact. In natural images, these statistical elements (luminance distributions, correlations of low and high order, edges, occlusions, etc.) are intermixed, and their effects are difficult to disentangle. Thus, there is a need for construction of stimuli in which one or more statistical elements are introduced in a controlled fashion, so that their individual and joint contributions can be analyzed. With this as motivation, we present algorithms to construct synthetic images in which local image statistics—including luminance distributions, pair-wise correlations, and higher-order correlations—are explicitly specified and all other statistics are determined implicitly by maximum-entropy. We then apply this approach to measure the sensitivity of the human visual system to local image statistics and to sample their interactions.

© 2012 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. C. Chubb, M. S. Landy, and J. Econopouly,“A visual mechanism tuned to black,” Vis. Res. 44, 3223–3232.(2004).
  2. B. Julesz,“Textons, the elements of texture perception, and their interactions,” Nature 290, 91–97.(1981).
    [CrossRef]
  3. B. Julesz, E. N. Gilbert, and J. D. Victor,“Visual discrimination of textures with identical third-order statistics,” Biol. Cybern. 31, 137–140.(1978).
  4. J. D. Victor and M. M. Conte,“Spatial organization of nonlinear interactions in form perception,” Vis. Res. 31, 1457–1488 (1991).
    [CrossRef]
  5. S. Lyu and E. P. Simoncelli,“Nonlinear extraction of independent components of natural images using radial gaussianization,” Neural Comput. 21, 1485–1519 (2009).
    [CrossRef]
  6. F. Sinz, E. P. Simoncelli, and M. Bethge, “Hierarchical modeling of local image features through Lp-nested symmetric distributions,” in Advances in Neural Information Processing Systems, Vol. 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds. (NIPS, 2009), pp. 1696–1704.
  7. D. J. Field, “Relations between the statistics of natural images and the response properties of cortical cells,” J. Opt. Soc. Am. A 4, 2379–2394 (1987).
  8. W. A. Richards, “Lightness scale from image intensity distributions,” Appl. Opt. 21, 2569–2582 (1982).
  9. C. Zetzsche and G. Krieger, “Nonlinear mechanism and higher-order statistics in biological vision and electronic image processing: review and perspectives,” J. Electron. Imaging 10, 56–99 (2001).
  10. G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
    [CrossRef]
  11. A. Torralba and A. Oliva, “Statistics of natural image categories,” Network 14, 391–412 (2003).
    [CrossRef]
  12. W. S. Geisler, “Visual perception and the statistical properties of natural scenes,” Annu. Rev. Psychol. 59, 167–192 (2008).
  13. S. V. David and J. L. Gallant, “Predicting neuronal responses during natural vision,” Network 16, 239–260 (2005).
    [CrossRef]
  14. H. B. Barlow, “Possible principles underlying the transformation of sensory messages,” in Sensory Communication, W. A. Rosenblith, ed. (MIT, 1961), pp. 217–234.
  15. R. Levine and M. Tribus, The Maximum Entropy Formalism (MIT, 1979).
  16. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, 1991).
  17. E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
    [CrossRef]
  18. S. H. Nirenberg and J. D. Victor, “Analyzing the activity of large populations of neurons: how tractable is the problem?” Curr. Opin. Neurobiol. 17, 397–400 (2007).
  19. J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
  20. S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling,” Int. J. Comput. Vis. 27, 107–126(1998).
    [CrossRef]
  21. D. K. Pickard, “Unilateral Markov fields,” Adv. Appl. Probab. 12, 655–671 (1980).
    [CrossRef]
  22. J. D. Victor, C. Chubb, and M. M. Conte, “Interaction of luminance and higher-order statistics in texture discrimination,” Vis. Res. 45, 311–328 (2005).
    [CrossRef]
  23. J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
    [CrossRef]
  24. S. I. Amari, “Information geometry on hierarchy of probability distributions,” IEEE Trans. Inf. Theory 47, 1701–1711(2001).
    [CrossRef]
  25. F. Champagnat, J. Idier, and Y. Goussard, “Stationary Markov Random Fields on a finite rectangular lattice,” IEEE Trans. Inf. Theory 44, 2901–2916 (1998).
    [CrossRef]
  26. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
    [CrossRef]
  27. P. Brodatz, Textures: a Photographic Album for Artists and Designers (Dover, 1965).
  28. A. Poirson, B. Wandell, D. Varner, and D. Brainard, “Surface characterizations of color thresholds,” J. Opt. Soc. Am. A 7, 783–789 (1990).
  29. J. Portilla and E. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,”. Int. J. Comput. Vis. 40, 49–71 (2000).
  30. J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
    [CrossRef]

2010 (1)

G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
[CrossRef]

2009 (1)

S. Lyu and E. P. Simoncelli,“Nonlinear extraction of independent components of natural images using radial gaussianization,” Neural Comput. 21, 1485–1519 (2009).
[CrossRef]

2008 (1)

W. S. Geisler, “Visual perception and the statistical properties of natural scenes,” Annu. Rev. Psychol. 59, 167–192 (2008).

2007 (1)

S. H. Nirenberg and J. D. Victor, “Analyzing the activity of large populations of neurons: how tractable is the problem?” Curr. Opin. Neurobiol. 17, 397–400 (2007).

2006 (3)

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
[CrossRef]

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

2005 (3)

S. V. David and J. L. Gallant, “Predicting neuronal responses during natural vision,” Network 16, 239–260 (2005).
[CrossRef]

J. D. Victor, C. Chubb, and M. M. Conte, “Interaction of luminance and higher-order statistics in texture discrimination,” Vis. Res. 45, 311–328 (2005).
[CrossRef]

J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
[CrossRef]

2004 (1)

C. Chubb, M. S. Landy, and J. Econopouly,“A visual mechanism tuned to black,” Vis. Res. 44, 3223–3232.(2004).

2003 (1)

A. Torralba and A. Oliva, “Statistics of natural image categories,” Network 14, 391–412 (2003).
[CrossRef]

2001 (2)

C. Zetzsche and G. Krieger, “Nonlinear mechanism and higher-order statistics in biological vision and electronic image processing: review and perspectives,” J. Electron. Imaging 10, 56–99 (2001).

S. I. Amari, “Information geometry on hierarchy of probability distributions,” IEEE Trans. Inf. Theory 47, 1701–1711(2001).
[CrossRef]

2000 (1)

J. Portilla and E. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,”. Int. J. Comput. Vis. 40, 49–71 (2000).

1998 (2)

F. Champagnat, J. Idier, and Y. Goussard, “Stationary Markov Random Fields on a finite rectangular lattice,” IEEE Trans. Inf. Theory 44, 2901–2916 (1998).
[CrossRef]

S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling,” Int. J. Comput. Vis. 27, 107–126(1998).
[CrossRef]

1991 (1)

J. D. Victor and M. M. Conte,“Spatial organization of nonlinear interactions in form perception,” Vis. Res. 31, 1457–1488 (1991).
[CrossRef]

1990 (1)

1987 (1)

1982 (1)

1981 (1)

B. Julesz,“Textons, the elements of texture perception, and their interactions,” Nature 290, 91–97.(1981).
[CrossRef]

1980 (1)

D. K. Pickard, “Unilateral Markov fields,” Adv. Appl. Probab. 12, 655–671 (1980).
[CrossRef]

1978 (1)

B. Julesz, E. N. Gilbert, and J. D. Victor,“Visual discrimination of textures with identical third-order statistics,” Biol. Cybern. 31, 137–140.(1978).

1953 (1)

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

Amari, S. I.

S. I. Amari, “Information geometry on hierarchy of probability distributions,” IEEE Trans. Inf. Theory 47, 1701–1711(2001).
[CrossRef]

Ashurova, A.

J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
[CrossRef]

Balasubramanian, V.

G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
[CrossRef]

Barlow, H. B.

H. B. Barlow, “Possible principles underlying the transformation of sensory messages,” in Sensory Communication, W. A. Rosenblith, ed. (MIT, 1961), pp. 217–234.

Berry, M. J.

E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
[CrossRef]

Bethge, M.

F. Sinz, E. P. Simoncelli, and M. Bethge, “Hierarchical modeling of local image features through Lp-nested symmetric distributions,” in Advances in Neural Information Processing Systems, Vol. 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds. (NIPS, 2009), pp. 1696–1704.

Bialek, W.

E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
[CrossRef]

Brainard, D.

Brodatz, P.

P. Brodatz, Textures: a Photographic Album for Artists and Designers (Dover, 1965).

Champagnat, F.

F. Champagnat, J. Idier, and Y. Goussard, “Stationary Markov Random Fields on a finite rectangular lattice,” IEEE Trans. Inf. Theory 44, 2901–2916 (1998).
[CrossRef]

Chichilnisky, E. J.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

Chubb, C.

J. D. Victor, C. Chubb, and M. M. Conte, “Interaction of luminance and higher-order statistics in texture discrimination,” Vis. Res. 45, 311–328 (2005).
[CrossRef]

J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
[CrossRef]

C. Chubb, M. S. Landy, and J. Econopouly,“A visual mechanism tuned to black,” Vis. Res. 44, 3223–3232.(2004).

Conte, M. M.

J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
[CrossRef]

J. D. Victor, C. Chubb, and M. M. Conte, “Interaction of luminance and higher-order statistics in texture discrimination,” Vis. Res. 45, 311–328 (2005).
[CrossRef]

J. D. Victor and M. M. Conte,“Spatial organization of nonlinear interactions in form perception,” Vis. Res. 31, 1457–1488 (1991).
[CrossRef]

Cover, T. M.

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, 1991).

David, S. V.

S. V. David and J. L. Gallant, “Predicting neuronal responses during natural vision,” Network 16, 239–260 (2005).
[CrossRef]

Econopouly, J.

C. Chubb, M. S. Landy, and J. Econopouly,“A visual mechanism tuned to black,” Vis. Res. 44, 3223–3232.(2004).

Field, D. J.

Field, G. D.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

Gallant, J. L.

S. V. David and J. L. Gallant, “Predicting neuronal responses during natural vision,” Network 16, 239–260 (2005).
[CrossRef]

Gauthier, J. L.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

Geisler, W. S.

W. S. Geisler, “Visual perception and the statistical properties of natural scenes,” Annu. Rev. Psychol. 59, 167–192 (2008).

Gilbert, E. N.

B. Julesz, E. N. Gilbert, and J. D. Victor,“Visual discrimination of textures with identical third-order statistics,” Biol. Cybern. 31, 137–140.(1978).

Goussard, Y.

F. Champagnat, J. Idier, and Y. Goussard, “Stationary Markov Random Fields on a finite rectangular lattice,” IEEE Trans. Inf. Theory 44, 2901–2916 (1998).
[CrossRef]

Grivich, M. I.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

Idier, J.

F. Champagnat, J. Idier, and Y. Goussard, “Stationary Markov Random Fields on a finite rectangular lattice,” IEEE Trans. Inf. Theory 44, 2901–2916 (1998).
[CrossRef]

Julesz, B.

B. Julesz,“Textons, the elements of texture perception, and their interactions,” Nature 290, 91–97.(1981).
[CrossRef]

B. Julesz, E. N. Gilbert, and J. D. Victor,“Visual discrimination of textures with identical third-order statistics,” Biol. Cybern. 31, 137–140.(1978).

Krieger, G.

C. Zetzsche and G. Krieger, “Nonlinear mechanism and higher-order statistics in biological vision and electronic image processing: review and perspectives,” J. Electron. Imaging 10, 56–99 (2001).

Landy, M. S.

C. Chubb, M. S. Landy, and J. Econopouly,“A visual mechanism tuned to black,” Vis. Res. 44, 3223–3232.(2004).

Levine, R.

R. Levine and M. Tribus, The Maximum Entropy Formalism (MIT, 1979).

Litke, A. M.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

Lyu, S.

S. Lyu and E. P. Simoncelli,“Nonlinear extraction of independent components of natural images using radial gaussianization,” Neural Comput. 21, 1485–1519 (2009).
[CrossRef]

Metropolis, N.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

Mumford, D.

S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling,” Int. J. Comput. Vis. 27, 107–126(1998).
[CrossRef]

Nirenberg, S. H.

S. H. Nirenberg and J. D. Victor, “Analyzing the activity of large populations of neurons: how tractable is the problem?” Curr. Opin. Neurobiol. 17, 397–400 (2007).

Oliva, A.

A. Torralba and A. Oliva, “Statistics of natural image categories,” Network 14, 391–412 (2003).
[CrossRef]

Petrusca, D.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

Pickard, D. K.

D. K. Pickard, “Unilateral Markov fields,” Adv. Appl. Probab. 12, 655–671 (1980).
[CrossRef]

Poirson, A.

Portilla, J.

J. Portilla and E. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,”. Int. J. Comput. Vis. 40, 49–71 (2000).

Prentice, J. S.

G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
[CrossRef]

Richards, W. A.

Rosenbluth, A. W.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

Rosenbluth, M. N.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

Schneidman, E.

E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
[CrossRef]

Segev, R.

E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
[CrossRef]

Sher, A.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

Shlens, J.

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

Simoncelli, E.

J. Portilla and E. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,”. Int. J. Comput. Vis. 40, 49–71 (2000).

Simoncelli, E. P.

S. Lyu and E. P. Simoncelli,“Nonlinear extraction of independent components of natural images using radial gaussianization,” Neural Comput. 21, 1485–1519 (2009).
[CrossRef]

F. Sinz, E. P. Simoncelli, and M. Bethge, “Hierarchical modeling of local image features through Lp-nested symmetric distributions,” in Advances in Neural Information Processing Systems, Vol. 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds. (NIPS, 2009), pp. 1696–1704.

Sinz, F.

F. Sinz, E. P. Simoncelli, and M. Bethge, “Hierarchical modeling of local image features through Lp-nested symmetric distributions,” in Advances in Neural Information Processing Systems, Vol. 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds. (NIPS, 2009), pp. 1696–1704.

Teller, A. H.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

Teller, E.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

Thomas, J. A.

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, 1991).

Tkacik, G.

G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
[CrossRef]

Torralba, A.

A. Torralba and A. Oliva, “Statistics of natural image categories,” Network 14, 391–412 (2003).
[CrossRef]

Tribus, M.

R. Levine and M. Tribus, The Maximum Entropy Formalism (MIT, 1979).

Varner, D.

Victor, J. D.

G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
[CrossRef]

S. H. Nirenberg and J. D. Victor, “Analyzing the activity of large populations of neurons: how tractable is the problem?” Curr. Opin. Neurobiol. 17, 397–400 (2007).

J. D. Victor, C. Chubb, and M. M. Conte, “Interaction of luminance and higher-order statistics in texture discrimination,” Vis. Res. 45, 311–328 (2005).
[CrossRef]

J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
[CrossRef]

J. D. Victor and M. M. Conte,“Spatial organization of nonlinear interactions in form perception,” Vis. Res. 31, 1457–1488 (1991).
[CrossRef]

B. Julesz, E. N. Gilbert, and J. D. Victor,“Visual discrimination of textures with identical third-order statistics,” Biol. Cybern. 31, 137–140.(1978).

Wandell, B.

Wu, Y.

S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling,” Int. J. Comput. Vis. 27, 107–126(1998).
[CrossRef]

Zetzsche, C.

C. Zetzsche and G. Krieger, “Nonlinear mechanism and higher-order statistics in biological vision and electronic image processing: review and perspectives,” J. Electron. Imaging 10, 56–99 (2001).

Zhu, S. C.

S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling,” Int. J. Comput. Vis. 27, 107–126(1998).
[CrossRef]

Adv. Appl. Probab. (1)

D. K. Pickard, “Unilateral Markov fields,” Adv. Appl. Probab. 12, 655–671 (1980).
[CrossRef]

Annu. Rev. Psychol. (1)

W. S. Geisler, “Visual perception and the statistical properties of natural scenes,” Annu. Rev. Psychol. 59, 167–192 (2008).

Appl. Opt. (1)

Biol. Cybern. (1)

B. Julesz, E. N. Gilbert, and J. D. Victor,“Visual discrimination of textures with identical third-order statistics,” Biol. Cybern. 31, 137–140.(1978).

Curr. Opin. Neurobiol. (1)

S. H. Nirenberg and J. D. Victor, “Analyzing the activity of large populations of neurons: how tractable is the problem?” Curr. Opin. Neurobiol. 17, 397–400 (2007).

IEEE Trans. Inf. Theory (2)

S. I. Amari, “Information geometry on hierarchy of probability distributions,” IEEE Trans. Inf. Theory 47, 1701–1711(2001).
[CrossRef]

F. Champagnat, J. Idier, and Y. Goussard, “Stationary Markov Random Fields on a finite rectangular lattice,” IEEE Trans. Inf. Theory 44, 2901–2916 (1998).
[CrossRef]

Int. J. Comput. Vis. (1)

S. C. Zhu, Y. Wu, and D. Mumford, “Filters, random fields and maximum entropy (FRAME): towards a unified theory for texture modeling,” Int. J. Comput. Vis. 27, 107–126(1998).
[CrossRef]

J. Chem. Phys. (1)

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equations of state calculations by fast computing machines,” J. Chem. Phys. 21, 1087–1091 (1953).
[CrossRef]

J. Electron. Imaging (1)

C. Zetzsche and G. Krieger, “Nonlinear mechanism and higher-order statistics in biological vision and electronic image processing: review and perspectives,” J. Electron. Imaging 10, 56–99 (2001).

J. Neurosci. (2)

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).

J. Shlens, G. D. Field, J. L. Gauthier, M. I. Grivich, D. Petrusca, A. Sher, A. M. Litke, and E. J. Chichilnisky, “The structure of multi-neuron firing patterns in primate retina,” J. Neurosci. 26, 8254–8266 (2006).
[CrossRef]

J. Opt. Soc. Am. A (2)

J. Vis. (1)

J. D. Victor, A. Ashurova, C. Chubb, and M. M. Conte, “Isodiscrimination contours in a three-parameter texture space,” J. Vis. 6(6):205, 205a, http://journalofvision.org/6/6/205/ , (2005).
[CrossRef]

Nature (2)

E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, “Weak pair-wise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007–1012(2006).
[CrossRef]

B. Julesz,“Textons, the elements of texture perception, and their interactions,” Nature 290, 91–97.(1981).
[CrossRef]

Network (2)

A. Torralba and A. Oliva, “Statistics of natural image categories,” Network 14, 391–412 (2003).
[CrossRef]

S. V. David and J. L. Gallant, “Predicting neuronal responses during natural vision,” Network 16, 239–260 (2005).
[CrossRef]

Neural Comput. (1)

S. Lyu and E. P. Simoncelli,“Nonlinear extraction of independent components of natural images using radial gaussianization,” Neural Comput. 21, 1485–1519 (2009).
[CrossRef]

Proc. Natl. Acad. Sci. U.S.A. (1)

G. Tkacik, J. S. Prentice, J. D. Victor, and V. Balasubramanian, “Local statistics in natural scenes predict the saliency of synthetic textures,” Proc. Natl. Acad. Sci. U.S.A. 107, 18149–18154 (2010).
[CrossRef]

Vis. Res. (3)

C. Chubb, M. S. Landy, and J. Econopouly,“A visual mechanism tuned to black,” Vis. Res. 44, 3223–3232.(2004).

J. D. Victor and M. M. Conte,“Spatial organization of nonlinear interactions in form perception,” Vis. Res. 31, 1457–1488 (1991).
[CrossRef]

J. D. Victor, C. Chubb, and M. M. Conte, “Interaction of luminance and higher-order statistics in texture discrimination,” Vis. Res. 45, 311–328 (2005).
[CrossRef]

Other (6)

P. Brodatz, Textures: a Photographic Album for Artists and Designers (Dover, 1965).

J. Portilla and E. Simoncelli, “A parametric texture model based on joint statistics of complex wavelet coefficients,”. Int. J. Comput. Vis. 40, 49–71 (2000).

F. Sinz, E. P. Simoncelli, and M. Bethge, “Hierarchical modeling of local image features through Lp-nested symmetric distributions,” in Advances in Neural Information Processing Systems, Vol. 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, eds. (NIPS, 2009), pp. 1696–1704.

H. B. Barlow, “Possible principles underlying the transformation of sensory messages,” in Sensory Communication, W. A. Rosenblith, ed. (MIT, 1961), pp. 217–234.

R. Levine and M. Tribus, The Maximum Entropy Formalism (MIT, 1979).

T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, 1991).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1.
Fig. 1.

The image-statistic coordinate axes. Each patch is a typical sample of an image ensemble in which the indicated statistic is set to a nonzero value and higher-order statistics are determined by maximum entropy.

Fig. 2.
Fig. 2.

The image-statistic coordinate planes. Each patch is a typical sample of an image ensemble in which the indicated pair of statistics is set to a nonzero value and the rest are determined according to Table 2.

Fig. 3.
Fig. 3.

Donut construction for maximizing entropy while maintaining 2 × 2 block probabilities. Left: a pair of 3 × 3 regions is identified, for which the outer eight pixels match identically. Right: the interior pixels of the two regions are swapped. This step preserves all 2 × 2 block probabilities. Iterations of this step destroy longer-range correlations.

Fig. 4.
Fig. 4.

Construction of images specified by ( θ , θ ) panel (a) and ( β \ , θ ) (b) (see next page). In each case, a starting image is created via an iterative rule [Appendix C, Section C1 for panel (a), Section C2 for panel (b)] and the donut algorithm is applied to ensure maximum entropy. Parameter values are 0.4 in each case. Autocorrelations (along rows, along columns, and in two dimensions) are shown adjacent to each image. The central pixel in the two-dimensional autocorrelogram has a value of 1.0, above the range of the colorbar. Entropy per pixel for 3 × 3 and 4 × 4 blocks is shown in the lower panels. Error bars (most smaller than the symbols) indicate s.e.m across 16 runs.

Fig. 5.
Fig. 5.

Synthesis of images with multiple specified parameter values by mixing images with specified parameter pairs. Each cycle of the donut algorithm constructs a pair of images, by mixing the pair created on the previous cycle. The starting pair (cycle 0) consists of an image with θ = θ = 0.4 (left) and an image with θ = θ = 0.4 (right); the final pair consists of images with θ = θ = θ = θ = 0.2 . All other texture parameters are equal to zero. Autocorrelation functions are shown adjacent to the images. The central pixel in the two-dimensional autocorrelogram has a value of 1.0, above the range of the colorbar. Entropy per pixel for 3 × 3 and 4 × 4 blocks is shown in the lower panels.

Fig. 6.
Fig. 6.

Application of the donut algorithm to a natural texture. The starting image is a region taken from Brodatz texture 1.4.01 [panel (a)], median-thresholded to create a 64 × 64 binary image (b). (c) and (d): The result of applying the donut algorithm. These images have the same 2 × 2 block probabilities as the starting image [panel (b)]; other statistics are determined by maximum entropy. Even though only short-range correlations are specified, long-range correlations result. Following application of the donut algorithm, the overall oblique slant remains apparent, but the distinction between the two scales (mortar and bricks) is lost.

Fig. 7.
Fig. 7.

Line graphs: performance in a brief four-alternative forced choice segmentation task for image statistics γ , β _ , β / , θ , and α . Chance performance is a fraction correct of 0.25; error bars are 95% confidence limits. Final panel: summary of thresholds a r , obtained by Weibull-function fits to the individual psychometric curves [Eq. (70)]. Consistently across subjects, thresholds for negative and positive variations of each statistic are closely matched and sensitivities across the statistics show systematic differences.

Fig. 8.
Fig. 8.

Isodiscrimination contours (ICs) in five coordinate planes. The distance of the contour from the origin indicates the threshold a r for individual image statistics (along the axes) and their mixtures (in oblique directions); threshold is defined by the value required to achieve a fraction correct of 0.625, halfway between chance and perfect [Eq. (70)]. The outermost circle corresponds to a coordinate value of 1.0. Error bars, most no larger than the contour line thickness, are 95% confidence limits. Task and subject key as in Fig. 7. ICs are approximately elliptical, and in some planes, they are tilted.

Fig. 9.
Fig. 9.

A symmetry argument (Subsection C.2.3) that the recursive “tee” construction for ( β \ , θ ) produces block probabilities whose coordinates { γ , β , β \ , β / , θ , θ , α } are zero. The intersection points represent the centers of individual pixels. The black dots select the pixels lying on two of every three diagonals. (a). Any placement of a “tee” glider contains two or three of the selected pixels. There are only three ways that these selected pixels can be configured within a glider, corresponding to the phase of each glider with respect to the diagonals. (b) All β and θ gliders contain an even number of selected pixels. (c1): For gliders in the set { γ , β , β | , β / , θ , θ , α } , two-thirds of their placements contain an odd number of selected pixels (shaded), and one-third contain an even number of selected pixels. (c2,c3): Shifting the selected pixels by one column permutes the placements of the gliders { γ , β , β | , β / , θ , θ , α } that contain an odd number of selected pixels. As shown in Subsection C.2.3, this implies that those image statistics must be exactly zero.

Tables (2)

Tables Icon

Table 1. Conversion between Block Probabilities and Coordinates a

Tables Icon

Table 2. Specification of Maximum-Entropy Ensembles from Pairs of Coordinates a , b

Equations (130)

Equations on this page are rendered with MathJax. Learn more.

p ( A B C ) = p ( A B C 0 ) + p ( A B C 1 )
p ( A B ) = p ( A B 0 ) + p ( A B 1 ) = p ( A B 0 0 ) + p ( A B 0 1 ) + p ( A B 1 0 ) + p ( A B 1 1 ) .
P ( C D ) = p ( 0 C D ) + p ( 1 C D ) = p ( 0 0 C D ) + p ( 0 1 C D ) + p ( 1 0 C D ) + p ( 1 1 C D ) .
p ( A B 0 0 ) + p ( A B 0 1 ) + p ( A B 1 0 ) + p ( A B 1 1 ) = p ( 0 0 A B ) + p ( 0 1 A B ) + p ( 1 0 A B ) + p ( 1 1 A B ) .
p ( A C ) = p ( A C )
p ( A ) = p ( A ) = p ( A ) = p ( A ) .
φ ( s 1 s 2 s 3 s 4 ) = A 1 , A 2 , A 3 , A 4 p ( A 1 A 2 A 3 A 4 ) ( 1 ) A 1 s 1 + A 2 s 2 + A 3 s 3 + A 4 s 4 ,
p ( A 1 A 2 A 3 A 4 ) = 1 16 s 1 , s 2 , s 3 , s 4 φ ( s 1 s 2 s 3 s 4 ) ( 1 ) A 1 s 1 + A 2 s 2 + A 3 s 3 + A 4 s 4 .
φ ( s 1 s 2 0 0 ) = φ ( 0 0 s 1 s 2 ) .
φ ( s 1 0 s 3 0 ) = φ ( 0 s 1 0 s 3 ) ,
φ ( s 0 0 0 ) = φ ( 0 s 0 0 ) = φ ( 0 0 s 0 ) = φ ( 0 0 0 s ) .
φ ( 0 0 0 0 ) = 1.
α = φ ( 1 1 1 1 ) ,
θ = φ ( 0 1 1 1 ) , θ = φ ( 1 0 1 1 ) , θ = φ ( 1 1 0 1 ) , θ = φ ( 1 1 1 0 ) ,
β = φ ( 1 1 0 0 ) = φ ( 0 0 1 1 ) , β | = φ ( 1 0 1 0 ) = φ ( 0 1 0 1 ) , β \ = φ ( 1 0 0 1 ) , β / = φ ( 0 1 1 0 ) ,
γ = φ ( 1 0 0 0 ) = φ ( 0 1 0 0 ) = φ ( 0 0 1 0 ) = φ ( 0 0 0 1 ) .
θ = p ( 1 1 1 ) p ( 0 1 1 ) p ( 1 0 1 ) + p ( 0 0 1 ) p ( 1 1 0 ) + p ( 0 1 0 ) + p ( 1 0 0 ) p ( 0 0 0 ) ,
β = p ( 1 1 ) p ( 0 1 ) p ( 1 0 ) + p ( 0 0 ) ,
γ = p ( 1 ) p ( 0 ) .
φ i ( R ) = n + ( R , i ) n ( R , i ) n ( R , i ) ,
2 | γ | 1 + β ;
| θ | + | θ | 1 + β ;
| θ | + | θ | 1 + β \ .
H = R p ( R ) ln p ( R )
φ i 0 = φ i ( R ) = R p ( R ) φ i ( R ) .
φ i 0 = R p ( R ) n + ( R , i ) n ( R , i ) n ( R , i ) .
Q = R p ( R ) ln p ( R ) + i λ i R p ( R ) n + ( R , i ) n ( R , i ) n ( R , i ) + λ 0 R p ( R ) ,
Q p ( S ) = 1 ln p ( S ) + i λ i n + ( S , i ) n ( S , i ) n ( S , i ) + λ 0 .
p ( S ) = exp ( μ 0 + i μ i ( n + ( S , i ) n ( S , i ) ) ) ,
p ( S ) = Z ( S ) Z ,
Z ( S ) = exp ( i μ i ( n + ( S , i ) n ( S , i ) ) )
Z = S Z ( S ) .
ln Z μ i = 1 Z Z μ i = n ( S , i ) φ i 0 .
Z = S Z ( S ) = S exp ( μ 1 ( n + ( S , 1 ) n ( S , 1 ) ) ) .
Z = n + + n = n n ! n + ! n ! exp ( μ 1 ( n + n ) ) .
Z = ( e μ 1 + e μ 1 ) n .
γ = φ 1 0 = 1 n ln Z μ 1 = e μ 1 e μ 1 e μ 1 + e μ 1 = tanh ( μ 1 ) .
μ 1 = tanh 1 ( γ ) = 1 2 ln 1 γ 1 + γ
Z = ( 1 + γ 1 γ + 1 γ 1 + γ ) n = ( 1 + γ 1 γ ( 1 + 1 γ 1 + γ ) ) n = ( 2 ( 1 + γ ) ( 1 γ ) ) n .
p ( S ) = 1 Z exp μ 1 ( n + ( S , 1 ) n ( S , 1 ) ) = 1 Z ( 1 γ 1 + γ ) n + ( S , 1 ) n ( S , 1 ) 2 .
p ( S ) = 2 n ( ( 1 + γ ) ( 1 γ ) ) n / 2 ( 1 γ 1 + γ ) n + ( S , 1 ) n ( S , 1 ) 2 .
p ( S ) = ( 1 γ 2 ) n + ( s , 1 ) ( 1 + γ 2 ) n ( s , 1 )
φ i = n + ( S , i ) n ( S , i ) n ( S , i ) = 1 + φ i 0 2 1 φ i 0 2 = φ i 0 .
p ( S ) = ( 1 2 ) n n ( S , i ) ( 1 + φ i 0 2 ) n + ( S , i ) ( 1 φ i 0 2 ) n ( S , i )
p ( S ) = 2 n ( ( 1 + φ i 0 ) ( 1 φ i 0 ) ) n ( S , i ) / 2 ( 1 + φ i 0 1 φ i 0 ) n + ( S , i ) n ( S , i ) 2 .
Z = 2 n init n + + n = n n init ( n n init ) ! n + ! n ! exp ( μ i ( n + n ) ) ,
Z = 2 n init ( e μ i + e μ i ) n n init .
μ i = tanh 1 ( φ i 0 ) = 1 2 ln 1 + φ i 0 1 φ i 0 ,
Z = 2 n init ( 2 ( 1 + φ i 0 ) ( 1 φ i 0 ) ) n n init .
p ( A 1 A 2 A 3 ) = p ( A 1 A 2 ) p ( A 3 | A 2 ) = p ( A 1 A 2 ) p ( A 2 A 3 ) p ( A 2 ) .
p ( A 1 A k ) = p ( A 1 A 2 ) p ( A 2 A 3 ) p ( A k 1 A k ) p ( A 2 ) p ( A 3 ) p ( A k 1 ) .
p ( S ) = p ( 0 0 ) n 00 p ( 0 1 ) n 01 p ( 1 0 ) n 10 p ( 1 1 ) n 11 p ( 0 ) n 0 p ( 1 ) n 1 ,
n 00 = n p 00 ( S ) = n 4 ( 1 2 γ ( S ) + β ( S ) ) ,
n 00 = n 4 + n + ( S , 2 ) n ( S , 2 ) 4 + n + ( S , 1 ) n ( S , 1 ) 2 n 01 = n 4 n + ( S , 2 ) n ( S , 2 ) 4 n 10 = n 4 n + ( S , 2 ) n ( S , 2 ) 4 n 11 = n 4 + n + ( S , 2 ) n ( S , 2 ) 4 n + ( S , 1 ) n ( S , 1 ) 2 n 0 = n 2 + n + ( S , 1 ) n ( S , 1 ) 2 n 1 = n 2 n + ( S , 1 ) n ( S , 1 ) 2 .
p ( A 11 A 12 A 13 A 21 A 22 A 23 ) = p ( A 11 A 12 A 21 A 22 ) p ( A 12 A 13 A 22 A 23 | A 12 A 22 ) = p ( A 11 A 12 A 21 A 22 ) p ( A 12 A 13 A 22 A 23 ) p ( A 12 A 22 ) .
p ( A 11 A 12 A 13 A 21 A 22 A 23 A 31 A 32 A 33 ) = p ( A 11 A 12 A 13 A 21 A 22 A 23 ) p ( A 21 A 22 A 23 A 31 A 32 A 33 ) p ( A 21 A 22 A 23 ) ,
p ( A 21 A 22 A 23 ) = A 11 , A 12 , A 13 p ( A 11 A 12 A 13 A 21 A 22 A 23 ) = A 11 , A 12 , A 13 p ( A 11 A 12 A 21 A 22 ) p ( A 12 A 13 A 22 A 23 ) p ( A 12 A 22 ) = A 12 p ( A 12 A 21 A 22 ) p ( A 12 A 22 A 23 ) p ( A 12 A 22 ) .
p ( B C D ) p ( D ) = p ( C D ) p ( B D ) and p ( A B C ) p ( A ) = p ( A B ) p ( A C ) .
p ( A C D ) p ( C ) = p ( C D ) p ( A C ) and p ( A B D ) p ( B ) = p ( A B ) p ( B D ) .
p ( ( B C D ) | D ) = p ( ( C D ) | D ) p ( ( B D ) | D ) and p ( ( A B C ) | A ) = p ( ( A B ) | A ) p ( ( A C ) | A ) .
p ( U V W ) = X p ( X U V ) p ( X V W ) p ( X V ) = X p ( U V ) p ( X V ) p ( X V W ) p ( V ) p ( X V ) = X p ( U V ) p ( X V W ) p ( V ) = p ( U V ) p ( V W ) p ( V ) ,
p ( A 11 A 12 A 13 A 21 A 22 A 23 A 31 A 32 A 33 ) = p ( A 11 A 12 A 13 A 21 A 22 A 23 ) p ( A 21 A 22 A 23 A 31 A 32 A 33 ) p ( A 21 A 22 A 23 ) = p ( A 11 A 12 A 21 A 22 ) p ( A 12 A 13 A 22 A 23 ) p ( A 12 A 22 ) p ( A 21 A 22 A 31 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 A 32 ) 1 p ( A 21 A 22 A 23 ) = p ( A 11 A 12 A 21 A 22 ) p ( A 12 A 13 A 22 A 23 ) p ( A 21 A 22 A 31 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 ) p ( A 12 A 22 ) p ( A 22 A 32 ) p ( A 21 A 22 ) p ( A 22 A 23 ) .
A 11 , A 12 , A 13 , A 21 , A 31 p ( A 11 A 12 A 13 A 21 A 22 A 23 A 31 A 32 A 33 ) = A 11 , A 12 , A 13 , A 21 , A 31 p ( A 11 A 12 A 21 A 22 ) p ( A 12 A 13 A 22 A 23 ) p ( A 21 A 22 A 31 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 ) p ( A 12 A 22 ) p ( A 22 A 32 ) p ( A 21 A 22 ) p ( A 22 A 23 ) = A 12 , A 21 p ( A 12 A 21 A 22 ) p ( A 12 A 22 A 23 ) p ( A 21 A 22 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 ) p ( A 12 A 22 ) p ( A 22 A 32 ) p ( A 21 A 22 ) p ( A 22 A 23 ) = A 12 , A 21 p ( A 21 A 22 ) p ( A 12 A 22 ) p ( A 22 ) p ( A 12 A 22 A 23 ) p ( A 21 A 22 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 ) p ( A 12 A 22 ) p ( A 22 A 32 ) p ( A 21 A 22 ) p ( A 22 A 23 ) = A 12 , A 21 p ( A 12 A 22 A 23 ) p ( A 21 A 22 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 A 32 ) p ( A 22 A 23 ) = p ( A 22 A 23 ) p ( A 22 A 32 ) p ( A 22 A 23 A 32 A 33 ) p ( A 22 A 32 ) p ( A 22 A 23 ) = p ( A 22 A 23 A 32 A 33 ) .
p ( B C D ) = 1 8 ( 1 ( ( 1 ) B + ( 1 ) C + ( 1 ) D ) γ + ( 1 ) C + D β + ( 1 ) B + D β | + ( 1 ) B + C β / ( 1 ) B + C + D θ ) p ( B D ) = 1 4 ( 1 ( ( 1 ) B + ( 1 ) D ) γ + ( 1 ) B + D β | ) p ( C D ) = 1 4 ( 1 ( ( 1 ) C + ( 1 ) D ) γ + ( 1 ) C + D β ) p ( D ) = 1 2 ( 1 ( 1 ) D γ ) .
( 1 ) D θ + β / ( 1 ) D γ β / + γ θ = γ 2 ( 1 ) D γ ( β + β | ) + β β | .
β / + γ θ = γ 2 + β β | ,
θ + γ β / = γ ( β + β | ) .
β / + γ θ = γ 2 + β β | ,
θ + γ β / = γ ( β + β | ) .
θ
F C ( x ) = 1 4 + 3 4 ( 1 2 ( x / a r ) b r ) .
φ ( s 1 s 2 s 3 s 4 ) = A 1 = 0 G 1 A 2 = 0 G 1 A 3 = 0 G 1 A 4 = 0 G 1 p ( A 1 A 2 A 3 A 4 ) e 2 π i A 1 s 1 G e 2 π i A 2 s 2 G e 2 π i A 3 s 3 G e 2 π i A 4 s 4 G = A 1 , A 2 , A 3 , A 4 p ( A 1 A 2 A 3 A 4 ) e ( 2 π i G ) ( A 1 s 1 + A 2 s 2 + A 3 s 3 + A 4 s 4 ) .
p ( A 1 A 2 A 3 A 4 ) = 1 G 4 s 1 , s 2 , s 3 , s 4 φ ( s 1 s 2 s 3 s 4 ) e ( 2 π i G ) ( A 1 s 1 + A 2 s 2 + A 3 s 3 + A 4 s 4 ) ,
D KL ( P Q ) = S p ( S ) ln p ( S ) q ( S )
p ( S ) = Z ( S ) Z ,
Z ( S ) = exp ( i μ i n ( S , i ) φ i ( S ) )
Z = S Z ( S ) .
H = S p ( S ) ln p ( S ) .
H = S p ( S ) ln Z ( S ) + ln Z ,
H = S p ( S ) i μ i n ( S , i ) φ i ( S ) + ln Z .
H = i μ i n ( S , i ) φ i 0 + ln Z .
h = lim n 1 n H n = i μ i φ i 0 + lim n 1 n ln Z n ,
μ i = 1 2 ln ( 1 + φ i 0 ) 1 2 ln ( 1 φ i 0 ) ,
Z n = 2 n ( 1 ( 1 + φ i 0 ) ( 1 φ i 0 ) ) n n init ,
lim n 1 n ln Z n = ln 2 1 2 ln ( 1 + φ i 0 ) 1 2 ln ( 1 φ i 0 ) .
h = ( 1 + φ i 0 2 ln 1 + φ i 0 2 + 1 φ i 0 2 ln 1 φ i 0 2 ) .
1 n ln Z n μ i = φ i 0 .
1 n ln Z n μ i 1 n j μ j 2 ln Z n μ j μ i .
2 ln Z n μ j μ i = μ j ( 1 Z n Z n μ i ) = ( 1 Z n Z n μ j ) ( 1 Z n Z n μ i ) + 1 Z n 2 Z n μ j μ i .
1 Z n Z n μ i = n φ i ,
1 Z n 2 Z n μ j μ i = 1 Z n 2 μ j μ i S exp ( n k μ k φ k ( S ) ) = 1 Z n S n 2 φ j ( S ) φ i ( S ) exp ( k μ k n ( S , k ) φ k ( S ) ) = S n 2 p ( S ) φ j ( S ) φ i ( S ) .
2 ln Z n μ j μ i = n 2 ( φ j φ i φ j φ i ) = n 2 ( φ j φ j ) ( φ i φ i ) .
2 ln Z n μ i μ j | μ i = μ j = 0 = n δ i j .
1 n ln Z n μ i 1 n j μ j 2 ln Z n μ j μ i μ i .
μ i φ i 0 .
ln Z n ( μ ) ln Z n ( 0 ) + 1 2 i , j 2 ln Z n μ i μ j | μ i = μ j = 0 μ i μ j = ln Z n ( 0 ) + n 2 | μ | 2 = n ln 2 + n 2 | μ | 2 .
h = lim n 1 n H n = ln 2 1 2 i ( φ i 0 ) 2 .
p ( A B D E G H ) = p ( A B D E ) p ( D E G H | A B D E ) = p ( A B D E ) p ( D E G H ) p ( D E ) .
p ( A B C D E F G H I ) = p ( A B D E G H ) p ( B C E F ) p ( B E ) p ( E F H I ) p ( E F H ) .
A , D , G p ( A B C D E F G H I ) = A , D , G p ( A B D E G H ) p ( B C E F ) p ( B E ) p ( E F H I ) p ( E F H ) = A , D , G p ( A B D E ) p ( D E G H ) p ( D E ) p ( B C E F ) p ( B E ) p ( E F H I ) p ( E F H ) = D p ( B D E ) p ( D E H ) p ( D E ) p ( B C E F ) p ( B E ) p ( E F H I ) p ( E F H ) .
A , D , G p ( A B C D E F G H I ) = D p ( B D E ) p ( D E ) p ( D E ) p ( E H ) p ( E ) p ( B C E F ) p ( E F H I ) p ( B E ) p ( E ) p ( E F ) p ( E H ) = D p ( B D E ) p ( B C E F ) p ( E F H I ) p ( B E ) p ( E F ) = p ( B E ) p ( B C E F ) p ( E F H I ) p ( B E ) p ( E F ) = p ( B C E F ) p ( E F H I ) p ( E F ) .
p ( A 1 A 2 A 3 ) = 1 8 s 1 , s 2 , s 3 φ ( s 1 s 2 s 3 ) ( 1 ) A 1 s 1 + A 2 s 2 + A 3 s 3 ,
φ ( 0 0 0 ) a n d φ ( 1 1 1 ) = g .
p ( A 1 A 2 A 3 A 4 ) = p (