K. L. Clarkson, "Nearest neighbor searching and metric space dimensions," in Nearest-Neighbor Methods for Learning and Vision: Theory and Practice, G.Shakhnarovich, T.Darrell, and P.Indyk, eds. (MIT Press, 2006), Chap. 2, pp. 15-59.

G. Shakhnarovich, T. Darrell, and P. Indyk, Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (MIT Press, 2006).

J. Kybic, "High-dimensional mutual information estimation for image registration," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2004), pp. 1779-1782.

J. Costa and A. O. Hero, "Geodesic entropic graphs for dimension and entropy estimation in manifold learning," IEEE Trans. Signal Process. 52, 2210-2221 (2004).

[Crossref]

A. Kraskov, H. Stgbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E 69, 066138 (2004).

[Crossref]

I. Nemenman, W. Bialek, and R. de Ruyter van Steveninck, "Entropy and information in neural spike trains: progress on the sampling problem," Phys. Rev. E 69, 056111 (2004).

[Crossref]

Y. Petrov and L. Zhoaping, "Local correlations, information redundancy, and the sufficient pixel depth in natural images," J. Opt. Soc. Am. A 20, 56-66 (2003).

[Crossref]

A. B. Lee, K. S. Pedersen, and D. Mumford, "The nonlinear statistics of high-contrast patches in natural images," Int. J. Comput. Vis. 54, 83-103 (2003).

[Crossref]

J. D. Victor, "Binless strategies for estimation of information from neural data," Phys. Rev. E 66, 051903 (2002).

[Crossref]

I. Nemenman, F. Shafee, and W. Bialek, "Entropy and inference, revisited," in Advances in Neural Information Processing Systems, Vol. 14, T.G.Dietterich, S.Becker, and Z.Ghahramani, eds. (MIT Press, 2002).

O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nat. Neurosci. 4, 819-825 (2001).

[Crossref]

E. P. Simoncelli and B. A. Olshausen, "Natural image statistics and neural representation," Annu. Rev. Neurosci. 24, 1193-1216 (2001).

[Crossref]

W. S. Geisler, J. S. Perry, B. J. Super, and D. P. Gallogly, "Edge co-occurence in natural images predicts contour grouping performance," Vision Res. 41, 711-724 (2001).

[Crossref]

M. Wainwright, E. P. Simoncelli, and A. Willsky, "Random cascades on wavelet trees and their use in modeling and analyzing natural imagery," Appl. Comput. Harmon. Anal. 11, 89-123 (2001).

[Crossref]

International Organization for Standardization, "Information technology--JPEG 2000 image coding system: core coding system," Tech. Rep. ISO/IEC FDIS15444-1:2000 (International Organization for Standardization, 2000).

J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science 290, 2319-2323 (2000).

[Crossref]
[PubMed]

S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science 290, 2323-2326 (2000).

[Crossref]
[PubMed]

J. H. van Hateren and A. van der Schaaf, "Independent component filters of natural images compared with simple cells in primary visual cortex," Proc. R. Soc. London, Ser. B 265, 359-366 (1998).

[Crossref]

R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inf. Theory 44, 2325-2384 (1998).

[Crossref]

J. Minguillon and J. Pujol, "Uniform quantization error for Laplacian sources with applications to JPEG standard," in Mathematics of Data/Image Coding, Compression, and Encryption, M.S.Schmalz, ed., Proc. SPIE 3456, 77-88 (1998).

T. S. Lee, D. Mumford, R. Romero, and V. A. F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Res. 38, 2429-2454 (1998).

[Crossref]
[PubMed]

A. J. Bell and T. J. Sejnowski, "The independent components of natural scenes are edge filters," Vision Res. 37, 3327-3338 (1997).

[Crossref]

S. Verdu and T. Han, "The role of the asymptotic equipartition property in noiseless source coding," IEEE Trans. Inf. Theory 43, 847-857 (1997).

[Crossref]

B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: a strategy employed by V1?" Vision Res. 37, 3311-3325 (1996).

[Crossref]

D. L. Ruderman and W. Bialek, "Statistics of natural images: scaling in the woods," Phys. Rev. Lett. 73, 814-817 (1994).

[Crossref]
[PubMed]

D. J. Field, "What is the goal of sensory coding?" Neural Comput. 6, 559-601 (1994).

[Crossref]

W. B. Pennebaker and J. L. Mitchell, The JPEG Still Image Data Compression Standard (Van Nostrand Reinhold, 1993).

D. J. Field, "Scale-invariance and self-similar 'Wavelet' transforms: an analysis of natural scenes and mammalian visual systems," in Wavelets, Fractals and Fourier Transforms: New Developments and New Applications (Oxford U. Press, 1993), pp. 151-193.

J. J. Atick, "Could information theory provide an ecological theory of sensory processing?" Network 3, 213-251 (1992).

[Crossref]

T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications (Wiley, 1991).

[Crossref]

W. van de Water and P. Schram, "Generalized dimensions from near-neighbor information," Phys. Rev. A 37, 3118-3125 (1988).

[Crossref]
[PubMed]

R. Badii and A. Politi, "Statistical description of chaotic attractors: the dimension function," J. Stat. Phys. 40, 725-750 (1985).

[Crossref]

P. Grassberger, "Generalizations of the Hausdorff dimension of fractal measures," Phys. Lett. A 107, 101-105 (1985).

[Crossref]

J. Guckenheimer and G. Buzyna, "Dimension measurements for geostrophic turbulence," Phys. Rev. Lett. 51, 1438-1441 (1983).

[Crossref]

K. Pettis, T. Bailey, A. K. Jain, and R. Dubes, "An intrinsic dimensionality estimator from near-neighbor information," IEEE Trans. Pattern Anal. Mach. Intell. 1, 25-36 (1979).

[Crossref]
[PubMed]

B. V. Dasarathy, Nearest Neighbour (NN) Norms: NN Pattern Classification Techniques (IEEE, 1973).

N. S. Tzannes, R. V. Spencer, and A. Kaplan, "On estimating the entropy of random fields," Inf. Control. 16, 1-6 (1970).

[Crossref]

J. R. Parks, "Prediction and entropy of half-tone pictures," Behav. Sci. 10, 436-445 (1965).

[Crossref]
[PubMed]

J. A. Nelder and R. Mead, "A simplex method for function minimization," J. Comput. 7, 308-313 (1965).

N. G. Deriugin, "The power spectrum and the correlation function of the television signal," Telecommun. 1, 1-12 (1957).

C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J. 27, 623-656 (1948).

Z. E. Schnabel, "The estimation of total fish population of a lake," Am. Math. Monthly 45, 348-352 (1938).

[Crossref]

J. J. Atick, "Could information theory provide an ecological theory of sensory processing?" Network 3, 213-251 (1992).

[Crossref]

R. Badii and A. Politi, "Statistical description of chaotic attractors: the dimension function," J. Stat. Phys. 40, 725-750 (1985).

[Crossref]

K. Pettis, T. Bailey, A. K. Jain, and R. Dubes, "An intrinsic dimensionality estimator from near-neighbor information," IEEE Trans. Pattern Anal. Mach. Intell. 1, 25-36 (1979).

[Crossref]
[PubMed]

A. J. Bell and T. J. Sejnowski, "The independent components of natural scenes are edge filters," Vision Res. 37, 3327-3338 (1997).

[Crossref]

I. Nemenman, W. Bialek, and R. de Ruyter van Steveninck, "Entropy and information in neural spike trains: progress on the sampling problem," Phys. Rev. E 69, 056111 (2004).

[Crossref]

I. Nemenman, F. Shafee, and W. Bialek, "Entropy and inference, revisited," in Advances in Neural Information Processing Systems, Vol. 14, T.G.Dietterich, S.Becker, and Z.Ghahramani, eds. (MIT Press, 2002).

D. L. Ruderman and W. Bialek, "Statistics of natural images: scaling in the woods," Phys. Rev. Lett. 73, 814-817 (1994).

[Crossref]
[PubMed]

J. Guckenheimer and G. Buzyna, "Dimension measurements for geostrophic turbulence," Phys. Rev. Lett. 51, 1438-1441 (1983).

[Crossref]

K. L. Clarkson, "Nearest neighbor searching and metric space dimensions," in Nearest-Neighbor Methods for Learning and Vision: Theory and Practice, G.Shakhnarovich, T.Darrell, and P.Indyk, eds. (MIT Press, 2006), Chap. 2, pp. 15-59.

J. Costa and A. O. Hero, "Geodesic entropic graphs for dimension and entropy estimation in manifold learning," IEEE Trans. Signal Process. 52, 2210-2221 (2004).

[Crossref]

T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications (Wiley, 1991).

[Crossref]

G. Shakhnarovich, T. Darrell, and P. Indyk, Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (MIT Press, 2006).

B. V. Dasarathy, Nearest Neighbour (NN) Norms: NN Pattern Classification Techniques (IEEE, 1973).

I. Nemenman, W. Bialek, and R. de Ruyter van Steveninck, "Entropy and information in neural spike trains: progress on the sampling problem," Phys. Rev. E 69, 056111 (2004).

[Crossref]

J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science 290, 2319-2323 (2000).

[Crossref]
[PubMed]

N. G. Deriugin, "The power spectrum and the correlation function of the television signal," Telecommun. 1, 1-12 (1957).

K. Pettis, T. Bailey, A. K. Jain, and R. Dubes, "An intrinsic dimensionality estimator from near-neighbor information," IEEE Trans. Pattern Anal. Mach. Intell. 1, 25-36 (1979).

[Crossref]
[PubMed]

B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: a strategy employed by V1?" Vision Res. 37, 3311-3325 (1996).

[Crossref]

D. J. Field, "What is the goal of sensory coding?" Neural Comput. 6, 559-601 (1994).

[Crossref]

D. J. Field, "Scale-invariance and self-similar 'Wavelet' transforms: an analysis of natural scenes and mammalian visual systems," in Wavelets, Fractals and Fourier Transforms: New Developments and New Applications (Oxford U. Press, 1993), pp. 151-193.

D. J. Field, "Relations between the statistics of natural images and the response properties of cortical cells," J. Opt. Soc. Am. A 4, 2379-2394 (1987).

[Crossref]
[PubMed]

W. S. Geisler, J. S. Perry, B. J. Super, and D. P. Gallogly, "Edge co-occurence in natural images predicts contour grouping performance," Vision Res. 41, 711-724 (2001).

[Crossref]

W. S. Geisler, J. S. Perry, B. J. Super, and D. P. Gallogly, "Edge co-occurence in natural images predicts contour grouping performance," Vision Res. 41, 711-724 (2001).

[Crossref]

A. Kraskov, H. Stgbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E 69, 066138 (2004).

[Crossref]

P. Grassberger, "Generalizations of the Hausdorff dimension of fractal measures," Phys. Lett. A 107, 101-105 (1985).

[Crossref]

R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inf. Theory 44, 2325-2384 (1998).

[Crossref]

J. Guckenheimer and G. Buzyna, "Dimension measurements for geostrophic turbulence," Phys. Rev. Lett. 51, 1438-1441 (1983).

[Crossref]

S. Verdu and T. Han, "The role of the asymptotic equipartition property in noiseless source coding," IEEE Trans. Inf. Theory 43, 847-857 (1997).

[Crossref]

J. Costa and A. O. Hero, "Geodesic entropic graphs for dimension and entropy estimation in manifold learning," IEEE Trans. Signal Process. 52, 2210-2221 (2004).

[Crossref]

G. Shakhnarovich, T. Darrell, and P. Indyk, Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (MIT Press, 2006).

K. Pettis, T. Bailey, A. K. Jain, and R. Dubes, "An intrinsic dimensionality estimator from near-neighbor information," IEEE Trans. Pattern Anal. Mach. Intell. 1, 25-36 (1979).

[Crossref]
[PubMed]

N. S. Tzannes, R. V. Spencer, and A. Kaplan, "On estimating the entropy of random fields," Inf. Control. 16, 1-6 (1970).

[Crossref]

L. F. Kozachenko and N. N. Leonenko, "A statistical estimate for the entropy of a random vector," Probl. Inf. Transm. 23, 9-16 (1987).

A. Kraskov, H. Stgbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E 69, 066138 (2004).

[Crossref]

J. Kybic, "High-dimensional mutual information estimation for image registration," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2004), pp. 1779-1782.

T. S. Lee, D. Mumford, R. Romero, and V. A. F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Res. 38, 2429-2454 (1998).

[Crossref]
[PubMed]

J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science 290, 2319-2323 (2000).

[Crossref]
[PubMed]

A. B. Lee, K. S. Pedersen, and D. Mumford, "The nonlinear statistics of high-contrast patches in natural images," Int. J. Comput. Vis. 54, 83-103 (2003).

[Crossref]

T. S. Lee, D. Mumford, R. Romero, and V. A. F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Res. 38, 2429-2454 (1998).

[Crossref]
[PubMed]

L. F. Kozachenko and N. N. Leonenko, "A statistical estimate for the entropy of a random vector," Probl. Inf. Transm. 23, 9-16 (1987).

J. A. Nelder and R. Mead, "A simplex method for function minimization," J. Comput. 7, 308-313 (1965).

J. Minguillon and J. Pujol, "Uniform quantization error for Laplacian sources with applications to JPEG standard," in Mathematics of Data/Image Coding, Compression, and Encryption, M.S.Schmalz, ed., Proc. SPIE 3456, 77-88 (1998).

W. B. Pennebaker and J. L. Mitchell, The JPEG Still Image Data Compression Standard (Van Nostrand Reinhold, 1993).

A. B. Lee, K. S. Pedersen, and D. Mumford, "The nonlinear statistics of high-contrast patches in natural images," Int. J. Comput. Vis. 54, 83-103 (2003).

[Crossref]

T. S. Lee, D. Mumford, R. Romero, and V. A. F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Res. 38, 2429-2454 (1998).

[Crossref]
[PubMed]

J. A. Nelder and R. Mead, "A simplex method for function minimization," J. Comput. 7, 308-313 (1965).

I. Nemenman, W. Bialek, and R. de Ruyter van Steveninck, "Entropy and information in neural spike trains: progress on the sampling problem," Phys. Rev. E 69, 056111 (2004).

[Crossref]

I. Nemenman, F. Shafee, and W. Bialek, "Entropy and inference, revisited," in Advances in Neural Information Processing Systems, Vol. 14, T.G.Dietterich, S.Becker, and Z.Ghahramani, eds. (MIT Press, 2002).

R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inf. Theory 44, 2325-2384 (1998).

[Crossref]

E. P. Simoncelli and B. A. Olshausen, "Natural image statistics and neural representation," Annu. Rev. Neurosci. 24, 1193-1216 (2001).

[Crossref]

B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: a strategy employed by V1?" Vision Res. 37, 3311-3325 (1996).

[Crossref]

J. R. Parks, "Prediction and entropy of half-tone pictures," Behav. Sci. 10, 436-445 (1965).

[Crossref]
[PubMed]

A. B. Lee, K. S. Pedersen, and D. Mumford, "The nonlinear statistics of high-contrast patches in natural images," Int. J. Comput. Vis. 54, 83-103 (2003).

[Crossref]

W. B. Pennebaker and J. L. Mitchell, The JPEG Still Image Data Compression Standard (Van Nostrand Reinhold, 1993).

W. S. Geisler, J. S. Perry, B. J. Super, and D. P. Gallogly, "Edge co-occurence in natural images predicts contour grouping performance," Vision Res. 41, 711-724 (2001).

[Crossref]

K. Pettis, T. Bailey, A. K. Jain, and R. Dubes, "An intrinsic dimensionality estimator from near-neighbor information," IEEE Trans. Pattern Anal. Mach. Intell. 1, 25-36 (1979).

[Crossref]
[PubMed]

R. Badii and A. Politi, "Statistical description of chaotic attractors: the dimension function," J. Stat. Phys. 40, 725-750 (1985).

[Crossref]

J. Minguillon and J. Pujol, "Uniform quantization error for Laplacian sources with applications to JPEG standard," in Mathematics of Data/Image Coding, Compression, and Encryption, M.S.Schmalz, ed., Proc. SPIE 3456, 77-88 (1998).

T. S. Lee, D. Mumford, R. Romero, and V. A. F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Res. 38, 2429-2454 (1998).

[Crossref]
[PubMed]

S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science 290, 2323-2326 (2000).

[Crossref]
[PubMed]

D. L. Ruderman and W. Bialek, "Statistics of natural images: scaling in the woods," Phys. Rev. Lett. 73, 814-817 (1994).

[Crossref]
[PubMed]

S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science 290, 2323-2326 (2000).

[Crossref]
[PubMed]

Z. E. Schnabel, "The estimation of total fish population of a lake," Am. Math. Monthly 45, 348-352 (1938).

[Crossref]

W. van de Water and P. Schram, "Generalized dimensions from near-neighbor information," Phys. Rev. A 37, 3118-3125 (1988).

[Crossref]
[PubMed]

O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nat. Neurosci. 4, 819-825 (2001).

[Crossref]

A. J. Bell and T. J. Sejnowski, "The independent components of natural scenes are edge filters," Vision Res. 37, 3327-3338 (1997).

[Crossref]

I. Nemenman, F. Shafee, and W. Bialek, "Entropy and inference, revisited," in Advances in Neural Information Processing Systems, Vol. 14, T.G.Dietterich, S.Becker, and Z.Ghahramani, eds. (MIT Press, 2002).

G. Shakhnarovich, T. Darrell, and P. Indyk, Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (MIT Press, 2006).

C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J. 27, 623-656 (1948).

M. Wainwright, E. P. Simoncelli, and A. Willsky, "Random cascades on wavelet trees and their use in modeling and analyzing natural imagery," Appl. Comput. Harmon. Anal. 11, 89-123 (2001).

[Crossref]

O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nat. Neurosci. 4, 819-825 (2001).

[Crossref]

E. P. Simoncelli and B. A. Olshausen, "Natural image statistics and neural representation," Annu. Rev. Neurosci. 24, 1193-1216 (2001).

[Crossref]

N. S. Tzannes, R. V. Spencer, and A. Kaplan, "On estimating the entropy of random fields," Inf. Control. 16, 1-6 (1970).

[Crossref]

A. Kraskov, H. Stgbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E 69, 066138 (2004).

[Crossref]

W. S. Geisler, J. S. Perry, B. J. Super, and D. P. Gallogly, "Edge co-occurence in natural images predicts contour grouping performance," Vision Res. 41, 711-724 (2001).

[Crossref]

J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science 290, 2319-2323 (2000).

[Crossref]
[PubMed]

T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications (Wiley, 1991).

[Crossref]

N. S. Tzannes, R. V. Spencer, and A. Kaplan, "On estimating the entropy of random fields," Inf. Control. 16, 1-6 (1970).

[Crossref]

W. van de Water and P. Schram, "Generalized dimensions from near-neighbor information," Phys. Rev. A 37, 3118-3125 (1988).

[Crossref]
[PubMed]

J. H. van Hateren and A. van der Schaaf, "Independent component filters of natural images compared with simple cells in primary visual cortex," Proc. R. Soc. London, Ser. B 265, 359-366 (1998).

[Crossref]

J. H. van Hateren and A. van der Schaaf, "Independent component filters of natural images compared with simple cells in primary visual cortex," Proc. R. Soc. London, Ser. B 265, 359-366 (1998).

[Crossref]

S. Verdu and T. Han, "The role of the asymptotic equipartition property in noiseless source coding," IEEE Trans. Inf. Theory 43, 847-857 (1997).

[Crossref]

J. D. Victor, "Binless strategies for estimation of information from neural data," Phys. Rev. E 66, 051903 (2002).

[Crossref]

M. Wainwright, E. P. Simoncelli, and A. Willsky, "Random cascades on wavelet trees and their use in modeling and analyzing natural imagery," Appl. Comput. Harmon. Anal. 11, 89-123 (2001).

[Crossref]

E. W. Weisstein, "Birthday problem," from MathWorld--A Wolfram Web Resource, http://mathworld.wolfram.com/BirthdayProblem.html.

E. W. Weisstein, "Chi-squared distribution," from MathWorld--A Wolfram Web Resource, http://mathworld.wolfram.com/Chi-SquaredDistribution.html.

M. Wainwright, E. P. Simoncelli, and A. Willsky, "Random cascades on wavelet trees and their use in modeling and analyzing natural imagery," Appl. Comput. Harmon. Anal. 11, 89-123 (2001).

[Crossref]

Z. E. Schnabel, "The estimation of total fish population of a lake," Am. Math. Monthly 45, 348-352 (1938).

[Crossref]

E. P. Simoncelli and B. A. Olshausen, "Natural image statistics and neural representation," Annu. Rev. Neurosci. 24, 1193-1216 (2001).

[Crossref]

M. Wainwright, E. P. Simoncelli, and A. Willsky, "Random cascades on wavelet trees and their use in modeling and analyzing natural imagery," Appl. Comput. Harmon. Anal. 11, 89-123 (2001).

[Crossref]

J. R. Parks, "Prediction and entropy of half-tone pictures," Behav. Sci. 10, 436-445 (1965).

[Crossref]
[PubMed]

C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J. 27, 623-656 (1948).

S. Verdu and T. Han, "The role of the asymptotic equipartition property in noiseless source coding," IEEE Trans. Inf. Theory 43, 847-857 (1997).

[Crossref]

R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inf. Theory 44, 2325-2384 (1998).

[Crossref]

K. Pettis, T. Bailey, A. K. Jain, and R. Dubes, "An intrinsic dimensionality estimator from near-neighbor information," IEEE Trans. Pattern Anal. Mach. Intell. 1, 25-36 (1979).

[Crossref]
[PubMed]

J. Costa and A. O. Hero, "Geodesic entropic graphs for dimension and entropy estimation in manifold learning," IEEE Trans. Signal Process. 52, 2210-2221 (2004).

[Crossref]

N. S. Tzannes, R. V. Spencer, and A. Kaplan, "On estimating the entropy of random fields," Inf. Control. 16, 1-6 (1970).

[Crossref]

A. B. Lee, K. S. Pedersen, and D. Mumford, "The nonlinear statistics of high-contrast patches in natural images," Int. J. Comput. Vis. 54, 83-103 (2003).

[Crossref]

J. A. Nelder and R. Mead, "A simplex method for function minimization," J. Comput. 7, 308-313 (1965).

J. Theiler, "Estimating fractal dimension," J. Opt. Soc. Am. A 7, 1055-1073 (1990).

[Crossref]

D. J. Field, "Relations between the statistics of natural images and the response properties of cortical cells," J. Opt. Soc. Am. A 4, 2379-2394 (1987).

[Crossref]
[PubMed]

D. Kersten, "Predictability and redundancy of natural images," J. Opt. Soc. Am. A 4, 2395-2400 (1987).

[Crossref]

Y. Petrov and L. Zhoaping, "Local correlations, information redundancy, and the sufficient pixel depth in natural images," J. Opt. Soc. Am. A 20, 56-66 (2003).

[Crossref]

R. Badii and A. Politi, "Statistical description of chaotic attractors: the dimension function," J. Stat. Phys. 40, 725-750 (1985).

[Crossref]

O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nat. Neurosci. 4, 819-825 (2001).

[Crossref]

J. J. Atick, "Could information theory provide an ecological theory of sensory processing?" Network 3, 213-251 (1992).

[Crossref]

D. J. Field, "What is the goal of sensory coding?" Neural Comput. 6, 559-601 (1994).

[Crossref]

P. Grassberger, "Generalizations of the Hausdorff dimension of fractal measures," Phys. Lett. A 107, 101-105 (1985).

[Crossref]

W. van de Water and P. Schram, "Generalized dimensions from near-neighbor information," Phys. Rev. A 37, 3118-3125 (1988).

[Crossref]
[PubMed]

I. Nemenman, W. Bialek, and R. de Ruyter van Steveninck, "Entropy and information in neural spike trains: progress on the sampling problem," Phys. Rev. E 69, 056111 (2004).

[Crossref]

J. D. Victor, "Binless strategies for estimation of information from neural data," Phys. Rev. E 66, 051903 (2002).

[Crossref]

A. Kraskov, H. Stgbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E 69, 066138 (2004).

[Crossref]

D. L. Ruderman and W. Bialek, "Statistics of natural images: scaling in the woods," Phys. Rev. Lett. 73, 814-817 (1994).

[Crossref]
[PubMed]

J. Guckenheimer and G. Buzyna, "Dimension measurements for geostrophic turbulence," Phys. Rev. Lett. 51, 1438-1441 (1983).

[Crossref]

L. F. Kozachenko and N. N. Leonenko, "A statistical estimate for the entropy of a random vector," Probl. Inf. Transm. 23, 9-16 (1987).

J. H. van Hateren and A. van der Schaaf, "Independent component filters of natural images compared with simple cells in primary visual cortex," Proc. R. Soc. London, Ser. B 265, 359-366 (1998).

[Crossref]

J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science 290, 2319-2323 (2000).

[Crossref]
[PubMed]

S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science 290, 2323-2326 (2000).

[Crossref]
[PubMed]

N. G. Deriugin, "The power spectrum and the correlation function of the television signal," Telecommun. 1, 1-12 (1957).

W. S. Geisler, J. S. Perry, B. J. Super, and D. P. Gallogly, "Edge co-occurence in natural images predicts contour grouping performance," Vision Res. 41, 711-724 (2001).

[Crossref]

T. S. Lee, D. Mumford, R. Romero, and V. A. F. Lamme, "The role of the primary visual cortex in higher level vision," Vision Res. 38, 2429-2454 (1998).

[Crossref]
[PubMed]

B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: a strategy employed by V1?" Vision Res. 37, 3311-3325 (1996).

[Crossref]

A. J. Bell and T. J. Sejnowski, "The independent components of natural scenes are edge filters," Vision Res. 37, 3327-3338 (1997).

[Crossref]

W. B. Pennebaker and J. L. Mitchell, The JPEG Still Image Data Compression Standard (Van Nostrand Reinhold, 1993).

International Organization for Standardization, "Information technology--JPEG 2000 image coding system: core coding system," Tech. Rep. ISO/IEC FDIS15444-1:2000 (International Organization for Standardization, 2000).

D. J. Field, "Scale-invariance and self-similar 'Wavelet' transforms: an analysis of natural scenes and mammalian visual systems," in Wavelets, Fractals and Fourier Transforms: New Developments and New Applications (Oxford U. Press, 1993), pp. 151-193.

J. Minguillon and J. Pujol, "Uniform quantization error for Laplacian sources with applications to JPEG standard," in Mathematics of Data/Image Coding, Compression, and Encryption, M.S.Schmalz, ed., Proc. SPIE 3456, 77-88 (1998).

I. Nemenman, F. Shafee, and W. Bialek, "Entropy and inference, revisited," in Advances in Neural Information Processing Systems, Vol. 14, T.G.Dietterich, S.Becker, and Z.Ghahramani, eds. (MIT Press, 2002).

E. W. Weisstein, "Birthday problem," from MathWorld--A Wolfram Web Resource, http://mathworld.wolfram.com/BirthdayProblem.html.

T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications (Wiley, 1991).

[Crossref]

K. L. Clarkson, "Nearest neighbor searching and metric space dimensions," in Nearest-Neighbor Methods for Learning and Vision: Theory and Practice, G.Shakhnarovich, T.Darrell, and P.Indyk, eds. (MIT Press, 2006), Chap. 2, pp. 15-59.

In this paper, we estimate differential entropy assuming that the original images are drawn from an underlying continuous distribution. Under high-rate quantization, the discrete entropy H is related to the differential entropy h by H?h+log?, where ? is the quantization step size (here ?=1, log?=0); see Refs. .

see http://redwood.psych.cornell.edu/proximity/.

J. Kybic, "High-dimensional mutual information estimation for image registration," in Proceedings of IEEE International Conference on Image Processing (IEEE, 2004), pp. 1779-1782.

As long as the means are identical, the distance between two patches does not depend on the (common) mean of the underlying Gaussian from which the pixels are drawn.

E. W. Weisstein, "Chi-squared distribution," from MathWorld--A Wolfram Web Resource, http://mathworld.wolfram.com/Chi-SquaredDistribution.html.

Although the pixel values of the spectrum-equalized noise patches were correlated (and were therefore statistically dependent), the real and imaginary components of the DFT coefficients of each block were independent. Accordingly, the entropy of the spectrum-equalized noise was computed by summing the individual entropies of the real and imaginary part of each DFT coefficient; the individual entropies were computed via Eq. (9).

B. V. Dasarathy, Nearest Neighbour (NN) Norms: NN Pattern Classification Techniques (IEEE, 1973).

G. Shakhnarovich, T. Darrell, and P. Indyk, Nearest-Neighbor Methods in Learning and Vision: Theory and Practice (MIT Press, 2006).