A wide variety of recent studies have argued that the human visual system provides an efficient means of processing the information in the natural environment. However, the amount of information (entropy) in the signal can be estimated in a number of ways, and it is has been unclear how much of the information is carried by the different sources of redundancy. The primary difficulty is that there has been no rational way to estimate the entropy of such complex scenes. In this paper, we provide a technique that uses a recent approach to estimating the entropy and dimensionality of natural scenes [D. M. Chandler and D. J. Field, J. Opt. Soc. Am. A 24, 922–941 (2007)] to estimate the amount of information attributable to the power and phase spectra in natural-scene patches. By comparing the entropies of patches that have swapped phase spectra and fixed phase spectra, we demonstrate how to estimate both the amount of information in each type of spectrum and the amount of information that is shared by these spectra (mutual information). We applied this technique to small patches ( and ). From our estimates, we show that the power spectrum of patches carries approximately 54% of the total information, the phase spectrum carries 56%, and 10% is mutual information (). This technique is currently limited to relatively small image patches, due to the number of patches currently in our collection (on the order of ). However, the technique can, in theory, be extended to larger images. Even with these relatively small patches, we discuss how these results can provide important insights into both compression techniques and efficient coding techniques that work with relatively small image patches (e.g., JPEG, sparse coding, independent components analysis).
© 2012 Optical Society of AmericaFull Article | PDF Article
Mitchell G. A. Thomson
J. Opt. Soc. Am. A 16(7) 1549-1553 (1999)
Michael S. Lewicki and Bruno A. Olshausen
J. Opt. Soc. Am. A 16(7) 1587-1601 (1999)
Stéphane J. M. Rainville and Frederick A. A. Kingdom
J. Opt. Soc. Am. A 16(9) 2112-2123 (1999)