Abstract

The two entropy expressions, log<i>B</i> and – <i>B</i> log<i>B</i> (where <i>B</i> is the local brightness of the object or its spatial spectral power) used in maximum entropy (ME) image restoration, are derived as limiting cases of a general entropy formula. The brightness <i>B</i> is represented by the <i>n</i> photons emitted from a small unit area of the object and imaged in the receiver. These <i>n</i> photons can be distributed over <i>z</i> degrees of freedom in <i>q</i>(<i>n</i>,<i>z</i>) different ways calculated by the Bose-Einstein statistics. The entropy to be maximized is interpreted, as in the original definition of entropy by Boltzmann and Planck, as log<i>q</i>(<i>n</i>,<i>z</i>). This entropy expression reduces to log<i>B</i> and – <i>B</i> log<i>B</i> in the limits of <i>n</i> ≫ <i>z</i> > 1 and <i>n</i> ≪ <i>z</i>, respectively. When <i>n</i> is interpreted as an average n¯ over an ensemble, the above two criteria remain the same (with <i>n</i> replaced by n¯), and in addition for the <i>z</i> = 1 case the log<i>B</i> expression, used in ME spectral power estimation, is derived for n¯ ≫ <i>z</i> = 1.

© 1978 Optical Society of America

PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. R. Frieden, in Picture Processing and Digital Filtering, edited by T. S. Huang (Springer-Verlag, New York, 1975).
  2. S. J. Wernecke and L. R. D'Addario, "Maximum Entropy Image Reconstruction, " IEEE Trans. Computers C-26, 351 (1977); S. J. Wernecke, "Two-Dimensional Maximum Entropy Reconstruction of Radio Brightness, " Radio Sci. (to be published).
  3. J. P. Burg, project scientist, "Analytical Studies of Techniques for the Computation of High-Resolution Wavenumber Spectra," prepared by T. E. Barnard, Texas Instruments., Advanced Array Research Special Report No. 9, Contract No. F33657-68-C-0867, May 14, 1969.
  4. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois, Urbana, 1949).
  5. J. E. B. Ponsonby, "An Entropy Measure for Partially Polarized Radiation and its Application to Estimating Radio Sky Polarization Distributions from Incomplete 'Aperture Synthesis' Data by the Maximum Entropy Method," Mon. Not. Roy. Astron. Soc. 163, 369–380 (1973).
  6. B. R. Frieden and D. C. Wells, "Restoring with Maximum Entropy III: Poisson Sources and Backgrounds," J. Opt. Soc. Am. (to be published).
  7. H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).
  8. T. Ulrych, "Maximum Entropy Power Spectrum of Long Period Geomagnetic Reversals," Nature 235, 218–219 (1972).
  9. A. Zardecki, C. Delisle and J. Bures, Coherence and Quantum Optics, edited by L. Mandel and E. Wolf, Proc. 3rd Rochester Conf., June 1972 (Plenum, New York, 1973).
  10. R. W. Ditchburn, Light, 3rd ed. (Interscience, New York, 1976), p. 697; or G. R. Fowles, Introduction to Modern Optics (Holt Rinehart & Winston, New York, 1968), p. 211.
  11. Ludwig Boltzmann, Vorlesunger über Gastheorie (J. A. Barth, Leipzig-Part T, 1896; Part II, 1898), translated by Stephen G. Brush as Lectures in Gas Theory (University of California, Berkeley, 1969). See pp. 56ff, 74ff, and 371, where Boltzmann refers to the proportionality between entropy and the logarithm of the probability of a state.
  12. M. Planck, Theory of Heat Radiation, translated by M. Masius (Blakiston's, Philadelphia, 1914), Part III, Chap. I. The equation S = k logW, which is now referred to as Boltzmann's Principle, first appears in this work.
  13. See, for example, L. Mandel, in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1963), Vol. II; L. Mandel and E. Wolf, "Coherence Properties of Optical Fields, " Rev. Mod. Phys. 37, 231–287 (1965). The quantity n/z is often referred to in these references and elsewhere in the literature as the degeneracy parameter of the radiation. We deliberately avoid this usage to prevent a later semantic confusion.
  14. J. Klauder and E. C. G. Sudarshan, Fundamentals of Quantum Optics (Benjamin, New York, 1968), p. 138.
  15. See, for example, E. L. O'Neill, and T. Asakura, "Optical Image Formation in Terms of Entropy Transformations," J. Phys. Soc. Japan 16, 301–308 (1961).
  16. M. Scully, in Quantum Optics Course XLII, edited by R. J. Glauber (Academic, New York, 1969), p. 620ff. A succinct statement of the problem in terms of the coherent state representation. See also Ref. 14.
  17. J. Perina, in Quantum Optics, edited by S. M. Kay and A. Maitland (Academic, New York, 1970).
  18. L. Mandel, "Fluctuations of Photon Beams: the Distribution of the Photo-Electrons," Proc. Phys. Soc. 75, 233–243 (1959).
  19. W. M. Rosenblum, "Effect of Photon Distributions on Photographic Grain," J. Opt. Soc. Am. 58, 60–62 (1968).
  20. Calculated from data given in J. Kraus, Radio Astronomy (McGraw-Hill, New York, 1966).
  21. R. von Mises, Mathematical Theory of Probability and Statistics (Academic, New York, 1964), Chap. IV, Sec. 3.2.
  22. The negative binomial distribution was first introduced by F. Eggenberger and G. Pólya, Zeits. Ang. Math. Mech. 3, 276 (1923). According to Mandel, it was first applied to this problem by R. Fürth, Z. Phys. 48, 323 (1928); 50, 310 (1928).
  23. H. Gamo, "Thermodynamic Entropy of Partially Coherent Light Beams," J. Phys. Soc. Japan 19, 1955–1961 (1964).
  24. The exponential distribution is sometimes referred to as the geometrical or the "pure Bose" distribution in the literature.
  25. See, for example, the extensive bibliography in Refs. 1 and 2.
  26. F. T. S. Yu, Optics and Information Theory (Wiley, New York, 1976); M. Ross, Laser Receivers (Wiley, New York, 1966); T. E. Stern, "Some Quantum Effects in Information Channels," IRE Trans. Information Theory IT-6, 435–440 (1960); T. E. Stern, "Information Rates in Photon Channels and Photon Amplifiers," IRE Int. Convention Record, Part 4, 182–188 (1960); J. P. Gordon, "Quantum Effects in Communication Systems," Proc. IRE 50, 1898–1908 (1962); B. M. Oliver, "Thermal and Quantum Noise," Proc. IEEE 53, 436–454 (1965).
  27. A. Rényi, "On an Extremal Property of the Poisson Process," Ann. Inst. Stat. Math. 16, 129–133 (1964); J. A. McFadden, "The Entropy of a Point Process," J. Soc. Indust. Appl. Math. 13, 988–994 (1965).
  28. Shannon and Weaver, Ref. 4, p. 19.
  29. J. C. Dainty, Proceedings of the SPSE International Conference on Image Analysis and Evaluation, July 19-23, 1976, Toronto, Canada; "The Statistics of Speckle Patterns" in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1976), Vol. XIV.

1975 (1)

H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).

1973 (1)

J. E. B. Ponsonby, "An Entropy Measure for Partially Polarized Radiation and its Application to Estimating Radio Sky Polarization Distributions from Incomplete 'Aperture Synthesis' Data by the Maximum Entropy Method," Mon. Not. Roy. Astron. Soc. 163, 369–380 (1973).

1972 (1)

T. Ulrych, "Maximum Entropy Power Spectrum of Long Period Geomagnetic Reversals," Nature 235, 218–219 (1972).

1968 (1)

1964 (2)

H. Gamo, "Thermodynamic Entropy of Partially Coherent Light Beams," J. Phys. Soc. Japan 19, 1955–1961 (1964).

A. Rényi, "On an Extremal Property of the Poisson Process," Ann. Inst. Stat. Math. 16, 129–133 (1964); J. A. McFadden, "The Entropy of a Point Process," J. Soc. Indust. Appl. Math. 13, 988–994 (1965).

1961 (1)

See, for example, E. L. O'Neill, and T. Asakura, "Optical Image Formation in Terms of Entropy Transformations," J. Phys. Soc. Japan 16, 301–308 (1961).

1959 (1)

L. Mandel, "Fluctuations of Photon Beams: the Distribution of the Photo-Electrons," Proc. Phys. Soc. 75, 233–243 (1959).

Asakura, T.

See, for example, E. L. O'Neill, and T. Asakura, "Optical Image Formation in Terms of Entropy Transformations," J. Phys. Soc. Japan 16, 301–308 (1961).

Boltzmann, Ludwig

Ludwig Boltzmann, Vorlesunger über Gastheorie (J. A. Barth, Leipzig-Part T, 1896; Part II, 1898), translated by Stephen G. Brush as Lectures in Gas Theory (University of California, Berkeley, 1969). See pp. 56ff, 74ff, and 371, where Boltzmann refers to the proportionality between entropy and the logarithm of the probability of a state.

Bures, J.

A. Zardecki, C. Delisle and J. Bures, Coherence and Quantum Optics, edited by L. Mandel and E. Wolf, Proc. 3rd Rochester Conf., June 1972 (Plenum, New York, 1973).

Burg, J. P.

J. P. Burg, project scientist, "Analytical Studies of Techniques for the Computation of High-Resolution Wavenumber Spectra," prepared by T. E. Barnard, Texas Instruments., Advanced Array Research Special Report No. 9, Contract No. F33657-68-C-0867, May 14, 1969.

D'Addario, L. R.

S. J. Wernecke and L. R. D'Addario, "Maximum Entropy Image Reconstruction, " IEEE Trans. Computers C-26, 351 (1977); S. J. Wernecke, "Two-Dimensional Maximum Entropy Reconstruction of Radio Brightness, " Radio Sci. (to be published).

Dainty, J. C.

J. C. Dainty, Proceedings of the SPSE International Conference on Image Analysis and Evaluation, July 19-23, 1976, Toronto, Canada; "The Statistics of Speckle Patterns" in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1976), Vol. XIV.

Delisle, C.

A. Zardecki, C. Delisle and J. Bures, Coherence and Quantum Optics, edited by L. Mandel and E. Wolf, Proc. 3rd Rochester Conf., June 1972 (Plenum, New York, 1973).

Ditchburn, R. W.

R. W. Ditchburn, Light, 3rd ed. (Interscience, New York, 1976), p. 697; or G. R. Fowles, Introduction to Modern Optics (Holt Rinehart & Winston, New York, 1968), p. 211.

Fougere, P. F.

H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).

Frieden, B. R.

B. R. Frieden and D. C. Wells, "Restoring with Maximum Entropy III: Poisson Sources and Backgrounds," J. Opt. Soc. Am. (to be published).

B. R. Frieden, in Picture Processing and Digital Filtering, edited by T. S. Huang (Springer-Verlag, New York, 1975).

Gamo, H.

H. Gamo, "Thermodynamic Entropy of Partially Coherent Light Beams," J. Phys. Soc. Japan 19, 1955–1961 (1964).

Klauder, J.

J. Klauder and E. C. G. Sudarshan, Fundamentals of Quantum Optics (Benjamin, New York, 1968), p. 138.

Kraus, J.

Calculated from data given in J. Kraus, Radio Astronomy (McGraw-Hill, New York, 1966).

Mandel, L.

L. Mandel, "Fluctuations of Photon Beams: the Distribution of the Photo-Electrons," Proc. Phys. Soc. 75, 233–243 (1959).

See, for example, L. Mandel, in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1963), Vol. II; L. Mandel and E. Wolf, "Coherence Properties of Optical Fields, " Rev. Mod. Phys. 37, 231–287 (1965). The quantity n/z is often referred to in these references and elsewhere in the literature as the degeneracy parameter of the radiation. We deliberately avoid this usage to prevent a later semantic confusion.

O'Neill, E. L.

See, for example, E. L. O'Neill, and T. Asakura, "Optical Image Formation in Terms of Entropy Transformations," J. Phys. Soc. Japan 16, 301–308 (1961).

Perina, J.

J. Perina, in Quantum Optics, edited by S. M. Kay and A. Maitland (Academic, New York, 1970).

Planck, M.

M. Planck, Theory of Heat Radiation, translated by M. Masius (Blakiston's, Philadelphia, 1914), Part III, Chap. I. The equation S = k logW, which is now referred to as Boltzmann's Principle, first appears in this work.

Ponsonby, J. E. B.

J. E. B. Ponsonby, "An Entropy Measure for Partially Polarized Radiation and its Application to Estimating Radio Sky Polarization Distributions from Incomplete 'Aperture Synthesis' Data by the Maximum Entropy Method," Mon. Not. Roy. Astron. Soc. 163, 369–380 (1973).

Radoski, H. R.

H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).

Rényi, A.

A. Rényi, "On an Extremal Property of the Poisson Process," Ann. Inst. Stat. Math. 16, 129–133 (1964); J. A. McFadden, "The Entropy of a Point Process," J. Soc. Indust. Appl. Math. 13, 988–994 (1965).

Rosenblum, W. M.

Scully, M.

M. Scully, in Quantum Optics Course XLII, edited by R. J. Glauber (Academic, New York, 1969), p. 620ff. A succinct statement of the problem in terms of the coherent state representation. See also Ref. 14.

Shannon, C. E.

C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois, Urbana, 1949).

Sudarshan, E. C. G.

J. Klauder and E. C. G. Sudarshan, Fundamentals of Quantum Optics (Benjamin, New York, 1968), p. 138.

Ulrych, T.

T. Ulrych, "Maximum Entropy Power Spectrum of Long Period Geomagnetic Reversals," Nature 235, 218–219 (1972).

von Mises, R.

R. von Mises, Mathematical Theory of Probability and Statistics (Academic, New York, 1964), Chap. IV, Sec. 3.2.

Weaver, W.

C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois, Urbana, 1949).

Wells, D. C.

B. R. Frieden and D. C. Wells, "Restoring with Maximum Entropy III: Poisson Sources and Backgrounds," J. Opt. Soc. Am. (to be published).

Wernecke, S. J.

S. J. Wernecke and L. R. D'Addario, "Maximum Entropy Image Reconstruction, " IEEE Trans. Computers C-26, 351 (1977); S. J. Wernecke, "Two-Dimensional Maximum Entropy Reconstruction of Radio Brightness, " Radio Sci. (to be published).

Yu, F. T. S.

F. T. S. Yu, Optics and Information Theory (Wiley, New York, 1976); M. Ross, Laser Receivers (Wiley, New York, 1966); T. E. Stern, "Some Quantum Effects in Information Channels," IRE Trans. Information Theory IT-6, 435–440 (1960); T. E. Stern, "Information Rates in Photon Channels and Photon Amplifiers," IRE Int. Convention Record, Part 4, 182–188 (1960); J. P. Gordon, "Quantum Effects in Communication Systems," Proc. IRE 50, 1898–1908 (1962); B. M. Oliver, "Thermal and Quantum Noise," Proc. IEEE 53, 436–454 (1965).

Zardecki, A.

A. Zardecki, C. Delisle and J. Bures, Coherence and Quantum Optics, edited by L. Mandel and E. Wolf, Proc. 3rd Rochester Conf., June 1972 (Plenum, New York, 1973).

Zawalick, E. J.

H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).

Ann. Inst. Stat. Math. (1)

A. Rényi, "On an Extremal Property of the Poisson Process," Ann. Inst. Stat. Math. 16, 129–133 (1964); J. A. McFadden, "The Entropy of a Point Process," J. Soc. Indust. Appl. Math. 13, 988–994 (1965).

J. Geophys. Res. (1)

H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).

J. Opt. Soc. Am. (1)

J. Phys. Soc. Japan (2)

H. Gamo, "Thermodynamic Entropy of Partially Coherent Light Beams," J. Phys. Soc. Japan 19, 1955–1961 (1964).

See, for example, E. L. O'Neill, and T. Asakura, "Optical Image Formation in Terms of Entropy Transformations," J. Phys. Soc. Japan 16, 301–308 (1961).

Mon. Not. Roy. Astron. Soc. (1)

J. E. B. Ponsonby, "An Entropy Measure for Partially Polarized Radiation and its Application to Estimating Radio Sky Polarization Distributions from Incomplete 'Aperture Synthesis' Data by the Maximum Entropy Method," Mon. Not. Roy. Astron. Soc. 163, 369–380 (1973).

Nature (1)

T. Ulrych, "Maximum Entropy Power Spectrum of Long Period Geomagnetic Reversals," Nature 235, 218–219 (1972).

Proc. Phys. Soc. (1)

L. Mandel, "Fluctuations of Photon Beams: the Distribution of the Photo-Electrons," Proc. Phys. Soc. 75, 233–243 (1959).

Other (21)

Shannon and Weaver, Ref. 4, p. 19.

J. C. Dainty, Proceedings of the SPSE International Conference on Image Analysis and Evaluation, July 19-23, 1976, Toronto, Canada; "The Statistics of Speckle Patterns" in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1976), Vol. XIV.

The exponential distribution is sometimes referred to as the geometrical or the "pure Bose" distribution in the literature.

See, for example, the extensive bibliography in Refs. 1 and 2.

F. T. S. Yu, Optics and Information Theory (Wiley, New York, 1976); M. Ross, Laser Receivers (Wiley, New York, 1966); T. E. Stern, "Some Quantum Effects in Information Channels," IRE Trans. Information Theory IT-6, 435–440 (1960); T. E. Stern, "Information Rates in Photon Channels and Photon Amplifiers," IRE Int. Convention Record, Part 4, 182–188 (1960); J. P. Gordon, "Quantum Effects in Communication Systems," Proc. IRE 50, 1898–1908 (1962); B. M. Oliver, "Thermal and Quantum Noise," Proc. IEEE 53, 436–454 (1965).

Calculated from data given in J. Kraus, Radio Astronomy (McGraw-Hill, New York, 1966).

R. von Mises, Mathematical Theory of Probability and Statistics (Academic, New York, 1964), Chap. IV, Sec. 3.2.

The negative binomial distribution was first introduced by F. Eggenberger and G. Pólya, Zeits. Ang. Math. Mech. 3, 276 (1923). According to Mandel, it was first applied to this problem by R. Fürth, Z. Phys. 48, 323 (1928); 50, 310 (1928).

A. Zardecki, C. Delisle and J. Bures, Coherence and Quantum Optics, edited by L. Mandel and E. Wolf, Proc. 3rd Rochester Conf., June 1972 (Plenum, New York, 1973).

R. W. Ditchburn, Light, 3rd ed. (Interscience, New York, 1976), p. 697; or G. R. Fowles, Introduction to Modern Optics (Holt Rinehart & Winston, New York, 1968), p. 211.

Ludwig Boltzmann, Vorlesunger über Gastheorie (J. A. Barth, Leipzig-Part T, 1896; Part II, 1898), translated by Stephen G. Brush as Lectures in Gas Theory (University of California, Berkeley, 1969). See pp. 56ff, 74ff, and 371, where Boltzmann refers to the proportionality between entropy and the logarithm of the probability of a state.

M. Planck, Theory of Heat Radiation, translated by M. Masius (Blakiston's, Philadelphia, 1914), Part III, Chap. I. The equation S = k logW, which is now referred to as Boltzmann's Principle, first appears in this work.

See, for example, L. Mandel, in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1963), Vol. II; L. Mandel and E. Wolf, "Coherence Properties of Optical Fields, " Rev. Mod. Phys. 37, 231–287 (1965). The quantity n/z is often referred to in these references and elsewhere in the literature as the degeneracy parameter of the radiation. We deliberately avoid this usage to prevent a later semantic confusion.

J. Klauder and E. C. G. Sudarshan, Fundamentals of Quantum Optics (Benjamin, New York, 1968), p. 138.

B. R. Frieden and D. C. Wells, "Restoring with Maximum Entropy III: Poisson Sources and Backgrounds," J. Opt. Soc. Am. (to be published).

M. Scully, in Quantum Optics Course XLII, edited by R. J. Glauber (Academic, New York, 1969), p. 620ff. A succinct statement of the problem in terms of the coherent state representation. See also Ref. 14.

J. Perina, in Quantum Optics, edited by S. M. Kay and A. Maitland (Academic, New York, 1970).

B. R. Frieden, in Picture Processing and Digital Filtering, edited by T. S. Huang (Springer-Verlag, New York, 1975).

S. J. Wernecke and L. R. D'Addario, "Maximum Entropy Image Reconstruction, " IEEE Trans. Computers C-26, 351 (1977); S. J. Wernecke, "Two-Dimensional Maximum Entropy Reconstruction of Radio Brightness, " Radio Sci. (to be published).

J. P. Burg, project scientist, "Analytical Studies of Techniques for the Computation of High-Resolution Wavenumber Spectra," prepared by T. E. Barnard, Texas Instruments., Advanced Array Research Special Report No. 9, Contract No. F33657-68-C-0867, May 14, 1969.

C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois, Urbana, 1949).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.