OSA's Digital Library

Journal of the Optical Society of America

Journal of the Optical Society of America

  • Vol. 67, Iss. 12 — Dec. 1, 1977
  • pp: 1656–1665

Maximum entropy image restoration. I. The entropy expression

Ryoichi Kikuchi and B. H. Soffer  »View Author Affiliations

JOSA, Vol. 67, Issue 12, pp. 1656-1665 (1977)

View Full Text Article

Acrobat PDF (1445 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



The two entropy expressions, logB and – B logB (where B is the local brightness of the object or its spatial spectral power) used in maximum entropy (ME) image restoration, are derived as limiting cases of a general entropy formula. The brightness B is represented by the n photons emitted from a small unit area of the object and imaged in the receiver. These n photons can be distributed over z degrees of freedom in q(n,z) different ways calculated by the Bose-Einstein statistics. The entropy to be maximized is interpreted, as in the original definition of entropy by Boltzmann and Planck, as logq(n,z). This entropy expression reduces to logB and – B logB in the limits of nz > 1 and nz, respectively. When n is interpreted as an average n¯ over an ensemble, the above two criteria remain the same (with n replaced by n¯), and in addition for the z = 1 case the logB expression, used in ME spectral power estimation, is derived for n¯ ≫ z = 1.

© 1978 Optical Society of America

Ryoichi Kikuchi and B. H. Soffer, "Maximum entropy image restoration. I. The entropy expression," J. Opt. Soc. Am. 67, 1656-1665 (1977)

Sort:  Author  |  Year  |  Journal  |  Reset


  1. B. R. Frieden, in Picture Processing and Digital Filtering, edited by T. S. Huang (Springer-Verlag, New York, 1975).
  2. S. J. Wernecke and L. R. D'Addario, "Maximum Entropy Image Reconstruction, " IEEE Trans. Computers C-26, 351 (1977); S. J. Wernecke, "Two-Dimensional Maximum Entropy Reconstruction of Radio Brightness, " Radio Sci. (to be published).
  3. J. P. Burg, project scientist, "Analytical Studies of Techniques for the Computation of High-Resolution Wavenumber Spectra," prepared by T. E. Barnard, Texas Instruments., Advanced Array Research Special Report No. 9, Contract No. F33657-68-C-0867, May 14, 1969.
  4. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois, Urbana, 1949).
  5. J. E. B. Ponsonby, "An Entropy Measure for Partially Polarized Radiation and its Application to Estimating Radio Sky Polarization Distributions from Incomplete 'Aperture Synthesis' Data by the Maximum Entropy Method," Mon. Not. Roy. Astron. Soc. 163, 369–380 (1973).
  6. B. R. Frieden and D. C. Wells, "Restoring with Maximum Entropy III: Poisson Sources and Backgrounds," J. Opt. Soc. Am. (to be published).
  7. H. R. Radoski, P. F. Fougere and E. J. Zawalick, "A Comparison of Power Spectral Estimates and Applications of the Maximum Entropy Method, " J. Geophys. Res. 80, 619–625 (1975).
  8. T. Ulrych, "Maximum Entropy Power Spectrum of Long Period Geomagnetic Reversals," Nature 235, 218–219 (1972).
  9. A. Zardecki, C. Delisle and J. Bures, Coherence and Quantum Optics, edited by L. Mandel and E. Wolf, Proc. 3rd Rochester Conf., June 1972 (Plenum, New York, 1973).
  10. R. W. Ditchburn, Light, 3rd ed. (Interscience, New York, 1976), p. 697; or G. R. Fowles, Introduction to Modern Optics (Holt Rinehart & Winston, New York, 1968), p. 211.
  11. Ludwig Boltzmann, Vorlesunger über Gastheorie (J. A. Barth, Leipzig-Part T, 1896; Part II, 1898), translated by Stephen G. Brush as Lectures in Gas Theory (University of California, Berkeley, 1969). See pp. 56ff, 74ff, and 371, where Boltzmann refers to the proportionality between entropy and the logarithm of the probability of a state.
  12. M. Planck, Theory of Heat Radiation, translated by M. Masius (Blakiston's, Philadelphia, 1914), Part III, Chap. I. The equation S = k logW, which is now referred to as Boltzmann's Principle, first appears in this work.
  13. See, for example, L. Mandel, in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1963), Vol. II; L. Mandel and E. Wolf, "Coherence Properties of Optical Fields, " Rev. Mod. Phys. 37, 231–287 (1965). The quantity n/z is often referred to in these references and elsewhere in the literature as the degeneracy parameter of the radiation. We deliberately avoid this usage to prevent a later semantic confusion.
  14. J. Klauder and E. C. G. Sudarshan, Fundamentals of Quantum Optics (Benjamin, New York, 1968), p. 138.
  15. See, for example, E. L. O'Neill, and T. Asakura, "Optical Image Formation in Terms of Entropy Transformations," J. Phys. Soc. Japan 16, 301–308 (1961).
  16. M. Scully, in Quantum Optics Course XLII, edited by R. J. Glauber (Academic, New York, 1969), p. 620ff. A succinct statement of the problem in terms of the coherent state representation. See also Ref. 14.
  17. J. Perina, in Quantum Optics, edited by S. M. Kay and A. Maitland (Academic, New York, 1970).
  18. L. Mandel, "Fluctuations of Photon Beams: the Distribution of the Photo-Electrons," Proc. Phys. Soc. 75, 233–243 (1959).
  19. W. M. Rosenblum, "Effect of Photon Distributions on Photographic Grain," J. Opt. Soc. Am. 58, 60–62 (1968).
  20. Calculated from data given in J. Kraus, Radio Astronomy (McGraw-Hill, New York, 1966).
  21. R. von Mises, Mathematical Theory of Probability and Statistics (Academic, New York, 1964), Chap. IV, Sec. 3.2.
  22. The negative binomial distribution was first introduced by F. Eggenberger and G. Pólya, Zeits. Ang. Math. Mech. 3, 276 (1923). According to Mandel, it was first applied to this problem by R. Fürth, Z. Phys. 48, 323 (1928); 50, 310 (1928).
  23. H. Gamo, "Thermodynamic Entropy of Partially Coherent Light Beams," J. Phys. Soc. Japan 19, 1955–1961 (1964).
  24. The exponential distribution is sometimes referred to as the geometrical or the "pure Bose" distribution in the literature.
  25. See, for example, the extensive bibliography in Refs. 1 and 2.
  26. F. T. S. Yu, Optics and Information Theory (Wiley, New York, 1976); M. Ross, Laser Receivers (Wiley, New York, 1966); T. E. Stern, "Some Quantum Effects in Information Channels," IRE Trans. Information Theory IT-6, 435–440 (1960); T. E. Stern, "Information Rates in Photon Channels and Photon Amplifiers," IRE Int. Convention Record, Part 4, 182–188 (1960); J. P. Gordon, "Quantum Effects in Communication Systems," Proc. IRE 50, 1898–1908 (1962); B. M. Oliver, "Thermal and Quantum Noise," Proc. IEEE 53, 436–454 (1965).
  27. A. Rényi, "On an Extremal Property of the Poisson Process," Ann. Inst. Stat. Math. 16, 129–133 (1964); J. A. McFadden, "The Entropy of a Point Process," J. Soc. Indust. Appl. Math. 13, 988–994 (1965).
  28. Shannon and Weaver, Ref. 4, p. 19.
  29. J. C. Dainty, Proceedings of the SPSE International Conference on Image Analysis and Evaluation, July 19-23, 1976, Toronto, Canada; "The Statistics of Speckle Patterns" in Progress in Optics, edited by E. Wolf (North-Holland, Amsterdam, 1976), Vol. XIV.

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited