OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A


  • Vol. 20, Iss. 1 — Jan. 1, 2003
  • pp: 56–66

Local correlations, information redundancy, and sufficient pixel depth in natural images

Yury Petrov and L. Zhaoping  »View Author Affiliations

JOSA A, Vol. 20, Issue 1, pp. 56-66 (2003)

View Full Text Article

Enhanced HTML    Acrobat PDF (389 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



A mathematical framework that permits the factorization of a joint probability distribution into its localized components for a two-dimensional array of pixels is presented. The factorization was used to estimate the contribution to mutual information of two- (I2) and three-pixel (I3) luminance correlations for a large ensemble of natural images analyzed at various spatial scales and pixel depths b. It is shown that both I2 and I3 saturate at b6 bits per pixel. Three-pixel correlations are shown to produce only a marginal increase of information redundancy (4%) over two-pixel correlations (50%). Implications for neural representation in visual cortex are discussed.

© 2003 Optical Society of America

OCIS Codes
(070.6020) Fourier optics and signal processing : Continuous optical signal processing
(110.2960) Imaging systems : Image analysis
(200.3050) Optics in computing : Information processing
(330.4270) Vision, color, and visual optics : Vision system neurophysiology

Original Manuscript: May 7, 2002
Revised Manuscript: July 19, 2002
Manuscript Accepted: August 1, 2002
Published: January 1, 2003

Yury Petrov and L. Zhaoping, "Local correlations, information redundancy, and sufficient pixel depth in natural images," J. Opt. Soc. Am. A 20, 56-66 (2003)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J. 27, 379–423 (1948). [CrossRef]
  2. E. R. Kretzmer, “Statistics of television signals,” Bell Syst. Tech. J. 31, 751–763 (1952). [CrossRef]
  3. W. F. Schreiber, “The measurement of third-order probability distributions of television signals,” IRE Trans. Inf. Theory IT-2, 94–105 (1956). [CrossRef]
  4. J. R. Parks, “Prediction and entropy of half-tone pictures,” Behav. Sci. 10, 436–445 (1965). [CrossRef] [PubMed]
  5. N. S. Tzannes, R. V. Spencer, A. J. Kaplan, “On estimating the entropy of random fields,” Inf. Control. 16, 1–6 (1970). [CrossRef]
  6. F. Attneave, “Some information aspects of visual perception,” Psychol. Rev. 61, 183–193 (1954). [CrossRef] [PubMed]
  7. H. B. Barlow, “Possible principles underlying the transformation of sensory messages,” in Sensory Communication, W. A. Rosenblith, ed. (MIT, Cambridge, Mass., 1961).
  8. J. J. Atick, “Could information theory provide an ecological theory of sensory processing?” Network 3, 213–251 (1992). [CrossRef]
  9. H. B. Barlow, P. Foldiak, The Computing Neuron (Addison-Wesley, Reading, Mass., 1989).
  10. S. Watanabe, “Pattern recognition as a quest for minimum entropy,” Pattern Recognition 13, 381–387 (1981). [CrossRef]
  11. S. Watanabe, Pattern Recognition: Human and Mechanical (Wiley, New York, 1985).
  12. S. B. Laughlin, “A simple coding procedure enhances a neuron’s information capacity,” Z. Naturforsch. 36, 910–912 (1981).
  13. J. J. Atick, A. N. Redlich, “Towards a theory of early visual processing,” Neural Comput. 2, 308–320 (1990). [CrossRef]
  14. J. J. Atick, Z. Li, A. N. Redlich, “Understanding retinal color coding from first principles,” Neural Comput. 4, 559–572 (1992). [CrossRef]
  15. J. J. Atick, A. N. Redlich, “What does the retina know about the natural scenes?” Neural Comput. 4, 196–210 (1992). [CrossRef]
  16. Zh. Li, J. J. Atick, “Towards a theory of the striate cortex,” Neural Comput. 6, 127–146 (1994). [CrossRef]
  17. A. J. Bell, T. J. Sejnowski, “The ‘independent components’ of natural scenes are edge filters,” Vision Res. 37, 3327–3338 (1997). [CrossRef]
  18. D. J. Field, “Relations between the statistics of natural images and the response properties of cortical cells,” J. Opt. Soc. Am. A 4, 2379–2394 (1987). [CrossRef] [PubMed]
  19. J. G. Daugman, “Entropy reduction and decorrelation in visual coding by oriented neural receptive fields,” IEEE Trans. Biomed. Eng. 36, 107–114 (1989). [CrossRef] [PubMed]
  20. D. J. Field, “What is the goal of sensory coding?” Neural Comput. 6, 559–601 (1994). [CrossRef]
  21. C. Jutten, J. Herault, “Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture,” Signal Process. 24, 1–10 (1991). [CrossRef]
  22. A. Hyvärinen, E. Oja, “Independent component analysis: algorithms and applications,” Neural Networks 13, 411–430 (2000). [CrossRef]
  23. J. H. van Hateren, A. van der Schaaf, “Independent component filters of natural images compared with simple cells in primary visual cortex,” Proc. R. Soc. London Ser. B 265, 359–366 (1998). [CrossRef]
  24. A. Hyvärinen, P. O. Hoyer, “Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces,” Neural Comput. 12, 1705–1720 (2000). [CrossRef]
  25. B. A. Olshausen, D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature 381, 607–609 (1996). [CrossRef] [PubMed]
  26. B. A. Olshausen, D. J. Field, “Sparse coding with an overcomplete basis set: a strategy employed by V1?” Vision Res. 37, 3311–3325 (1997). [CrossRef]
  27. M. G. A. Thomson, “Higher-order structure in natural scenes,” J. Opt. Soc. Am. A 16, 1549–1553 (1999). [CrossRef]
  28. M. G. A. Thomson, “Beats, kurtosis and visual coding,” Network Comput. Neural Syst. 12, 271–287 (2001). [CrossRef]
  29. A. B. Lee, K. S. Pedersen, D. Mumford, “The complex statistics of high-contrast patches in natural images,” presented at the IEEE Workshop on Statistical and Computational Theories of Vision, Vancouver, Canada, 2001; http://www.cis.ohio-state.edu/szhu/SCTV2001.html
  30. N. G. Deriugin, “The power spectrum and the correlation function of the television signal,” Telecommunications 1, 1–12 (1957).
  31. D. J. Tolhurst, Y. Tadmor, T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiol. Opt. 12, 229–232 (1992). [CrossRef] [PubMed]
  32. D. L. Ruderman, “The statistics of natural images,” Network 5, 517–548 (1994). [CrossRef]
  33. D. Kersten, “Predictability and redundancy in natural images,” J. Opt. Soc. Am. A 4, 2395–2400 (1987). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


Fig. 1 Fig. 2 Fig. 3
Fig. 4

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited