Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Factorial coding of natural images: how effective are linear models in removing higher-order dependencies?

Not Accessible

Your library or personal account may give you access

Abstract

The performance of unsupervised learning models for natural images is evaluated quantitatively by means of information theory. We estimate the gain in statistical independence (the multi-information reduction) achieved with independent component analysis (ICA), principal component analysis (PCA), zero-phase whitening, and predictive coding. Predictive coding is translated into the transform coding framework, where it can be characterized by the constraint of a triangular filter matrix. A randomly sampled whitening basis and the Haar wavelet are included in the comparison as well. The comparison of all these methods is carried out for different patch sizes, ranging from 2×2to16×16  pixels. In spite of large differences in the shape of the basis functions, we find only small differences in the multi-information between all decorrelation transforms (5% or less) for all patch sizes. Among the second-order methods, PCA is optimal for small patch sizes and predictive coding performs best for large patch sizes. The extra gain achieved with ICA is always less than 2%. In conclusion, the edge filters found with ICA lead to only a surprisingly small improvement in terms of its actual objective.

© 2006 Optical Society of America

Full Article  |  PDF Article
More Like This
Probabilistic framework for the adaptation and comparison of image codes

Michael S. Lewicki and Bruno A. Olshausen
J. Opt. Soc. Am. A 16(7) 1587-1601 (1999)

Chromatic structure of natural scenes

Thomas Wachtler, Te-Won Lee, and Terrence J. Sejnowski
J. Opt. Soc. Am. A 18(1) 65-77 (2001)

Higher-order structure in natural scenes

Mitchell G. A. Thomson
J. Opt. Soc. Am. A 16(7) 1549-1553 (1999)

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (12)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (1)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (25)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All Rights Reserved