OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A


  • Editor: Franco Gori
  • Vol. 30, Iss. 5 — May. 1, 2013
  • pp: 831–853

Information optimal compressive sensing: static measurement design

Amit Ashok, Liang-Chih Huang, and Mark A. Neifeld  »View Author Affiliations

JOSA A, Vol. 30, Issue 5, pp. 831-853 (2013)

View Full Text Article

Enhanced HTML    Acrobat PDF (3169 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



The compressive sensing paradigm exploits the inherent sparsity/compressibility of signals to reduce the number of measurements required for reliable reconstruction/recovery. In many applications additional prior information beyond signal sparsity, such as structure in sparsity, is available, and current efforts are mainly limited to exploiting that information exclusively in the signal reconstruction problem. In this work, we describe an information-theoretic framework that incorporates the additional prior information as well as appropriate measurement constraints in the design of compressive measurements. Using a Gaussian binomial mixture prior we design and analyze the performance of optimized projections relative to random projections under two specific design constraints and different operating measurement signal-to-noise ratio (SNR) regimes. We find that the information-optimized designs yield significant, in some cases nearly an order of magnitude, improvements in the reconstruction performance with respect to the random projections. These improvements are especially notable in the low measurement SNR regime where the energy-efficient design of optimized projections is most advantageous. In such cases, the optimized projection design departs significantly from random projections in terms of their incoherence with the representation basis. In fact, we find that the maximizing incoherence of projections with the representation basis is not necessarily optimal in the presence of additional prior information and finite measurement noise/error. We also apply the information-optimized projections to the compressive image formation problem for natural scenes, and the improved visual quality of reconstructed images with respect to random projections and other compressive measurement design affirms the overall effectiveness of the information-theoretic design framework.

© 2013 Optical Society of America

OCIS Codes
(100.7410) Image processing : Wavelets
(110.1758) Imaging systems : Computational imaging
(110.3055) Imaging systems : Information theoretical analysis

ToC Category:
Imaging Systems

Original Manuscript: August 22, 2012
Revised Manuscript: December 22, 2012
Manuscript Accepted: February 14, 2013
Published: April 10, 2013

Amit Ashok, Liang-Chih Huang, and Mark A. Neifeld, "Information optimal compressive sensing: static measurement design," J. Opt. Soc. Am. A 30, 831-853 (2013)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. E. T. Whittaker, “On the functions which are represented by the expansions of the interpolation theory,” Proc. R. Soc. Edinburgh 35, 181–194 (1915).
  2. H. Nyquist, “Certain topics in telegraph transmission theory,” Trans. Am. Inst. Electr. Eng. 47, 617–644 (1928). [CrossRef]
  3. C. E. Shannon, “Communication in the presence of noise,” Proc. IRE 37, 10–21 (1949). [CrossRef]
  4. W. Chen and W. Pratt, “Scene adaptive coder,” IEEE Trans. Commun. 32, 225–232 (1984). [CrossRef]
  5. G. Wallace, “The JPEG still picture compression standard,” Commun. ACM 34, 30–44 (1991). [CrossRef]
  6. D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice (Kluwer, 2001).
  7. Y. Tsaig and D. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006). [CrossRef]
  8. D. Donoho and Y. Tsaig, “Extensions of compressed sensing,” Signal Process. 86, 549–571 (2006). [CrossRef]
  9. E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5674, 76–86 (2005). [CrossRef]
  10. E. Candes and T. Tal, “Near-optimal signal recovery from random projections: universal encoding strategies?,” IEEE Trans. Inf. Theory 52, 5406–5425 (2006). [CrossRef]
  11. F. Krahmer and R. Ward, “New and improved Johnson–Lindenstrauss embeddings via the restricted isometry property,” SIAM J. Math. Anal.43, 1269–1281 (2010).
  12. J. Romberg, “Imaging via compressing sampling,” IEEE Signal Process. Mag. 25, (2) 14–20 (2008). [CrossRef]
  13. M. A. Neifeld and P. Shankar, “Feature-specific imaging,” Appl. Opt. 42, 3379–3389 (2003). [CrossRef]
  14. H. Pal and M. A. Neifeld, “Multispectral principal component imaging,” Opt. Express 11, 2118–2125 (2003). [CrossRef]
  15. M. A. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt. 46, 5293–5303 (2007). [CrossRef]
  16. A. Ashok and M. A. Neifeld, “Compressive imaging: hybrid measurement basis design,” J. Opt. Soc. Am. A 28, 1041–1050 (2011). [CrossRef]
  17. P. B. Fellgett, “On the ultimate sensitivity and practical performance of radiation detectors,” J. Opt. Soc. Am. 39, 970–976 (1949). [CrossRef]
  18. E. Candes and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory 51, 4203–4215 (2005). [CrossRef]
  19. H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Trans. Inf. Theory 54, 2210–2219 (2008). [CrossRef]
  20. L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process. 57, 3488–3497 (2009). [CrossRef]
  21. Y. C. Eldar and M. Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Inf. Theory 55, 5302–5316 (2009). [CrossRef]
  22. M. Chen, J. Silva, J. Paisley, C. Wang, D. Dunson, and L. Carin, “Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds,” IEEE Trans. Signal Process. 58, 6140–6155 (2010). [CrossRef]
  23. R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory 56, 1982–2001 (2010). [CrossRef]
  24. M. Elad, “Optimized projections for compressed sensing,” IEEE Trans. Signal Process. 55, 5695–5702 (2007). [CrossRef]
  25. G. Puy, P. Vandergheynst, and Y. Wiaux, “On variable density compressive sampling,” IEEE Signal Process. Lett. 18, 595–598 (2011). [CrossRef]
  26. H. S. Chang, Y. Weiss, and W. T. Freeman, “Informative sensing,” submitted to IEEE Trans. Inf. Theory, preprint available at arXiv http://arxiv.org/abs/0901.4275 (2009).
  27. W. R. Carson, M. Chen, M. R. D. Rodrigues, R. Calderbank, and L. Carin, “Communications-inspired projection design with application to compressive sensing,” preprint available at arXiv:1206.1973 http://arxiv.org/abs/1206.1973 (2012).
  28. R. M. Willett, R. F. Marcia, and J. M. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng. 50, 072601 (2012). [CrossRef]
  29. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Trans. Image Process. 12, 1338–1351 (2003). [CrossRef]
  30. E. Candes and J. Romberg, “Sparsity and incoherence in compressive sampling,” Inverse Problems 23, 969–985 (2007). [CrossRef]
  31. S. Ji and L. Carin, “Bayesian compressive sensing and projection optimization,” in Proceedings of the 24th International Conference on Machine Learning (ICML) (ACM, 2007), pp. 377–384.
  32. M. W. Seeger and H. Nickisch, “Compressed sensing and Bayesian experimental design,” in Proceedings of the 25th International Conference on Machine Learning (ICML) (ACM, 2008), pp. 912–919.
  33. M. A. Neifeld, A. Ashok, and P. K. Baheti, “Task-specific information for imaging system analysis,” J. Opt. Soc. Am. A 24, B25–B41 (2007). [CrossRef]
  34. A. Ashok, P. K. Baheti, and M. A. Neifeld, “Compressive imaging system design using task-specific information,” Appl. Opt. 47, 4457–4471 (2008). [CrossRef]
  35. D. Stowell and M. D. Plumbley, “Fast multidimensional entropy estimation by k-d partitioning,” IEEE Signal Process. Lett. 16, 537–540 (2009). [CrossRef]
  36. M. A. Tanner, Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, 3rd ed. (Springer-Verlag, 1996).
  37. D. J. Tolhurst, Y. Tadmor, and T. Chao, “Amplitude spectra of natural images,” Ophthalmic Physiolog. Opt. 12, 229–232 (1992). [CrossRef]
  38. D. L. Ruderman, “Origins of scaling in natural images,” Vis. Res. 37, 3385–3398 (1997). [CrossRef]
  39. E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193–1216 (2001). [CrossRef]
  40. USC-SIPI Image Database. Available at http://sipi.usc.edu/database .
  41. H. Lee, A. Battle, R. Raina, and A. Y. Ng, “Efficient sparse coding algorithms,” in Proceedings of Advances in Neural Information Processing Systems (NIPS), Vol. 19 (MIT, 2007), pp. 801–808.
  42. Y. Yang, J. Wright, T. S. Huang, and M. Yi, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. 19, 2861–2873 (2010). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited