OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A


  • Editor: Stephen A. Burns
  • Vol. 25, Iss. 5 — May. 1, 2008
  • pp: 1199–1214

Sparsity constrained regularization for multiframe image restoration

Premchandra M. Shankar and Mark A. Neifeld  »View Author Affiliations

JOSA A, Vol. 25, Issue 5, pp. 1199-1214 (2008)

View Full Text Article

Enhanced HTML    Acrobat PDF (1320 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



In this paper we present a new algorithm for restoring an object from multiple undersampled low-resolution (LR) images that are degraded by optical blur and additive white Gaussian noise. We formulate the multiframe superresolution problem as maximum a posteriori estimation. The prior knowledge that the object is sparse in some domain is incorporated in two ways: first we use the popular 1 norm as the regularization operator. Second, we model wavelet coefficients of natural objects using generalized Gaussian densities. The model parameters are learned from a set of training objects, and the regularization operator is derived from these parameters. We compare the results from our algorithms with an expectation-maximization (EM) algorithm for 1 norm minimization and also with the linear minimum-mean-squared error (LMMSE) estimator. Using only eight 4 × 4 pixel downsampled LR images the reconstruction errors of object estimates obtained from our algorithm are 5.5% smaller than by the EM method and 14.3% smaller than by the LMMSE method.

© 2008 Optical Society of America

OCIS Codes
(100.6640) Image processing : Superresolution
(110.4155) Imaging systems : Multiframe image processing

ToC Category:
Image Processing

Original Manuscript: November 29, 2007
Manuscript Accepted: February 21, 2008
Published: April 30, 2008

Premchandra M. Shankar and Mark A. Neifeld, "Sparsity constrained regularization for multiframe image restoration," J. Opt. Soc. Am. A 25, 1199-1214 (2008)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. S.Chaudhuri, ed., Super-Resolution Imaging (Kluwer, 2001).
  2. K. Aizawa, T. Komatsu, and T. Saito, “A scheme for acquiring very high resolution images using multiple cameras,” in IEEE International Conference on Acoustics, Speech, and Signal Processing (IEEE, 1992), Vol. 3, pp. 23-26.
  3. P. Shankar, W. Hassenplaugh, R. Morrison, R. Stack, and M. Neifeld, “Multiaperture imaging,” Appl. Opt. 45, 2871-2883 (2006). [CrossRef] [PubMed]
  4. A. Fruchter and R. Hook, “Drizzle: a method for the linear reconstruction of undersampled images,” Publ. Astron. Soc. Pac. 114, 144-152 (2002). [CrossRef]
  5. R. Tsai and T. Huang, “Multiframe image restoration and registration,” Advances in Computer Vision and Image Processing 1, 317-339 (1984).
  6. H. Andrews and B. Hunt, Digital Image Restoration (Prentice-Hall, 1977).
  7. M. Irani and S. Peleg, “Improving resolution by image registration,” Comput. Vis. Graph. Image Process. 53, 231-239 (1991).
  8. P. Vandewalle, L. Sbaiz, J. Vandewalle, and M. Vetterli, “How to take advantage of aliasing in bandlimited signals,” in Proceedings of the IEEE International Conference on Accoustics, Speech, and Signal Processing (IEEE, 2004), pp. 948-951.
  9. N. Nguyen, G. Golub, and P. Milanfar, “Blind restoration/superresolution with generalized cross-validation using Gauss-type quadrature rules,” in Proceedings of the 33rd Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, Calif., October 1999, pp. 1257-1261.
  10. R. Hardie, K. Bernard, and E. Armstrong, “Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,” IEEE Trans. Image Process. 6, 1621-1633 (1997). [CrossRef] [PubMed]
  11. R. Hardie, K. Barnard, J. Bognar, E. Armstrong, and E. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. (Bellingham) 37, 247-260 (1998). [CrossRef]
  12. S. Baker and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002). [CrossRef]
  13. P. Shankar and M. Neifeld, “Multiframe superresolution of binary images,” Appl. Opt. 46, 1211-1222 (2007). [CrossRef] [PubMed]
  14. E. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52, 489-509 (2006). [CrossRef]
  15. S. Chen, D. Donoho, and M. Saunders, “Atomic decomposition by basis pursuit,” SIAM J. Sci. Comput. (USA) 20, 33-61 (1998). [CrossRef]
  16. E. Candes and J. Romberg, “Signal recovery from random projections,” Proc. SPIE 5678, 76-86 (2005). [CrossRef]
  17. I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413-1457 (2004). [CrossRef]
  18. E. Cands, M. Wakin, and S. Boyd, “Enhancing sparsity by reweighted L1 minimization,” Technical Report, California Institute of Technology, http://www.acm.caltech.edu/emmanuel/papers/rwll-oct2007.pdf.
  19. M. Duarte, M. Wakin, and R. Baraniuk, “Fast reconstruction of piecewise smooth signals from random projections,” Online Proceedings of the Workshop on Signal Processing with Adaptative Sparse Structured Representations, SPARS 2005, http://spars05.irisa.fr/ACTES/TS5-3.pdf.
  20. D. Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inf. Theory 41, 613-627 (1995). [CrossRef]
  21. M. Figueiredo and R. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. Image Process. 12, 906916 (2003). [CrossRef]
  22. D. Wipf and B. Rao, “Sparse Bayesian learning for basis selection,” IEEE Trans. Signal Process. 52, 2153-2164 (2004). [CrossRef]
  23. C. Paige and M. Saunders, “LSQR: An algorithm for sparse linear equations and sparse least squares,” ACM Trans. Math. Softw. 8, 43-71 (1982). [CrossRef]
  24. C. Paige and M. Saunders, “LSQR: Sparse linear equations and least squares problems,” ACM Trans. Math. Softw. 8, 195-209 (1982). [CrossRef]
  25. S. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell. 11, 674-693 (1989). [CrossRef]
  26. M. A. Turk and A. P. Pentland, “Face recognition using eigenfaces,” in IEEE Proceedings on Computer Vision and Pattern Recognition (IEEE, 1991), pp. 586-591. [CrossRef]
  27. E. Candes, J. Romberg, and T. Tao, “Stable signal recovery from incomplete and inaccurate measurements,” Commun. Pure Appl. Math. 59, 1207-1223 (2006). [CrossRef]
  28. M. Figueiredo, R. Nowak, and S. Wright, “Gradient projections for sparse reconstruction: Application to compressed sensing and other inverse problems,” IEEE J. Sel. Top. Signal Process. 4, 586-597 (1987).
  29. H. Barrett and K. Myers, Foundations of Image Science (Wiley Series in Pure and Applied Optics, 2004).
  30. M. Kilmer and D. O'Leary, “Choosing regularization parameters in iterative methods for ill-posed problems,” SIAM J. Matrix Anal. Appl. 22, 1204-1221 (2001). [CrossRef]
  31. P. Hansen, Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion (SIAM, 1998). [CrossRef]
  32. N. Valdivia and E. Williams, “Krylov subspace iterative methods for boundary element method based near-field acoustic holography,” J. Acoust. Soc. Am. 117, 711-724 (2005). [CrossRef] [PubMed]
  33. M. Jiang, L. Xia, G. Shou, and M. Tang, “Combination of the LSQR method and a genetic algorithm for solving the electrocardiography inverse problem,” Phys. Med. Biol. 52, 1277-1294 (2007). [CrossRef] [PubMed]
  34. B. Rao and K. Kreutz-Delgado, “An affine scaling methodology for best basis selection,” IEEE Trans. Signal Process. 47, 187-200 (1999). [CrossRef]
  35. P. Hansen, “Analysis of discrete ill-posed problems by means of the L-curve,” SIAM Rev. 34, 561-580 (1992). [CrossRef]
  36. S. Mallat, “A compact multiresolution representation: the wavelet model,” presented at the IEEE Workshop Computer Society on Computer Vision, Miami, Florida, December 2-7, 1987.
  37. M. Belge, M. Kilmer, and E. Miller. “Wavelet domain image restoration with adaptive edge-preserving regularization,” IEEE Trans. Image Process. 9, 597-608 (2000). [CrossRef]
  38. B. Jeffs and M. Gunsay, “Restoration of blurred star field images by maximally sparse optimization,” IEEE Trans. Image Process. 2, 202-211 (1993). [CrossRef] [PubMed]
  39. G. Harikumar and Y. Bresler, “A new algorithm for computing sparse solutions to linear inverse problems,” in Proceedings of the 1996 IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 1996), Vol. 3, pp. 1131-1334.
  40. J. Mannos and D. Sakrison, “The effects of visual fidelity criterion on the encoding of image,” IRE Trans. Inf. Theory 20, 525-536 (1974). [CrossRef]
  41. USC-SIPI image database, Signal and Image Processing Institute at the University of Southern California, http://sipi.usc.edu/database.
  42. A. Goshtasby, “Image registration by local approximation methods,” Image Vis. Comput. 6, 255-261 (1988). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article

OSA is a member of CrossRef.

CrossCheck Deposited