OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A


  • Editor: Franco Gori
  • Vol. 28, Iss. 3 — Mar. 1, 2011
  • pp: 381–390

Multiframe image super-resolution adapted with local spatial information

Liangpei Zhang, Qiangqiang Yuan, Huanfeng Shen, and Pingxiang Li  »View Author Affiliations

JOSA A, Vol. 28, Issue 3, pp. 381-390 (2011)

View Full Text Article

Enhanced HTML    Acrobat PDF (1114 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Super-resolution image reconstruction, which has been a hot research topic in recent years, is a process to reconstruct high-resolution images from shifted, low-resolution, degraded observations. Among the available reconstruction frameworks, the maximum a posteriori (MAP) model is widely used. However, existing methods usually employ a fixed prior item and regularization parameter for the entire HR image, ignoring local spatially adaptive properties, and the large computation load caused by the solution of the large-scale ill-posed problem is another issue to be noted. In this paper, a block-based local spatially adaptive reconstruction algorithm is proposed. To reduce the large computation load and realize the local spatially adaptive process of the prior model and regularization parameter, first the target image is divided into several same-sized blocks and the structure tensor is used to analyze the local spatial properties of each block. Different property prior items and regularization parameters are then applied adaptively to different properties’ blocks. Experimental results show that the proposed method achieves better performance than methods with a fixed prior item and regularization parameter.

© 2011 Optical Society of America

OCIS Codes
(100.0100) Image processing : Image processing
(100.3020) Image processing : Image reconstruction-restoration
(100.3190) Image processing : Inverse problems
(100.6640) Image processing : Superresolution

ToC Category:
Image Processing

Original Manuscript: September 14, 2010
Revised Manuscript: December 15, 2010
Manuscript Accepted: December 15, 2010
Published: February 23, 2011

Virtual Issues
Vol. 6, Iss. 4 Virtual Journal for Biomedical Optics

Liangpei Zhang, Qiangqiang Yuan, Huanfeng Shen, and Pingxiang Li, "Multiframe image super-resolution adapted with local spatial information," J. Opt. Soc. Am. A 28, 381-390 (2011)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. R. Y. Tsai and T. S. Huang, “Multi-frame image restoration and registration,” Adv. Comput. Vision Image Process. 1, 317–339(1984).
  2. S. P. Kim, N. K. Bose, and H. M. Valenzuela, “Recursive reconstruction of high resolution image from noisy undersampled multiframes,” IEEE Trans. Acoust. Speech Signal Process. 38, 1013–1027 (1990). [CrossRef]
  3. S. P. Kim and W. Y. Su, “Recursive high-resolution reconstruction of blurred multiframe images,” IEEE Trans. Image Process. 2, 534–539 (1993). [CrossRef] [PubMed]
  4. N. K. Bose, H. C. Kim, and H. M. Valenzuela, “Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes,” Proceedings of IEEE Conference on Acoustics, Speech and Signal Processing (IEEE, 1993), pp. 269–272. [CrossRef]
  5. H. Ur and D. Gross, “Improved resolution from sub-pixel shifted pictures,” CVGIP: Graph. Models Image Process 54, 181–186(1992). [CrossRef]
  6. M. S. Alam, J. G. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49, 915–923 (2000). [CrossRef]
  7. M. Irani and S. Peleg, “Improving resolution by image registration,” CVGIP: Graph. Models Image Process. 53, 231–239 (1991). [CrossRef]
  8. M. Irani and S. Peleg, “Motion analysis for image enhancement resolution, occlusion, and transparency,” J. Visual Commun. Image Represent. 4, 324–335 (1993). [CrossRef]
  9. R. Gonsalves and F. Khaghani, “Super resolution based on low-resolution, warped images,” Proc. SPIE 4790, 10–20(2002).
  10. H. Stark and P. Oskoui, “High-resolution image recovery from image plane arrays, using convex projections,” J. Opt. Soc. Am. A 6, 1715–1726 (1989). [CrossRef] [PubMed]
  11. A. J. Patti, M. I. Sezan, and A. M. Tekalp, “Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1076(1997). [CrossRef] [PubMed]
  12. R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247–260(1998). [CrossRef]
  13. B. C. Tom and A. K. Katsaggelos, “Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images,” Proceedings of 1995 IEEE International Conference on Image Processing (IEEE, 1995), pp. 539–542. [CrossRef]
  14. M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Proc. 6, 1646–1658 (1997). [CrossRef]
  15. R. R. Schulz and R. L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996–1011 (1996). [CrossRef]
  16. R. C. Hardie, K. J. Barnard, and E. E. Armstrong, “Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,” IEEE Trans. Image Process. 6, 1621–1633 (1997). [CrossRef] [PubMed]
  17. H. Shen, L. Zhang, B. Huang, and P. Li, “A MAP approach for joint motion estimation, segmentation and super-resolution,” IEEE Trans. Image Process. 16, 479–490 (2007). [CrossRef] [PubMed]
  18. N. A. Woods, N. P. Galatsanos, and A. K. Katsaggelos, “Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images,” IEEE Trans. Image Process. 15, 201–213 (2006). [CrossRef] [PubMed]
  19. L. C. Pickup, D. P. Capel, S. J. Roberts, and A. Zisserman, “Bayesian methods for image super-resolution,” Comput. J. (UK) 52, 101–113 (2008). [CrossRef]
  20. M. Protter, M. Elad, H. Takeda, and P. Milanfar, “Generalizing the nonlocal-means to super-resolution reconstruction,” IEEE Trans. Image Process. 18, pp. 36–51 (2009). [CrossRef]
  21. Takeda, P. Milanfar, M. Protter, and M. Elad, “Super-resolution without explicit subpixel motion estimation,” IEEE Trans. Image Process. 18, pp. 1958–1975 (2009). [CrossRef] [PubMed]
  22. R. H. Chan, T. F. Chan, L. Shen, and Z. Shen, “Wavelet algorithms for high-resolution image reconstruction,” SIAM J. Sci. Comput. 24, 1408–1432 (2003). [CrossRef]
  23. M. K. Ng, C. K. Sze, and S. P. Yung, “Wavelet algorithms for deblurring models,” Int. J. Imaging Syst. Technol. 14, 113–121(2004). [CrossRef]
  24. N. Nguyen and P. Milanfar, “A wavelet-based interpolation restoration method for superresolution (wavelet superresolution),” Circuits, Systems, Signal Process. 19, 321–338 (2000). [CrossRef]
  25. H. Ji and C. Fermuller, “Robust wavelet-based super-resolution reconstruction: theory and algorithm,” IEEE Trans. Pattern Anal. Mach. Intell. 31, 649–660 (2009). [CrossRef] [PubMed]
  26. S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20, 21–36 (2003). [CrossRef]
  27. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Fast and robust multi-frame super resolution,” IEEE Trans. Image Process. 13, 1327–1344 (2004). [CrossRef] [PubMed]
  28. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, “Advances and challenges in super-resolution,” Int. J. Imaging Syst. Technol. 14, 47–57 (2004). [CrossRef]
  29. A. K. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video (Morgan and Claypool, 2007).
  30. S. Chaudhuri, Ed., Super-Resolution Imaging (Kluwer, 2001).
  31. P. Milanfar, Super Resolution imaging (CRC Press, 2010).
  32. L. C. Pickup, “Machine learning in multi-frame image super-resolution,” Ph.D. (University of Oxford, 2007).
  33. M. K. Ng, H. Shen, E. Y. Lam, and L. Zhang, “A total variation regularization based super-resolution reconstruction algorithm for digital video,” EURASIP J. Adv. Signal Process. Article ID 74585 (2007). [CrossRef]
  34. T. G. Stockman, “Image processing in the context of a visual model,” Proc. IEEE 60, 828–842 (1972). [CrossRef]
  35. T. Peli and J. S. Lim, “Adaptive filtering for image enhancement,” Opt. Eng. 21, 108–112 (1982)
  36. H. Su, L. Tang, D. Tretter, and J. Zhou, “A practical and adaptive framework for super-resolution,” Proceedings of IEEE International Conference on Image Processing 1 (IEEE, 2008), pp. 1236–1249.
  37. J. Lim, “Image restoration by short space spectral subtraction,” IEEE Trans. Acoust. Speech Signal Process. 28, 191–197(1980). [CrossRef]
  38. X. Zhang and K.-M. Lam, “Image magnification based on a blockwise adaptive Markov random field model,” Image Vision Comput. 26, 1277–1284 (2008). [CrossRef]
  39. R. Pan and S. J. Reeves., “Efficient Huber-Markov edge-preserving image restoration,” IEEE Trans. Image Process. 15, 3728–3735 (2006). [CrossRef] [PubMed]
  40. W. Forstner and E. Gulch, “A fast operator for detection and precise location of distinct points, corners and centres of circular features,” Proceedings ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data (Academic, 1987), pp. 281–305.
  41. J. Bigun and G. H. Granlund, “Optimal orientation detection of linear symmetry,” Proceedings First International Conference on Computer Vision (IEEE, 1987), pp. 433–438.
  42. T. Brox, J. Weickert, B. Burgeth, and P. Mrázek, “Nonlinear structure tensors,” Image Vis. Comput. 24, 41–55 (2006). [CrossRef]
  43. G. H. Golub and C. F. van Loan, Matrix Computation, 3rd ed. (Johns Hopkins University Press, 1996)
  44. H. Shen and L. Zhang, “A MAP-based algorithm for destriping and inpainting of remotely sensed images,” IEEE Trans. Geosci. Remote Sens. 47, 1492–1502 (2009) [CrossRef]
  45. P. Vandewalle, S. Susstrunk, and A. Vetterli, “A frequency domain approach to registration of aliased images with application to super-resolution,” EURASIP J. Appl. Signal Process. , Article ID 71459 (2006). [CrossRef]
  46. Z. Wang, A. C. Bovik, and H. R. Sheikh, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Proc. 13, 600–612 (2004). [CrossRef]
  47. http://users.soe.ucsc.edu/~milanfar/software/sr-datasets.html
  48. Z. Lin and H. Shum, “Fundamental limits of reconstruction-based superresolution algorithms under local translation,” IEEE Trans. Pattern Anal. Mach. Intell. 26, 83––97 (2004). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited