OSA's Digital Library

Optics Express

Optics Express

  • Editor: Andrew M. Weiner
  • Vol. 21, Iss. 17 — Aug. 26, 2013
  • pp: 19850–19866

Multi-frame super-resolution algorithm for complex motion patterns

A. V. Kanaev and C. W. Miller  »View Author Affiliations

Optics Express, Vol. 21, Issue 17, pp. 19850-19866 (2013)

View Full Text Article

Enhanced HTML    Acrobat PDF (4491 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Multi-frame super-resolution algorithms offer resolution enhancement for sequences of images with sampling limited resolution. However, classical approaches have been constrained by the accuracy of motion estimation while nonlocal approaches that use implicit motion estimation have attained only modest resolution improvement. In this paper, we propose a new multi-frame optical flow based super-resolution algorithm, which provides significant resolution enhancement for image sequences containing complex motion. The algorithm uses the standard camera image formation model and a variational super-resolution formulation with an anisotropic smoothness term adapting to local image structures. The key elements enabling super-resolution of complex motion patterns are the computation of two-way optical flow between the images and use of two corresponding uncertainty measures that approximate the optical flow interpolation error. Using the developed algorithm, we are able to demonstrate super-resolution of images for which optical flow estimation experiences near breakdown, due to the complexity of the motion patterns and the large magnitudes of the displacements. In comparison, we show that for these images some conventional super-resolution approaches fail, while others including nonlocal super-resolution technique produce distortions and provide lower (1-1.8dB) image quality enhancement compared to the proposed algorithm.

© 2013 OSA

OCIS Codes
(100.0100) Image processing : Image processing
(100.6640) Image processing : Superresolution
(150.4620) Machine vision : Optical flow

ToC Category:
Image Processing

Original Manuscript: April 16, 2013
Revised Manuscript: June 24, 2013
Manuscript Accepted: July 27, 2013
Published: August 16, 2013

Virtual Issues
Vol. 8, Iss. 9 Virtual Journal for Biomedical Optics

A. V. Kanaev and C. W. Miller, "Multi-frame super-resolution algorithm for complex motion patterns," Opt. Express 21, 19850-19866 (2013)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. S. C.  Park, M. K.  Park, M. G.  Kang, “Super-resolution image reconstruction: A technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003). [CrossRef]
  2. R. C.  Hardie, K. J.  Barnard, “Fast super-resolution using an adaptive Wiener filter with robustness to local motion,” Opt. Express 20(19), 21053–21073 (2012). [CrossRef] [PubMed]
  3. S. P.  Belekos, N. P.  Galatsanos, A. K.  Katsaggelos, “Maximum a posteriori video super-resolution using a new multichannel image prior,” IEEE Trans. Image Process. 19(6), 1451–1464 (2010). [CrossRef] [PubMed]
  4. M. K.  Park, M. G.  Kang, A. K.  Katsaggelos, “Regularized high-resolution image reconstruction considering inaccurate motion information,” Opt. Eng. 46(11), 117004 (2007). [CrossRef]
  5. O. A.  Omer, T.  Tanaka, “Multiframe image and video super-resolution algorithm with inaccurate motion registration errors rejection,” Proc. SPIE 6822, 682222, 682222-9 (2008). [CrossRef]
  6. O. A.  Omer, T.  Tanaka, “Region-based weighted-norm with adaptive regularization for resolution enhancement,” Digit. Signal Process. 21(4), 508–516 (2011). [CrossRef]
  7. D.  Mitzel, T.  Pock, T.  Schoenemann, D.  Cremers, “Video super resolution using duality based TV-L1 optical flow,” Lect. Notes Comput. Sci. 5748, 432–441 (2009). [CrossRef]
  8. M.  Unger, T.  Pock, M.  Werlberger, H.  Bischof, “A convex approach for variational super-resolution,” Lect. Notes Comput. Sci. 6376, 313–322 (2010). [CrossRef]
  9. C. Liu and D. Sun, “A Bayesian approach to adaptive video super resolution,” in Proceeding of Computer Vision and Pattern Recognition Conference (IEEE, 2011), pp. 209–216. [CrossRef]
  10. A. Danielyan, A. Foi, V. Katkovnik, and K. Egiazarian, “Image and video super-resolution via spatially adaptive block-matching filtering,” in Proceedings of Int. Workshop on Local and Non-Local Approx. in Image Processing, Lausanne, Switzerland, 23–24 August, 2008.
  11. M.  Protter, M.  Elad, H.  Takeda, P.  Milanfar, “Generalizing the Nonlocal-Means to Super-Resolution Reconstruction,” IEEE Trans. Image Process. 18(1), 36–51 (2009). [CrossRef] [PubMed]
  12. H.  Takeda, P.  Milanfar, M.  Protter, M.  Elad, “Super-resolution without explicit subpixel motion estimation,” IEEE Trans. Image Process. 18(9), 1958–1975 (2009). [CrossRef] [PubMed]
  13. D. Sun, S. Roth, and M. J. Black, “Secrets of optical flow estimation and their principles,” in Proceeding of Computer Vision and Pattern Recognition Conference (IEEE, 2010), pp. 2432–2439. [CrossRef]
  14. L.  Xu, J.  Jia, Y.  Matsushita, “Motion detail preserving optical flow estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1744–1757 (2012). [CrossRef] [PubMed]
  15. T.  Brox, J.  Malik, “Large displacement optical flow: descriptor matching in variational motion estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500–513 (2011). [CrossRef] [PubMed]
  16. H.  Zimmer, A.  Bruhn, J.  Weickert, L.  Valgaerts, A.  Salgado, B.  Rosenhahn, H.-P.  Seidel, “Complementary optic flow,” Lect. Notes Comput. Sci. 5681, 207–220 (2009). [CrossRef]
  17. H. Zimmer, A. Bruhn, and J. Weickert, “Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement,” in Proceeding of Eurographics, Llandudno, UK, 11–15 April, 2011. [CrossRef]
  18. T.  Brox, A.  Bruhn, N.  Papenberg, J.  Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004). [CrossRef]
  19. S.  Baker, D.  Scharstein, J. P.  Lewis, S.  Roth, M. J.  Black, R.  Szeliski, “A database and evaluation methodology for optical flow,” Int. J. Comput. Vis. 92(1), 1–31 (2011). [CrossRef]
  20. K.  Dabov, A.  Foi, V.  Katkovnik, K.  Egiazarian, “Image denoising by sparse 3D transform-domain collaborative filtering,” IEEE Trans. Image Process. 16(8), 2080–2095 (2007). [CrossRef] [PubMed]
  21. D. G.  Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004). [CrossRef]
  22. A. V.  Kanaev, “Confidence measures of optical flow estimation suitable for multi-frame super-resolution,” Proc. SPIE 8399, 839903, 839903-12 (2012). [CrossRef]
  23. A. Bruhn and J. Weickert, “A confidence measure for variational optic flow methods,” in Geometric Properties for Incomplete Data, (Springer-Verlag, 2006).
  24. K.  Dabov, A.  Foi, V.  Katkovnik, K.  Egiazarian, “Image restoration by sparse 3D transform-domain collaborative filtering,” Proc. SPIE 6812, 681207 (2008). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Supplementary Material

» Media 1: AVI (280 KB)     
» Media 2: AVI (115 KB)     
» Media 3: AVI (348 KB)     
» Media 4: AVI (388 KB)     
» Media 5: AVI (4378 KB)     

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited