OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A

| OPTICS, IMAGE SCIENCE, AND VISION

  • Vol. 16, Iss. 3 — Mar. 1, 1999
  • pp: 493–507

Noise-resilient estimation of optical flow by use of overlapped basis functions

Sridhar Srinivasan and Rama Chellappa  »View Author Affiliations


JOSA A, Vol. 16, Issue 3, pp. 493-507 (1999)
http://dx.doi.org/10.1364/JOSAA.16.000493


View Full Text Article

Enhanced HTML    Acrobat PDF (2094 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

Conventional techniques for the computation of optical flow from image gradients are used to formulate the problem as a nonlinear optimization that comprises a gradient constraint term and a field smoothness factor. The results of these techniques are often erroneous, highly sensitive to noise and numerical precision, determined sparsely, and computationally expensive. We regularize the gradient constraint equation by modeling optical flow as a linear combination of a set of overlapped basis functions. We develop a theory for estimating model parameters robustly and reliably. We prove that the extended-least-squares solution proposed here is unbiased and robust to small perturbations in the gradient estimates and to mild deviations from the gradient constraint. Our solution is obtained with a numerically stable sparse matrix inversion, which gives a reliable flow-field estimate over the entire frame. To validate our claims, we perform a series of experiments on standard benchmark data sets at a range of noise levels. Overall, our algorithm outperforms by a wide margin the others considered in the comparison. We demonstrate the applicability of our algorithm to image mosaicking and to motion superresolution through experiments on noisy compressed sequences. We conclude that our flow-field model offers greater accuracy and robustness than conventional optical flow techniques in a variety of situations and permits real-time operation.

© 1999 Optical Society of America

OCIS Codes
(100.2960) Image processing : Image analysis
(100.6640) Image processing : Superresolution
(150.4620) Machine vision : Optical flow

History
Original Manuscript: December 18, 1997
Revised Manuscript: November 3, 1998
Manuscript Accepted: September 17, 1998
Published: March 1, 1999

Citation
Sridhar Srinivasan and Rama Chellappa, "Noise-resilient estimation of optical flow by use of overlapped basis functions," J. Opt. Soc. Am. A 16, 493-507 (1999)
http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-16-3-493


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. J. Barron, D. Fleet, S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vis. 12, 43–77 (1994) (software and test sequences available at ftp.csd.uwo.ca/pub/vision). [CrossRef]
  2. B. Horn, B. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981). [CrossRef]
  3. H. Nagel, “On the estimation of optical flow: relations between different approaches and some new results,” Artif. Intel. 33, 299–324 (1987). [CrossRef]
  4. S. Uras, F. Girosi, A. Verri, V. Torre, “A computational approach to motion perception,” Biol. Cybern. 60, 79–87 (1989). [CrossRef]
  5. B. Lucas, T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the International Joint Conference on Artificial Intelligence (IEEE Computer Society Press, Los Alamitos, Calif., 1981), pp. 674–679.
  6. P. Anandan, “Measuring visual motion from image sequences,” Ph.D. dissertation (University of Massachusetts, Amherst, Mass., 1987).
  7. A. Singh, “An Estimation-Theoretic Framework for Image-Flow Computation,” in Proceedings of the Third International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1990), pp. 168–177.
  8. E. P. Simoncelli, “Distributed representation and analysis of visual motion,” Ph.D. dissertation (Massachusetts Institute of Technology, Cambridge, Mass., 1993).
  9. A. Waxman, J. Wu, F. Bergholm, “Convected activation profiles and the measurement of visual motion,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1988), pp. 717–723.
  10. D. Fleet, A. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vis. 5, 77–104 (1990). [CrossRef]
  11. J. Kearney, W. Thompson, D. Boley, “Optical flow estimation: an error analysis of gradient-based methods with local optimization,” IEEE Trans. Pattern. Anal. Mach. Intell. 9, 229–244 (1987). [CrossRef] [PubMed]
  12. J. Weber, J. Malik, “Robust computation of optical flow in a multiscale differential framework,” Int. J. Comput. Vis. 14, 67–81 (1995). [CrossRef]
  13. H. Liu, T. Hong, M. Herman, R. Chellappa, “A general motion model and spatiotemporal filters for computing optical-flow,” Int. J. Comput. Vis. 22, 141–172 (1997). [CrossRef]
  14. S. Ju, M. Black, A. Jepson, “Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency,” In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 307–314.
  15. N. Namazi, J. Lipp, “Nonuniform image motion estimation in reduced coefficient transformed-domains,” IEEE Trans. Image Process. 2, 236–246 (1993). [CrossRef]
  16. C. Fan, N. Namazi, P. Penafiel, “A new image motion estimation algorithm based on the EM technique,” IEEE Trans. Pattern. Anal. Mach. Intell. 18, 348–352 (1996). [CrossRef]
  17. R. Szeliski, J. Coughlan, “Spline-based image registration,” Int. J. Comput. Vis. 22, 199–218 (1997). [CrossRef]
  18. S. Rakshit, C. Anderson, “Computation of optical-flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997). [CrossRef]
  19. C. Fennema, W. Thompson, “Velocity determination in scenes containing several moving objects,” Comput. Graph. Image Process. 9, 301–315 (1979). [CrossRef]
  20. S. Srinivasan, R. Chellappa, “Robust modeling and estimation of optical flow with overlapped basis functions,” (University of Maryland, College Park, Md., 1996) (software available at ftp.cfar.umd.edu/pub/shridhar/Software).
  21. L. Ng, V. Solo, “Errors-in-variables modeling in optical flow problems,” In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1998), pp. 2373–2376.
  22. S. V. Huffel, J. Vandewalle, The Total Least Squares Problem—Computational Aspects and Analysis (Society for Industrial and Applied Mathematics, Philadelphia, Pa., 1991).
  23. O. Axelsson, Iterated Solution Methods (Cambridge U. Press, Cambridge, UK, 1994).
  24. W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery, Numerical Recipes in C, 2nd ed. (Cambridge U. Press, Cambridge, UK, 1992).
  25. W. H. Press, S. A. Teukolsky, W. T. Vetterling, B. P. Flannery, Numerical Recipes in C, (Cambridge U. Press, Cambridge, UK, 1988).
  26. Since the publication Ref. 1 researchers14,17 have used the Yosemite sequence as a benchmark and have shown signifi-cantly improved performance figures. However, their comparisons use a cloud-free rendering of the sequence or explicitly crop out the upper portion. In the former case, the sky area is uniformly black and velocity is zero. Figure of merit (32) is meaningless over a significant portion of the frame. Owing to the unavailability of implementations, and to other issues such as sparsity that disallow a direct comparison, we are unable to present an evaluation of these techniques vis a vis ours.
  27. S. Srinivasan, R. Chellappa, “Image stabilization and mosaicking using the overlapped basis optical flow field,” in Proceedings of the IEEE International Conference on Image Processing (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 356–359.
  28. A. M. Tekalp, Digital Video Processing (Prentice-Hall, Englewood Cliffs, N.J., 1995).
  29. M. Irani, S. Peleg, “Improving resolution by image registration,” Graph. Models Image Process. 53, 231–239 (1991). [CrossRef]
  30. P. Moulin, R. Krishnamurthy, J. Woods, “Multiscale modeling and estimation of motion fields for video coding,” IEEE Trans. Image Process. 6, 1606–1620 (1997). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited