## Noise-resilient estimation of optical flow by use of overlapped basis functions

JOSA A, Vol. 16, Issue 3, pp. 493-507 (1999)

http://dx.doi.org/10.1364/JOSAA.16.000493

Acrobat PDF (2094 KB)

### Abstract

Conventional techniques for the computation of optical flow from image gradients are used to formulate the problem as a nonlinear optimization that comprises a gradient constraint term and a field smoothness factor. The results of these techniques are often erroneous, highly sensitive to noise and numerical precision, determined sparsely, and computationally expensive. We regularize the gradient constraint equation by modeling optical flow as a linear combination of a set of overlapped basis functions. We develop a theory for estimating model parameters robustly and reliably. We prove that the extended-least-squares solution proposed here is unbiased and robust to small perturbations in the gradient estimates and to mild deviations from the gradient constraint. Our solution is obtained with a numerically stable sparse matrix inversion, which gives a reliable flow-field estimate over the entire frame. To validate our claims, we perform a series of experiments on standard benchmark data sets at a range of noise levels. Overall, our algorithm outperforms by a wide margin the others considered in the comparison. We demonstrate the applicability of our algorithm to image mosaicking and to motion superresolution through experiments on noisy compressed sequences. We conclude that our flow-field model offers greater accuracy and robustness than conventional optical flow techniques in a variety of situations and permits real-time operation.

© 1999 Optical Society of America

**OCIS Codes**

(100.2960) Image processing : Image analysis

(100.6640) Image processing : Superresolution

(150.4620) Machine vision : Optical flow

**Citation**

Sridhar Srinivasan and Rama Chellappa, "Noise-resilient estimation of optical flow by use of overlapped basis functions," J. Opt. Soc. Am. A **16**, 493-507 (1999)

http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-16-3-493

Sort: Year | Journal | Reset

### References

- J. Barron, D. Fleet, and S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vis. 12, 43–77 (1994) (software and test sequences available at ftp.csd.uwo.ca/pub/vision).
- B. Horn and B. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981).
- H. Nagel, “On the estimation of optical flow: relations between different approaches and some new results,” Artif. Intel. 33, 299–324 (1987).
- S. Uras, F. Girosi, A. Verri, and V. Torre, “A computational approach to motion perception,” Biol. Cybern. 60, 79–87 (1989).
- B. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the International Joint Conference on Artificial Intelligence (IEEE Computer Society Press, Los Alamitos, Calif., 1981), pp. 674–679.
- P. Anandan, “Measuring visual motion from image sequences,” Ph.D. dissertation (University of Massachusetts, Amherst, Mass., 1987).
- A. Singh, “An Estimation-Theoretic Framework for Image-Flow Computation,” in Proceedings of the Third International Conference on Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif., 1990), pp. 168–177.
- E. P. Simoncelli, “Distributed representation and analysis of visual motion,” Ph.D. dissertation (Massachusetts Institute of Technology, Cambridge, Mass., 1993).
- A. Waxman, J. Wu, and F. Bergholm, “Convected activation profiles and the measurement of visual motion,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1988), pp. 717–723.
- D. Fleet and A. Jepson, “Computation of component image velocity from local phase information,” Int. J. Comput. Vis. 5, 77–104 (1990).
- J. Kearney, W. Thompson, and D. Boley, “Optical flow estimation: an error analysis of gradient-based methods with local optimization,” IEEE Trans. Pattern. Anal. Mach. Intell. 9, 229–244 (1987).
- J. Weber and J. Malik, “Robust computation of optical flow in a multiscale differential framework,” Int. J. Comput. Vis. 14, 67–81 (1995).
- H. Liu, T. Hong, M. Herman, and R. Chellappa, “A general motion model and spatiotemporal filters for computing optical-flow,” Int. J. Comput. Vis. 22, 141–172 (1997).
- S. Ju, M. Black, and A. Jepson, “Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency,” In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 307–314.
- N. Namazi and J. Lipp, “Nonuniform image motion estimation in reduced coefficient transformed-domains,” IEEE Trans. Image Process. 2, 236–246 (1993).
- C. Fan, N. Namazi, and P. Penafiel, “A new image motion estimation algorithm based on the EM technique,” IEEE Trans. Pattern. Anal. Mach. Intell. 18, 348–352 (1996).
- R. Szeliski and J. Coughlan, “Spline-based image registration,” Int. J. Comput. Vis. 22, 199–218 (1997).
- S. Rakshit and C. Anderson, “Computation of optical-flow using basis functions,” IEEE Trans. Image Process. 6, 1246–1254 (1997).
- C. Fennema and W. Thompson, “Velocity determination in scenes containing several moving objects,” Comput. Graph. Image Process. 9, 301–315 (1979).
- S. Srinivasan and R. Chellappa, “Robust modeling and estimation of optical flow with overlapped basis functions,” Tech. Rep. CAR-TR-845 (University of Maryland, College Park, Md., 1996) (software available at ftp.cfar.umd.edu/pub/shridhar/Software).
- L. Ng and V. Solo, “Errors-in-variables modeling in optical flow problems,” In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, New York, 1998), pp. 2373–2376.
- S. V. Huffel and J. Vandewalle, The Total Least Squares Problem—Computational Aspects and Analysis (Society for Industrial and Applied Mathematics, Philadelphia, Pa., 1991).
- O. Axelsson, Iterated Solution Methods (Cambridge U. Press, Cambridge, UK, 1994).
- W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C, 2nd ed. (Cambridge U. Press, Cambridge, UK, 1992).
- W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C, (Cambridge U. Press, Cambridge, UK, 1988).
- Since the publication Ref. 1 researchers1417 have used the Yosemite sequence as a benchmark and have shown signifi-cantly improved performance figures. However, their comparisons use a cloud-free rendering of the sequence or explicitly crop out the upper portion. In the former case, the sky area is uniformly black and velocity is zero. Figure of merit (32) is meaningless over a significant portion of the frame. Owing to the unavailability of implementations, and to other issues such as sparsity that disallow a direct comparison, we are unable to present an evaluation of these techniques vis a vis ours.
- S. Srinivasan and R. Chellappa, “Image stabilization and mosaicking using the overlapped basis optical flow field,” in Proceedings of the IEEE International Conference on Image Processing (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 356–359.
- A. M. Tekalp, Digital Video Processing (Prentice-Hall, Englewood Cliffs, N.J., 1995).
- M. Irani and S. Peleg, “Improving resolution by image registration,” Graph. Models Image Process. 53, 231–239 (1991).
- P. Moulin, R. Krishnamurthy, and J. Woods, “Multiscale modeling and estimation of motion fields for video coding,” IEEE Trans. Image Process. 6, 1606–1620 (1997).

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.