OSA's Digital Library

Applied Optics

Applied Optics


  • Editor: Joseph N. Mait
  • Vol. 53, Iss. 13 — May. 1, 2014
  • pp: C1–C13

Computationally efficient video restoration for Nyquist sampled imaging sensors combining an affine-motion-based temporal Kalman filter and adaptive Wiener filter

Michael Rucci, Russell C. Hardie, and Kenneth J. Barnard  »View Author Affiliations

Applied Optics, Vol. 53, Issue 13, pp. C1-C13 (2014)

View Full Text Article

Enhanced HTML    Acrobat PDF (1413 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



In this paper, we present a computationally efficient video restoration algorithm to address both blur and noise for a Nyquist sampled imaging system. The proposed method utilizes a temporal Kalman filter followed by a correlation-model based spatial adaptive Wiener filter (AWF). The Kalman filter employs an affine background motion model and novel process-noise variance estimate. We also propose and demonstrate a new multidelay temporal Kalman filter designed to more robustly treat local motion. The AWF is a spatial operation that performs deconvolution and adapts to the spatially varying residual noise left in the Kalman filter stage. In image areas where the temporal Kalman filter is able to provide significant noise reduction, the AWF can be aggressive in its deconvolution. In other areas, where less noise reduction is achieved with the Kalman filter, the AWF balances the deconvolution with spatial noise reduction. In this way, the Kalman filter and AWF work together effectively, but without the computational burden of full joint spatiotemporal processing. We also propose a novel hybrid system that combines a temporal Kalman filter and BM3D processing. To illustrate the efficacy of the proposed methods, we test the algorithms on both simulated imagery and video collected with a visible camera.

© 2014 Optical Society of America

OCIS Codes
(100.1830) Image processing : Deconvolution
(100.2000) Image processing : Digital image processing
(100.3020) Image processing : Image reconstruction-restoration

Original Manuscript: December 9, 2013
Revised Manuscript: February 23, 2014
Manuscript Accepted: March 3, 2014
Published: April 11, 2014

Michael Rucci, Russell C. Hardie, and Kenneth J. Barnard, "Computationally efficient video restoration for Nyquist sampled imaging sensors combining an affine-motion-based temporal Kalman filter and adaptive Wiener filter," Appl. Opt. 53, C1-C13 (2014)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. G. Holst and T. Lomheim, CMOS/CCD Sensors and Camera Systems, 2nd ed. (SPIE, 2011).
  2. J. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts & Company, 1968).
  3. R. D. Fiete, “Image quality and λ FN/p for remote sensing systems,” Opt. Eng. 38, 1229–1240 (1999). [CrossRef]
  4. S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003). [CrossRef]
  5. A. Buades, B. Coll, and J.-M. Morel, “Nonlocal image and movie denoising,” Int. J. Comput. Vis. 76, 123–139 (2008). [CrossRef]
  6. Y. Han and R. Chen, “Efficient video denoising based on dynamic nonlocal means,” Image Vis. Comput. 30, 78–85 (2012). [CrossRef]
  7. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Trans. Image Process. 16, 2080–2095 (2007). [CrossRef]
  8. K. Dabov, A. Foi, and K. Egiazarian, “Video denoising by sparse 3d transform-domain collaborative filtering,” in Proceedings of the 15th European Signal Processing Conference (2007), Vol. 1, p. 145–149.
  9. M. Maggioni, G. Boracchi, A. Foi, and K. Egiazarian, “Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotemporal transforms,” IEEE Trans. Image Process. 21, 3952–3966 (2012). [CrossRef]
  10. J. C. Brailean, R. P. Kleihorst, S. Efstratiadis, A. K. Katsaggelos, and R. L. Lagendijk, “Noise reduction filters for dynamic image sequences: a review,” Proc. IEEE 83, 1272–1292 (1995). [CrossRef]
  11. J. Biemond, J. Rieske, and J. Gerbrands, “A fast Kalman filter for images degraded by both blur and noise,” IEEE Trans. Acoust., Speech, Signal Process. 31, 1248–1256 (1983). [CrossRef]
  12. J. Kim and J. W. Woods, “Spatiotemporal adaptive 3-d Kalman filter for video,” IEEE Trans. Image Process. 6, 414–424 (1997). [CrossRef]
  13. R. Dugad and N. Ahuja, “Video denoising by combining Kalman and Wiener estimates,” in Proceedings of the International Conference on Image Processing (ICIP), Kobe (1999), pp. 152–156.
  14. R. D. Turney, A. M. Reza, and J. G. Delva, “FPGA implementation of adaptive temporal Kalman filter for real time video filtering,” in Proceedings of the 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing (IEEE, 1999), Vol. 4, pp. 2231–2234.
  15. F. Jin, P. Fieguth, and L. Winger, “Wavelet video denoising with regularized multiresolution motion estimation,” EURASIP J. Adv. Signal Process. 2006, 072705 (2006). [CrossRef]
  16. L. Jovanov, A. Pizurica, S. Schulte, P. Schelkens, A. Munteanu, E. Kerre, and W. Philips, “Combined wavelet-domain and motion-compensated video denoising based on video codec motion estimation methods,” IEEE Trans. Circuits Syst. Video Technol. 19, 417–421 (2009). [CrossRef]
  17. R. Hardie, “A fast image super-resolution algorithm using an adaptive Wiener filter,” IEEE Trans. Image Process. 16, 2953–2964 (2007). [CrossRef]
  18. R. C. Hardie, K. J. Barnard, and R. Ordonez, “Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging,” Opt. Express 19, 26208–26231 (2011). [CrossRef]
  19. R. C. Hardie and K. J. Barnard, “Fast super-resolution using an adaptive Wiener filter with robustness to local motion,” Opt. Express 20, 21053–21073 (2012). [CrossRef]
  20. G. Bishop and G. Welch, “An introduction to the Kalman filter,” Proc. SIGGRAPH, Course 8 (2001).
  21. M. Makitalo and A. Foi, “Poisson-Gaussian denoising using the exact unbiased inverse of the generalized anscombe transformation,” in Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, 2012), pp. 1081–1084.
  22. S. Farsiu, M. Elad, and P. Milanfar, “Video-to-video dynamic super-resolution for grayscale and color sequences,” EURASIP J. Adv. Signal Process. 2006, 061859 (2006). [CrossRef]
  23. R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37, 247–260 (1998). [CrossRef]
  24. M. A. Makar and H. K. Raghunandan, “Wiener and Kalman filters for denoising video signals,” EE378 Class Project, Spring (2008).
  25. S. E. Reichenbach and S. K. Park, “Small convolution kernels for high-fidelity image restoration,” IEEE Trans. Signal Process. 39, 2263–2274 (1991). [CrossRef]
  26. S. G. Johnson and M. Frigo, “A modified split-radix fft with fewer arithmetic operations,” IEEE Trans. Signal Process. 55, 111–119 (2007). [CrossRef]
  27. M. Makitalo and A. Foi, “Optimal inversion of the generalized anscombe transformation for Poisson-Gaussian noise,” IEEE Trans. Image Process. 22, 91–103 (2013). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Supplementary Material

» Media 1: MOV (4241 KB)     
» Media 2: MOV (4385 KB)     
» Media 3: MOV (3808 KB)     
» Media 4: MOV (3770 KB)     
» Media 5: MOV (2651 KB)     
» Media 6: MOV (3906 KB)     
» Media 7: MOV (4527 KB)     

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited