OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A

| OPTICS, IMAGE SCIENCE, AND VISION

  • Vol. 16, Iss. 3 — Mar. 1, 1999
  • pp: 481–492

Data-driven multichannel superresolution with application to video sequences

H. Shekarforoush and R. Chellappa  »View Author Affiliations


JOSA A, Vol. 16, Issue 3, pp. 481-492 (1999)
http://dx.doi.org/10.1364/JOSAA.16.000481


View Full Text Article

Enhanced HTML    Acrobat PDF (1040 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

A method is proposed for superresolving multichannel data with application to video sequences. Based on a generalization of Papoulis’ sampling theorem, nonuniform samples of multiple channels are merged to generate high-resolution data. To overcome sampling ill posedness in the presence of noise, image frames are projected from standard orthonormal bases onto optimal Riesz bases defined by channel point spread functions (PSF’s). The method is therefore designed to perform under practical conditions of noise and other degradations. Unlike existing methods, where empirical models such as Gaussian, sinc, etc., are commonly used for characterizing channel PSF’s, the PSF’s are assumed unknown and possibly different and hence are blindly estimated from the observed data. The estimated PSF’s are then used to construct biorthogonal projection filters for the superresolution algorithm. This approach gives rise to a closed-form solution leading to a high-speed algorithm. The method has been tested and verified on PREDATOR video sequences (PREDATOR data are airborne video sequences of the Defense Advanced Research Projects Agency obtained by unmanned aircrafts).

© 1999 Optical Society of America

OCIS Codes
(100.2960) Image processing : Image analysis
(100.2980) Image processing : Image enhancement
(100.3020) Image processing : Image reconstruction-restoration
(100.3190) Image processing : Inverse problems
(100.6640) Image processing : Superresolution

History
Original Manuscript: June 17, 1998
Revised Manuscript: September 23, 1998
Manuscript Accepted: October 26, 1998
Published: March 1, 1999

Citation
H. Shekarforoush and R. Chellappa, "Data-driven multichannel superresolution with application to video sequences," J. Opt. Soc. Am. A 16, 481-492 (1999)
http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-16-3-481


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. Y. Yacoob, L. Davis, R. Chellappa, Q. Zheng, “Visual surveillance and monitoring of human and vehicular activity,” in Proceedings of the Image Understanding Workshop (Omnipress, Madison, Wis., 1997), Vol. I, pp. 19–23.
  2. S. Srinivasan, R. Chellapa, “An integrated approach to image stabilization, mosaicking and super-resolution,” in Proceedings of the Image Understanding Workshop (Omnipress, Madison, Wis., 1997), Vol. I, pp. 247–253.
  3. J. L. Harris, “Diffraction and resolving power,” J. Opt. Soc. Am. 54, 931–936 (1964). [CrossRef]
  4. D. Slepian, H. O. Pollak, “Prolate spheroidal wave functions, Fourier analysis and uncertainty. I,” Bell Syst. Tech. J. 40, 43–63 (1961). [CrossRef]
  5. D. Slepian, “Prolate spheroidal wave functions, Fourier analysis and uncertainty. IV: Extension to many dimensions; generalized prolate spheroidal wave functions,” Bell Syst. Tech. J. 43, 3009–3057 (1964). [CrossRef]
  6. D. Slepian, “Prolate spheroidal wave functions, Fourier analysis and uncertainty. V: The discrete case,” Bell Syst. Tech. J. 57, 1371–1430 (1978). [CrossRef]
  7. R. W. Gerchberg, “Super-resolution through error energy reduction,” Opt. Acta 21, 709–720 (1974). [CrossRef]
  8. A. Papoulis, “A new algorithm in spectral analysis and bandlimited signal extrapolation,” IEEE Trans. Circuits Syst. CS-22, 735–742 (1975). [CrossRef]
  9. B. R. Hunt, P. Sementilli, “Description of Poisson imagery super resolution algorithm,” in Astronomical Data Analysis Software and Systems, C. Biemesderfer, D. Worral, J. Barnes, eds. (Astronomical Society of the Pacific, San Francisco, Calif., 1992), pp. 196–199.
  10. R. Tsai, T. Huang, “Multiframe image restoration and registration,” in Advances in Computer Vision and Image Processing (JAI, Greenwich, Conn., 1984), Vol. 1.
  11. S. P. Kim, N. K. Bose, H. M. Valenzuela, “Recursive reconstruction of high resolution image from noisy under-sampled multiframes,” IEEE Trans. Acoust., Speech, Signal Process. 38, 1013–1027 (1990). [CrossRef]
  12. S. P. Kim, W. Y. Su, “Recursive high-resolution reconstruction of blurred multiframe images,” IEEE Trans. Image Process. 2, 534–539 (1993). [CrossRef] [PubMed]
  13. M. Irani, S. Peleg, “Improving resolution by image registration,” Graphic. Models Image Process. 53, 231–239 (1991). [CrossRef]
  14. H. Ur, D. Gross, “Improved resolution from subpixel shifted pictures,” Comput. Vis. Graph. Image Process. 54, 181–186 (1991).
  15. K. Aizawa, T. Komatsu, T. Saito, “A scheme for acquiring very high resolution images using multiple cameras,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1992), Vol. 3, pp. 289–292.
  16. A. M. Tekalp, M. K. Ozkan, M. I. Sezan, “High resolution image reconstruction from low resolution image sequences and space varying image restoration,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1992), pp. 169–172.
  17. T. Numnonda, M. Andrews, R. Kakarala, “High resolution image reconstruction by simulated annealing,” in Proceedings of Image & Vision Computing (Industrial Research, Ltd., Aukland, N.Z., 1993), 213–220.
  18. M. Berthod, H. Shekarforoush, M. Werman, J. Zerubia, “Reconstruction of high resolution 3D visual information,” in Proceedings of Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1994), pp. 654–657.
  19. H. Shekarforoush, M. Berthod, J. Zerubia, “3D super-resolution using generalized sampling expansion,” in Proceedings of the International Conference on Image Processing (IEEE Computer Society Press, Los Alamitos, Calif., 1995), pp. 300–303.
  20. H. Shekarforoush, M. Berthod, J. Zerubia, M. Werman, “Sub-pixel Bayesian estimation of albedo and height,” Int. J. Comput. Vis. 19, 289–300 (1996). [CrossRef]
  21. M. C. Chiang, T. E. Boult, “Local blur estimation and super-resolution,” in Proceedings of Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 821–826.
  22. A. J. Patti, M. I. Sezan, A. M. Tekalp, “High resolution standards conversion of low resolution video,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1995), pp. 2197–2200.
  23. A. J. Patti, M. I. Sezan, A. M. Tekalp, “Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time,” IEEE Trans. Image Process. 6, 1064–1076 (1997). [CrossRef] [PubMed]
  24. M. Elad, A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process. 6, 1646–1658 (1997). [CrossRef] [PubMed]
  25. C. A. Bernstein, E. V. Patrick, “Exact deconvolution or multiple operators: an overview plus performance characterizations for imaging sensors,” in Proceedings of Multidimensional Signal Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1990), pp. 723–734.
  26. A. Lorette, H. Shekarforoush, J. Zerubia, “Super-resolution with adaptive regularization,” in Proceedings of the International Conference on Image Processing (IEEE Computer Society Press, Los Alamitos, Calif., 1997), pp. 169–172.
  27. A. Papoulis, “Generalized sampling expansion,” IEEE Trans. Circuits Syst. CS-24, 652–654 (1977). [CrossRef]
  28. R. E. A. C. Paley, N. Wiener, Fourier Transforms in the Complex Domain, Vol. 19 of Colloquium Publications (American Mathematical Society, New York, 1934).
  29. H. Shekarforoush, “Super-resolution in computer vision,” Ph.D. thesis (University of Nice, Institut National de la Recherche en Informatique et en Automatique, Sophia Antipolis, France, 1996).
  30. E. De-Castro, C. Morandi, “Registration of translated and rotated images using finite Fourier transforms,” IEEE Trans. Pattern. Anal. Mach. Intell. PAMI-9, 700–703 (1987). [CrossRef]
  31. H. Shekarforoush, M. Berthod, J. Zerubia, “Subpixel image registration by estimating the polyphase decomposition of cross power spectrum,” in Proceedings of Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 532–537.
  32. G. B. Giannakis, R. W. Heath, “Blurs and perfect image restoration,” in Proceedings of the International Conference on Image Processing (IEEE Computer Society Press, Los Alamitos, Calif., 1997), Vol. 1, pp. 717–720.
  33. R. W. Heath, S. D. Halford, G. B. Giannakis, “Adaptive blind channel identification of FIR channels for Viterbi decoding,” in Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers (IEEE Computer Society Press, Los Alamitos, Calif., 1996), Vol. 1, pp. 320–324.
  34. M. S. Scivier, M. A. Fiddy, “Phase ambiguities and the zeros of multidimensional band-limited functions,” J. Opt. Soc. Am. A 2, 693–697 (1985). [CrossRef]
  35. A. Papoulis, Signal Analysis (McGraw-Hill, New York, 1977).
  36. R. R. Coifman, M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. Inf. Theory 38, 713–718 (1992). [CrossRef]
  37. S. G. Mallat, Z. Zhang, “Matching pursuits with time-frequency dictionaries” IEEE Trans. Signal Process. 41, 3397–2415 (1993). [CrossRef]
  38. S. Chen, D. L. Donoho, “Basis pursuit,” in Proceedings of the 28th Asilomar Conference on Signals, Systems, and Computers (IEEE Computer Society Press, Los Alamitos, Calif., 1994), pp. 41–44.
  39. H. Krim, S. G. Mallat, D. L. Donoho, A. Willsky, “Best basis algorithm for signal enhancement,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, Piscataway, N.J., 1995), pp. 1561–1564.
  40. R. M. Young, An Introduction to Nonharmonic Fourier Series (Academic, New York, 1980).
  41. F. Riesz, B. Sz.-Nagy, Functional Analysis (Dover, New York, 1990).
  42. A. Gray, T. M. Mathews, A Treatise on Bessel Functions and Their Applications to Physics (Dover, New York, 1966).

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited