OSA's Digital Library

Applied Optics

Applied Optics


  • Editor: Joseph N. Mait
  • Vol. 51, Iss. 23 — Aug. 10, 2012
  • pp: 5686–5697

Skeleton body pose tracking from efficient three-dimensional motion estimation and volumetric reconstruction

Zheng Zhang and Hock Soon, Seah  »View Author Affiliations

Applied Optics, Vol. 51, Issue 23, pp. 5686-5697 (2012)

View Full Text Article

Enhanced HTML    Acrobat PDF (1371 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



We address the problem of body pose tracking in a scenario of multiple camera setup with the aim of recovering body motion robustly and accurately. The tracking is performed on three-dimensional (3D) space using 3D data, including colored volume and 3D optical flow, which are reconstructed at each time step. We introduce strategies to compute multiple camera-based 3D optical flow and have attained efficient and robust 3D motion estimation. Body pose estimation starts with a prediction using 3D optical flow and then is changed to a lower-dimensional global optimization problem. Our method utilizes a voxel subject-specific body model, exploits multiple 3D image cues, and incorporates physical constraints into a stochastic particle-based search initialized from the deterministic prediction and stochastic sampling. It leads to a robust 3D pose tracker. Experiments on publicly available sequences show the robustness and accuracy of our approach.

© 2012 Optical Society of America

OCIS Codes
(110.4153) Imaging systems : Motion estimation and optical flow
(110.4155) Imaging systems : Multiframe image processing
(100.4999) Image processing : Pattern recognition, target tracking

ToC Category:
Imaging Systems

Original Manuscript: June 1, 2012
Revised Manuscript: July 13, 2012
Manuscript Accepted: July 15, 2012
Published: August 7, 2012

Zheng Zhang and Hock Soon, "Skeleton body pose tracking from efficient three-dimensional motion estimation and volumetric reconstruction," Appl. Opt. 51, 5686-5697 (2012)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. J. Deutscher, A. Blake, and I. Reid, “Articulated body motion capture by annealed particle filtering,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2000), Vol. 2, pp. 126–133.
  2. C. Theobalt, J. Carranza, M. A. Magnor, and H. P. Seidel, “Enhancing silhouette-based human motion capture with 3D motion fields,” in IEEE Pacific Conference on Computer Graphics and Applications (IEEE, 2003), pp. 185–193.
  3. I. Mikic, M. Trivedi, E. Hunter, and P. Cosman, “Human body model acquisition and tracking using voxel data,” Int. J. Comput. Vis. 53, 199–223 (2003). [CrossRef]
  4. R. Kehl, M. Bray, and L. V. Gool, “Full body tracking from multiple views using stochastic sampling,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2005), Vol. 2, pp. 129–136.
  5. L. Mundermann, S. Corazza, and T. P. Andriacchi, “Accurately measuring human movement using articulated ICP with soft-joint constraints and a repository of articulated models,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–6.
  6. J. Gall, C. Stoll, E. de Aguiar, C. Theobalt, B. Rosenhahn, and H.-P. Seidel, “Motion capture using joint skeleton tracking and surface estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (2009), pp. 1746–1753.
  7. J. MacCormick and M. Isard, “Partitioned sampling, articulated objects, and interface-quality hand tracking,” in European Conference on Computer Vision, Vol. 1843 of Lecture Notes in Computer Science (Springer, 2000), pp. 3–19.
  8. C. Sminchisescu, A. Kanaujia, Z. Li, and D. Metaxas, “Discriminative density propagation for 3D human motion estimation,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2005), Vol. 1, pp. 390–397.
  9. M. Bray, E. Koller-Meier, and L. V. Gool, “Smart particle filtering for high-dimensional tracking,” Comput. Vis. Image Understanding 106, 116–129 (2007). [CrossRef]
  10. Z. Zhang, H. S. Seah, C. K. Quah, and J. Sun, “A markerless motion capture system with automatic subject-specific body model acquisition and robust pose tracking from 3D data,” in IEEE International Conference on Image Processing (IEEE, 2011), pp. 525–528.
  11. A. Laurentini, “The visual hull: a new tool for contour-based image understanding,” in 7th Scandinavian Conference on Image Analysis (Springer, 1991), pp. 993–1002.
  12. R. Szeliski, “Rapid octree construction from image sequences,” CVGIP Image Understanding 58, 23–32 (1993). [CrossRef]
  13. G. K. Cheung, T. Kanade, J.-Y. Bouguet, and M. Holler, “A real time system for robust 3D voxel reconstruction of human motions,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2000), Vol. 2, pp. 714–720.
  14. W. Matusik, C. Buehler, R. Raskar, S. Gortler, and L. McMillan, “Image-based visual hulls,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM, 2000), pp. 369–374.
  15. W. Matusik, C. Buehler, and L. McMillan, “Polyhedral visual hulls for real-time rendering,” in Proceedings of the 12th Eurographics Workshop on Rendering Techniques (Springer, 2001), pp. 115–126.
  16. S. Lazebnik, E. Boyer, and J. Ponce, “On computing exact visual hull of solids bounded by smooth surfaces,” in IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2001), Vol. 1, pp. I156–I161.
  17. A. Ladikos, S. Benhimane, and N. Navab, “Efficient visual hull computation for real-time 3D reconstruction using CUDA,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (IEEE, 2008), pp. 1–8.
  18. S. Vedula, S. Baker, P. Rander, R. Collins, and T. Kanade, “Three-dimensional scene flow,” IEEE Trans. Pattern Anal. Machine Intell. 27, 475–480 (2005). [CrossRef]
  19. M. Gong and Y.-H. Yang, “Disparity flow estimation using orthogonal reliability-based dynamic programming,” in IEEE International Conference on Pattern Recognition (IEEE, 2006), pp. 70–73.
  20. R. L. Carceroni and K. N. Kutulakos, “Multi-view scene capture by surfel sampling: from video streams to non-rigid 3D motion, shape and reflectance,” Int. J. Comput. Vis. 49, 175–214 (2002). [CrossRef]
  21. T. Can, A. O. Karali, and T. Aytac, “Detection and tracking of sea-surface targets in infrared and visual band videos using the bag-of-features technique with scale-invariant feature transform,” Appl. Opt. 50, 6302–6312 (2011). [CrossRef]
  22. C. Theobalt, J. Carranza, M. Magnor, and H.-P. Seidel, “Combining 3D flow fields with silhouette-based human motion capture for immersive video,” Graph. Models 22, 540–547 (2004).
  23. J.-Y. Bouguet, “Pyramidal implementation of the Lucas-Kanade feature tracker,” Microprocessor Research Labs Tech. Rep. (2000).
  24. K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Q. Appl. Math. 2, 164–168(1994).
  25. Z. Zhang, H. S. Seah, and J. Sun, “A hybrid particle swarm optimization with cooperative method for multi-object tracking,” presented at the IEEE Congress on Evolutionary Computing, Brisbane, Australia, 10–15June2012.
  26. NVIDIA, NVIDIA CUDA programming guide, http://www.nvidia.com/object/cudahomenew.html .
  27. J. Marzat, Y. Dumortier, and A. Ducrot, “Real-time dense and accurate parallel optical flow using CUDA,” 7th International Conference WSCG (2009).
  28. L. Sigal and M. J. Black, “HumanEva: synchronized video and motion capture dataset for evaluation of articulated human motion,” Tech. Rep. (Brown University, 2006).
  29. T. Horprasert, D. Harwood, and L. S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” in IEEE International Conference on Computer Vision (IEEE, 1999), pp. 1–19.

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited