Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera

Mao Ye, Yang Shen, Chao Du, Zhigeng Pan, Ruigang Yang

Research output: Contribution to journalArticlepeer-review

32 Scopus citations


In this paper we present a novel real-time algorithm for simultaneous pose and shape estimation for articulated objects, such as human beings and animals. The key of our pose estimation component is to embed the articulated deformation model with exponential-maps-based parametrization into a Gaussian Mixture Model. Benefiting from this probabilistic measurement model, our algorithm requires no explicit point correspondences as opposed to most existing methods. Consequently, our approach is less sensitive to local minimum and handles fast and complex motions well. Moreover, our novel shape adaptation algorithm based on the same probabilistic model automatically captures the shape of the subjects during the dynamic pose estimation process. The personalized shape model in turn improves the tracking accuracy. Furthermore, we propose novel approaches to use either a mesh model or a sphere-set model as the template for both pose and shape estimation under this unified framework. Extensive evaluations on publicly available data sets demonstrate that our method outperforms most state-of-the-art pose estimation algorithms with large margin, especially in the case of challenging motions. Furthermore, our shape estimation method achieves comparable accuracy with state of the arts, yet requires neither statistical shape model nor extra calibration procedure. Our algorithm is not only accurate but also fast, we have implemented the entire processing pipeline on GPU. It can achieve up to 60 frames per second on a middle-range graphics card.

Original languageEnglish
Article number7457693
Pages (from-to)1517-1532
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number8
StatePublished - Aug 1 2016

Bibliographical note

Publisher Copyright:
© 1979-2012 IEEE.


  • Generative pose tracking
  • depth cues
  • motion
  • range data
  • real-time tracking
  • shape registration
  • surface fitting

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera'. Together they form a unique fingerprint.

Cite this