Semi-Supervised video object segmentation with super-trajectories

Wenguan Wang, Jianbing Shen, Fatih Porikli, Ruigang Yang

Research output: Contribution to journalArticlepeer-review

121 Scopus citations

Abstract

We introduce a semi-supervised video segmentation approach based on an efficient video representation, called as 'super-trajectory'. A super-trajectory corresponds to a group of compact point trajectories that exhibit consistent motion patterns, similar appearances, and close spatiotemporal relationships. We generate the compact trajectories using a probabilistic model, which enables handling of occlusions and drifts effectively. To reliably group point trajectories, we adopt the density peaks based clustering algorithm that allows capturing rich spatiotemporal relations among trajectories in the clustering process. We incorporate two intuitive mechanisms for segmentation, called as reverse-tracking and object re-occurrence, for robustness and boosting the performance. Building on the proposed video representation, our segmentation method is discriminative enough to accurately propagate the initial annotations in the first frame onto the remaining frames. Our extensive experimental analyses on three challenging benchmarks demonstrate that, our method is capable of extracting the target objects from complex backgrounds, and even reidentifying them after prolonged occlusions, producing high-quality video object segments. The code and results are available at: https://github.com/wenguanwang/SupertrajectorySeg.

Original languageEnglish
Article number8325298
Pages (from-to)985-998
Number of pages14
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume41
Issue number4
DOIs
StatePublished - Apr 1 2019

Bibliographical note

Funding Information:
This work was supported in part by the Beijing Natural Science Foundation under Grant 4182056, the National Basic Research Program of China under grant 2013CB328805, the Australian Research Council’s Discovery Projects funding scheme under grant DP150104645, and the Fok Ying-Tong Education Foundation for Young Teachers. Specialized Fund for Joint Building Program of Beijing Municipal Education Commission.

Publisher Copyright:
© 1979-2012 IEEE.

Keywords

  • Video segmentation
  • density peaks clustering
  • trajectory extraction

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Semi-Supervised video object segmentation with super-trajectories'. Together they form a unique fingerprint.

Cite this