Joint Multi-Leaf Segmentation, Alignment, and Tracking for Fluorescence Plant Videos

Xi Yin, Xiaoming Liu, Jin Chen, David M. Kramer

Research output: Contribution to journalArticlepeer-review

33 Scopus citations


This paper proposes a novel framework for fluorescence plant video processing. The plant research community is interested in the leaf-level photosynthetic analysis within a plant. A prerequisite for such analysis is to segment all leaves, estimate their structures, and track them over time. We identify this as a joint multi-leaf segmentation, alignment, and tracking problem. First, leaf segmentation and alignment are applied on the last frame of a plant video to find a number of well-aligned leaf candidates. Second, leaf tracking is applied on the remaining frames with leaf candidate transformation from the previous frame. We form two optimization problems with shared terms in their objective functions for leaf alignment and tracking respectively. A quantitative evaluation framework is formulated to evaluate the performance of our algorithm with four metrics. Two models are learned to predict the alignment accuracy and detect tracking failure respectively in order to provide guidance for subsequent plant biology analysis. The limitation of our algorithm is also studied. Experimental results show the effectiveness, efficiency, and robustness of the proposed method.

Original languageEnglish
Pages (from-to)1411-1423
Number of pages13
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number6
StatePublished - Jun 1 2018

Bibliographical note

Publisher Copyright:
© 1979-2012 IEEE.


  • Arabidopsis
  • Chamfer matching
  • Plant phenotyping
  • alignment
  • leaf segmentation
  • multi-object
  • tracking

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Joint Multi-Leaf Segmentation, Alignment, and Tracking for Fluorescence Plant Videos'. Together they form a unique fingerprint.

Cite this