Inferring Salient Objects from Human Fixations

Wenguan Wang, Jianbing Shen, Xingping Dong, Ali Borji, Ruigang Yang

Research output: Contribution to journalArticlepeer-review

130 Scopus citations

Abstract

Previous research in visual saliency has been focused on two major types of models namely fixation prediction and salient object detection. The relationship between the two, however, has been less explored. In this work, we propose to employ the former model type to identify salient objects. We build a novel Attentive Saliency Network (ASNet)11.Available at: https://github.com/wenguanwang/ASNet. that learns to detect salient objects from fixations. The fixation map, derived at the upper network layers, mimics human visual attention mechanisms and captures a high-level understanding of the scene from a global view. Salient object detection is then viewed as fine-grained object-level saliency segmentation and is progressively optimized with the guidance of the fixation map in a top-down manner. ASNet is based on a hierarchy of convLSTMs that offers an efficient recurrent mechanism to sequentially refine the saliency features over multiple steps. Several loss functions, derived from existing saliency evaluation metrics, are incorporated to further boost the performance. Extensive experiments on several challenging datasets show that our ASNet outperforms existing methods and is capable of generating accurate segmentation maps with the help of the computed fixation prior. Our work offers a deeper insight into the mechanisms of attention and narrows the gap between salient object detection and fixation prediction.

Original languageEnglish
Article number8668551
Pages (from-to)1913-1927
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume42
Issue number8
DOIs
StatePublished - Aug 1 2020

Bibliographical note

Publisher Copyright:
© 1979-2012 IEEE.

Keywords

  • Image saliency
  • deep learning
  • fixation prediction
  • salient object detection

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Inferring Salient Objects from Human Fixations'. Together they form a unique fingerprint.

Cite this