CP-Detector: Using Configuration-related Performance Properties to Expose Performance Bugs

Haochen He, Zhouyang Jia, Shanshan Li, Erci Xu, Tingting Yu, Yue Yu, Ji Wang, Xiangke Liao

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

16 Scopus citations

Abstract

Performance bugs are often hard to detect due to their non fail-stop symptoms. Existing debugging techniques can only detect performance bugs with known patterns (e.g., inefficient loops). The key reason behind this incapability is the lack of a general test oracle. Here, we argue that the performance (e.g., throughput, latency, execution time) expectation of configuration can serve as a strong oracle candidate for performance bug detection. First, prior work shows that most performance bugs are related to configurations. Second, the configuration change reflects common expectation on performance changes. If the actual performance is contrary to the expectation, the related code snippet is likely to be problematic. In this paper, we first conducted a comprehensive study on 173 real-world configuration-related performance bugs (CPBugs) from 12 representative software systems. We then derived seven configuration-related performance properties, which can serve as the test oracle in performance testing. Guided by the study, we designed and evaluated an automated performance testing framework, CP-DETECTOR, for detecting real-world configuration-related performance bugs. CP-DETECTOR was evaluated on 12 open-source projects. The results showed that it detected 43 out of 61 existing bugs and reported 13 new bugs.

Original languageEnglish
Title of host publicationProceedings - 2020 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020
Pages623-634
Number of pages12
ISBN (Electronic)9781450367684
DOIs
StatePublished - Sep 2020
Event35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020 - Virtual, Melbourne, Australia
Duration: Sep 22 2020Sep 25 2020

Publication series

NameProceedings - 2020 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020

Conference

Conference35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020
Country/TerritoryAustralia
CityVirtual, Melbourne
Period9/22/209/25/20

Bibliographical note

Publisher Copyright:
© 2020 ACM.

Keywords

  • Performance bug detection
  • Performance property
  • Software configuration

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'CP-Detector: Using Configuration-related Performance Properties to Expose Performance Bugs'. Together they form a unique fingerprint.

Cite this