On failure classification: The impact of "getting it wrong"

Davide Falessi, Bill Kidwell, Jane Huffman Hayes, Forrest Shull

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Bug classification is a well-established practice which supports important activities such as enhancing verification and validation (V&V) efficiency and effectiveness. The state of the practice is manual and hence classification errors occur. This paper investigates the sensitivity of the value of bug classification (specifically, failure type classification) to its error rate; i.e., the degree to which misclassified historic bugs decrease the V&V effectiveness (i.e., the ability to find bugs of a failure type of interest). Results from the analysis of an industrial database of more than 3,000 bugs show that the impact of classification error rate on V&V effectiveness significantly varies with failure type. Specifically, there are failure types for which a 5% classification error can decrease the ability to find them by 66%. Conversely, there are failure types for which the V&V effectiveness is robust to very high error rates. These results show the utility of future research aimed at: 1) providing better tool support for decreasing human errors in classifying the failure type of bugs, 2) providing more robust approaches for the selection of V&V techniques, and 3) including robustness as an important criterion when evaluating technologies.

Original languageEnglish
Title of host publication36th International Conference on Software Engineering, ICSE Companion 2014 - Proceedings
Pages512-515
Number of pages4
DOIs
StatePublished - 2014
Event36th International Conference on Software Engineering, ICSE 2014 - Hyderabad, India
Duration: May 31 2014Jun 7 2014

Publication series

Name36th International Conference on Software Engineering, ICSE Companion 2014 - Proceedings

Conference

Conference36th International Conference on Software Engineering, ICSE 2014
Country/TerritoryIndia
CityHyderabad
Period5/31/146/7/14

Keywords

  • Bug classification
  • Human factor
  • Metrics
  • Software quality
  • Testing
  • Verification and validation

ASJC Scopus subject areas

  • Software

Cite this