Error leakage and wasted time: sensitivity and effort analysis of a requirements consistency checking process

Wenbin Li, Jane Huffman Hayes, Giulio Antoniol, Yann Gaël Guéhéneuc, Bram Adams

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Several techniques are used by requirements engineering practitioners to address difficult problems such as specifying precise requirements while using inherently ambiguous natural language text and ensuring the consistency of requirements. Often, these problems are addressed by building processes/tools that combine multiple techniques where the output from 1 technique becomes the input to the next. While powerful, these techniques are not without problems. Inherent errors in each technique may leak into the subsequent step of the process. We model and study 1 such process, for checking the consistency of temporal requirements, and assess error leakage and wasted time. We perform an analysis of the input factors of our model to determine the effect that sources of uncertainty may have on the final accuracy of the consistency checking process. Convinced that error leakage exists and negatively impacts the results of the overall consistency checking process, we perform a second simulation to assess its impact on the analysts' efforts to check requirements consistency. We show that analyst's effort varies depending on the precision and recall of the subprocesses and that the number and capability of analysts affect their effort. We share insights gained and discuss applicability to other processes built of piped techniques.

Original languageEnglish
Pages (from-to)1061-1080
Number of pages20
JournalJournal of software: Evolution and Process
Issue number12
StatePublished - Dec 1 2016

Bibliographical note

Funding Information:
We thank Drs Stromberg and Bathke, and Joshua Lambert of the Statistics Dept for reviewing earlier versions of the paper. We thank the NSF, under grant CCF-1511117, for partially funding this research.

Publisher Copyright:
Copyright © 2016 John Wiley & Sons, Ltd.


  • classification
  • consistency checking
  • error leakage
  • error propagation
  • genetic algorithms
  • information retrieval
  • machine learning
  • natural language processing
  • process model
  • requirements engineering
  • search-based software engineering
  • semantic role labeling
  • sensitivity analysis
  • work flow process

ASJC Scopus subject areas

  • Software


Dive into the research topics of 'Error leakage and wasted time: sensitivity and effort analysis of a requirements consistency checking process'. Together they form a unique fingerprint.

Cite this