Abstract
A wide range of commercial consumer devices such as mobile phones and smart televisions rely on embedded systems software to provide their functionality. Testing is one of the most commonly used methods for validating this software, and improved testing approaches could increase these devices' dependability. In this article we present an approach for performing such testing. Our approach is composed of two techniques. The first technique involves the selection of test data; it utilizes test adequacy criteria that rely on dataflow analysis to distinguish points of interaction between specific layers in embedded systems and between individual software components within those layers, while also tracking interactions between tasks. The second technique involves the observation of failures: it utilizes a family of test oracles that rely on instrumentation to record various aspects of a system's execution behavior, and compare observed behavior to certain intended system properties that can be derived through program analysis. Empirical studies of our approach show that our adequacy criteria can be effective at guiding the creation of test cases that detect faults, and our oracles can help expose faults that cannot easily be found using typical output-based oracles. Moreover, the use of our criteria accentuates the fault-detection effectiveness of our oracles.
Original language | English |
---|---|
Pages (from-to) | 207-230 |
Number of pages | 24 |
Journal | Journal of Systems and Software |
Volume | 88 |
Issue number | 1 |
DOIs | |
State | Published - 2014 |
Bibliographical note
Funding Information:Witawas Srisa-an received the MS and PhD in Computer Science from Illinois Institute of Technology. He joined the University of Nebraska-Lincoln (UNL) in 2002 and is currently an Associate Professor in the Department of Computer Science and Engineering. Prior to joining UNL, he was a researcher at Iowa State University. His research interests include programming languages, operating systems, virtual machines, and cyber security. His research projects have been supported by NSF, AFOSR, Microsoft, and DARPA.
Funding Information:
Gregg Rothermel received the Ph.D. in Computer Science from Clemson University, an M.S. in Computer Science from SUNY Albany, and a B.A. in Philosophy from Reed College. He is currently Professor and Jensen Chair of Software Engineering in the Department of Computer Science and Engineering at University of Nebraska - Lincoln. His research interests include software engineering and program analysis, with emphases on the application of program analysis techniques to problems in software maintenance and testing, and on empirical studies. Previous positions include Vice President, Quality Assurance and Quality Control, Palette Systems, Incorporated. His research has been supported by NSF, AFOSR, DARPA, Microsoft, Lockheed-Martin, and Boeing.
Funding Information:
This work was supported in part by the AFOSR through award FA9550-09-1-0129 to the University of Nebraska - Lincoln and through the Korea Research Foundation under award KRF-2007-357-D00215 . Wayne Motycka helped with the setup for the empirical study.
Keywords
- Embedded systems
- Software test adequacy criteria
- Test oracles
ASJC Scopus subject areas
- Software
- Information Systems
- Hardware and Architecture