An approach to testing commercial embedded systems

Tingting Yu, Ahyoung Sung, Witawas Srisa-An, Gregg Rothermel

Research output: Contribution to journalArticlepeer-review

9 Scopus citations


A wide range of commercial consumer devices such as mobile phones and smart televisions rely on embedded systems software to provide their functionality. Testing is one of the most commonly used methods for validating this software, and improved testing approaches could increase these devices' dependability. In this article we present an approach for performing such testing. Our approach is composed of two techniques. The first technique involves the selection of test data; it utilizes test adequacy criteria that rely on dataflow analysis to distinguish points of interaction between specific layers in embedded systems and between individual software components within those layers, while also tracking interactions between tasks. The second technique involves the observation of failures: it utilizes a family of test oracles that rely on instrumentation to record various aspects of a system's execution behavior, and compare observed behavior to certain intended system properties that can be derived through program analysis. Empirical studies of our approach show that our adequacy criteria can be effective at guiding the creation of test cases that detect faults, and our oracles can help expose faults that cannot easily be found using typical output-based oracles. Moreover, the use of our criteria accentuates the fault-detection effectiveness of our oracles.

Original languageEnglish
Pages (from-to)207-230
Number of pages24
JournalJournal of Systems and Software
Issue number1
StatePublished - 2014

Bibliographical note

Funding Information:
Witawas Srisa-an received the MS and PhD in Computer Science from Illinois Institute of Technology. He joined the University of Nebraska-Lincoln (UNL) in 2002 and is currently an Associate Professor in the Department of Computer Science and Engineering. Prior to joining UNL, he was a researcher at Iowa State University. His research interests include programming languages, operating systems, virtual machines, and cyber security. His research projects have been supported by NSF, AFOSR, Microsoft, and DARPA.

Funding Information:
Gregg Rothermel received the Ph.D. in Computer Science from Clemson University, an M.S. in Computer Science from SUNY Albany, and a B.A. in Philosophy from Reed College. He is currently Professor and Jensen Chair of Software Engineering in the Department of Computer Science and Engineering at University of Nebraska - Lincoln. His research interests include software engineering and program analysis, with emphases on the application of program analysis techniques to problems in software maintenance and testing, and on empirical studies. Previous positions include Vice President, Quality Assurance and Quality Control, Palette Systems, Incorporated. His research has been supported by NSF, AFOSR, DARPA, Microsoft, Lockheed-Martin, and Boeing.

Funding Information:
This work was supported in part by the AFOSR through award FA9550-09-1-0129 to the University of Nebraska - Lincoln and through the Korea Research Foundation under award KRF-2007-357-D00215 . Wayne Motycka helped with the setup for the empirical study.


  • Embedded systems
  • Software test adequacy criteria
  • Test oracles

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture


Dive into the research topics of 'An approach to testing commercial embedded systems'. Together they form a unique fingerprint.

Cite this