CAREER: Testing Evolving Embedded Software Systems

Throughout their lifetimes, software systems undergo numerous changes. A considerable percentage of development and maintenance costs are due to regression testing, which has been widely used to assess whether changes have adversely affected system behavior. While significant work is accomplished by the software engineering community in improving effectiveness and efficiency of regression testing, most of existing techniques focus on traditional software that is homogeneous, environment-independent and non-distributed. However, real-world embedded systems are far more complex: they frequently interact with environment via hardware devices, and employ different concurrency mechanisms to coordinate interrupts, signals, threads and processes. Changes in these systems may affect not only the sequential logic of a single piece of application, but also the interleaving space of a variety of events across the whole system layers. These characteristics affect the change impact analysis, dynamic analysis, and test case generation techniques on which existing regressiontesting approaches rely. Therefore, applying these approaches can result in inadequately tested software during maintenance, and ultimately impair software quality. The overall goal of this proposal is to create a novel unified testing and analysis framework for ensuring dependability of real-world embedded software systems throughout their lifetime. To achieve this goal, we propose a set of novel techniques that can model software systems in the presence of various levels concurrency and frequent hardware interactions, perform change impact analysis as the model evolves, and ultimately make available a new breed of regression testing techniques. To reduce costs, we propose a feedback-driven hybrid approaches blending of environment-dependent impact analysis, dynamic analysis and predictive scheduling techniques for testing a subset of system execution space with high precision. To improve effectiveness, we will investigate novel combinations of taint analysis, symbolic execution and dynamic invariant inference for automatically generating additional test cases and system event interleaving schedules. We also propose new context-aware dependency analysis techniques combined with search-based testing techniques to guide regression testing towards identifying performance regression due to over-constrained synchronizations (e.g., after fixing a race condition). The proposed framework will take advantage of virtual platforms, that have been widely used in various kind of activities in system development and cloud adaptations. Intellectual Merit. Modern embedded systems are prevalent and evolve rapidly, yet their most distinct characteristics, concurrency and environment-dependency, are often ignored in regression testing. This research will provide the first unified framework to tackle this problem. It goes well beyond the state of the art in regression testing, by addressing issues and developing technical solutions that are specific to these systems. The transformative nature of the work derives from rethinking traditional assumptions about software evolution. The PI has a history of successful research on developing software engineering techniques to address reliability issues in concurrent and embedded systems, and a history of publishing research results in top-tier conferences. Thus, the PI has a set of unique skills to accomplish this research. Broader Impacts. Concurrency and hardware related software defects are among the most dangerous in practice, and due to the increasingly widespread use of embedded devices, multicore processors, cloud platforms, and distributed data-centers, have become a major threat to the reliability and security of modern systems and the nation’s critical infrastructure. Our research will help create more reliable and economical software systems and infrastructures. Our integrated education plan aims to quickly incorporate courses such as software engineering in embedded systems into the curriculum materials that will be made available to the broader scientific community, which will help prepare a globally competitive workforce and further benefit society. The PI will leverage her collaborations with industry to engage industry partners (e.g. ABB corporate research) in curricular development, problem sharing, technology transfer, and providing summer intern opportunities. KeyWords: regression testing, test case generation, embedded systems, concurrency, virtual platforms