Combining Genetic Algorithms and Constraint Programming to Support Stress Testing of Task Deadlines

in ACM Transactions on Software Engineering and Methodology (2015), 25(1),

Tasks in Real Time Embedded Systems (RTES) are often subject to hard deadlines, that constrain how quickly the system must react to external inputs. These inputs and their timing vary in a large domain ... [more ▼]

Tasks in Real Time Embedded Systems (RTES) are often subject to hard deadlines, that constrain how quickly the system must react to external inputs. These inputs and their timing vary in a large domain depending on the environment state, and can never be fully predicted prior to system execution. Therefore, approaches for stress testing must be developed to uncover possible deadline misses of tasks for different input arrival times. In this paper, we describe stress test case generation as a search problem over the space of task arrival times. Specifically, we search for worst case scenarios maximizing deadline misses where each scenario characterizes a test case. In order to scale our search to large industrial-size problems, we combine two state-of-the-art search strategies, namely Genetic Algorithms (GA) and Constraint Programming (CP). Our experimental results show that, in comparison with GA and CP in isolation, GA+CP achieves nearly the same effectiveness as CP and the same efficiency and solution diversity as GA, thus combining the advantages of the two strategies. In light of these results, we conclude that a combined GA+CP approach to stress testing is more likely to scale to large and complex systems. [less ▲]

Stress Testing of Task Deadlines: A Constraint Programming Approach

in The 24th IEEE International Symposium on Software Reliability Engineering (ISSRE 2013), Pasadena, CA, November 2013 (2013)

Safety-critical Real Time Embedded Systems (RTESs) are usually subject to strict timing and performance requirements that must be satisfied for the system to be deemed safe. In this paper, we use ... [more ▼]

Safety-critical Real Time Embedded Systems (RTESs) are usually subject to strict timing and performance requirements that must be satisfied for the system to be deemed safe. In this paper, we use effective search strategies that aim at finding worst case scenarios with respect to deadline misses. Such scenarios can in turn be used to test the target RTES and ensure that, even under worst case conditions, it satisfies its timing requirements. Specifically, we develop a solution based on Constraint Programming (CP) to automate the generation of test cases that reveal, or are likely to, task deadline misses. We evaluate it through a comparison with a recent, state-of-the-art approach based on Genetic Algorithms (GA). In particular, we compare CP and GA in five industry-inspired case studies for efficiency, effectiveness, and scalability. Our experimental results show that, on the largest and more complex case studies, CP performs significantly better than GA. Since CP has interesting properties, such as guaranteeing complete search when there is sufficient time, and enables the definition of effective heuristics to converge faster towards optimal solutions, we conclude that our results are encouraging and suggest this is an advantageous solution for the stress testing of RTESs with respect to timing constraints. [less ▲]

Modeling and Analysis of CPU Usage in Safety-Critical Embedded Systems to Support Stress Testing

in 15th ACM/IEEE International Conference on Model Driven Engineering Languages & Systems (2012)