References of "Ben Abdessalem (helali), Raja 50003537"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailDigital Twins Are Not Monozygotic -- Cross-Replicating ADAS Testing in Two Industry-Grade Automotive Simulators
Borg, Markus; Ben Abdessalem (helali), Raja UL; Nejati, Shiva UL et al

in 2021 IEEE 14th International Conference on Software Testing, Validation and Verification (ICST) (2021, May 25)

The increasing levels of software- and data-intensive driving automation call for an evolution of automotive software testing. As a recommended practice of the Verification and Validation (V&V) process of ... [more ▼]

The increasing levels of software- and data-intensive driving automation call for an evolution of automotive software testing. As a recommended practice of the Verification and Validation (V&V) process of ISO/PAS 21448, a candidate standard for safety of the intended functionality for road vehicles, simulation-based testing has the potential to reduce both risks and costs. There is a growing body of research on devising test automation techniques using simulators for Advanced Driver-Assistance Systems (ADAS). However, how similar are the results if the same test scenarios are executed in different simulators? We conduct a replication study of applying a Search-Based Software Testing (SBST) solution to a real-world ADAS (PeVi, a pedestrian vision detection system) using two different commercial simulators, namely, TASS/Siemens PreScan and ESI Pro-SiVIC. Based on a minimalistic scene, we compare critical test scenarios generated using our SBST solution in these two simulators. We show that SBST can be used to effectively and efficiently generate critical test scenarios in both simulators, and the test results obtained from the two simulators can reveal several weaknesses of the ADAS under test. However, executing the same test scenarios in the two simulators leads to notable differences in the details of the test outputs, in particular, related to (1) safety violations revealed by tests, and (2) dynamics of cars and pedestrians. Based on our findings, we recommend future V&V plans to include multiple simulators to support robust simulation-based testing and to base test objectives on measures that are less dependant on the internals of the simulators. [less ▲]

Detailed reference viewed: 70 (7 UL)
Full Text
See detailEffective Testing Of Advanced Driver Assistance Systems Using Evolutionary Algorithms And Machine Learning
Ben Abdessalem (helali), Raja UL

Doctoral thesis (2019)

Improving road safety is a major concern for most car manufacturers. In recent years, the development of Advanced Driver Assistance Systems (ADAS) has subsequently seen a tremendous boost. The development ... [more ▼]

Improving road safety is a major concern for most car manufacturers. In recent years, the development of Advanced Driver Assistance Systems (ADAS) has subsequently seen a tremendous boost. The development of such systems requires complex testing to ensure vehicle’s safety and reliability. Performing road tests tends to be dangerous, time-consuming, and costly. Hence, a large part of testing for ADAS has to be carried out using physics-based simulation platforms, which are able to emulate a wide range of virtual traffic scenarios and road environments. The main difficulties with simulation-based testing of ADAS are: (1) the test input space is large and multidimensional, (2) simulation platforms provide no guidance to engineers as to which scenarios should be selected for testing, and hence, simulation is limited to a small number of scenarios hand-picked by engineers, and (3) test executions are computationally expensive because they often involve executing high-fidelity mathematical models capturing continuous dynamic behaviors of vehicles and their environment. The complexity of testing ADAS is further exacerbated when many ADAS are employed together in a self-driving system. In particular, when self-driving systems include many ADAS (i.e., features), they tend to interact and impact one another’s behavior in an unknown way and may lead to conflicting situations. The main challenge here is to detect and manage feature interactions, in particular, those that violate system safety requirements, hence leading to critical failures. In practice, once feature interaction failures are detected, engineers need to devise resolution strategies to resolve potential conflicts between features. Developing resolution strategies is a complex task and despite the extensive domain expertise, these resolution strategies can be erroneous and are too complex to be manually repaired. In this dissertation, in addition to testing individual ADAS, we focus on testing self-driving systems that include several ADAS. In this dissertation, we propose a set of approaches based on meta-heuristic search and machine learning techniques to automate ADAS testing and to repair feature interaction failures in self-driving systems. The work presented in this dissertation is motivated by ADAS testing needs at IEE, a world-leading part supplier to the automotive industry. In this dissertation, we focus on the problem of design time testing of ADAS in a simulated environment, relying on Simulink models. The main research contributions in this dissertation are: - A testing approach for ADAS that combines multi-objective search with surrogate models to guide testing towards the most critical behaviors of ADAS, and to explore a larger part of the input search space with less computational resources. - An automated testing algorithm that builds on learnable evolution models and uses classification decision trees to guide the generation of new test scenarios within complex and multidimensional input spaces and help engineers interpret test results. - An automated technique that detects feature interaction failures in the context of self-driving systems based on analyzing executable function models typically developed to specify system behaviors at early development stages. - An automated technique that uses a new many-objective search algorithm to localize and repair errors in the feature interaction resolution rules for self-driving systems. [less ▲]

Detailed reference viewed: 295 (35 UL)
Full Text
Peer Reviewed
See detailTesting Autonomous Cars for Feature Interaction Failures using Many-Objective Search
Ben Abdessalem (helali), Raja UL; Panichella, Annibale; Nejati, Shiva UL et al

in Proceedings of the 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2018) (2018)

Complex systems such as autonomous cars are typically built as a composition of features that are independent units of functionality. Features tend to interact and impact one another’s behavior in unknown ... [more ▼]

Complex systems such as autonomous cars are typically built as a composition of features that are independent units of functionality. Features tend to interact and impact one another’s behavior in unknown ways. A challenge is to detect and manage feature interactions, in particular, those that violate system requirements, hence leading to failures. In this paper, we propose a technique to detect feature interaction failures by casting our approach into a search-based test generation problem. We define a set of hybrid test objectives (distance functions) that combine traditional coverage-based heuristics with new heuristics specifically aimed at revealing feature interaction failures. We develop a new search-based test generation algorithm, called FITEST, that is guided by our hybrid test objectives. FITEST extends recently proposed many-objective evolutionary algorithms to reduce the time required to compute fitness values. We evaluate our approach using two versions of an industrial self-driving system. Our results show that our hybrid test objectives are able to identify more than twice as many feature interaction failures as two baseline test objectives used in the software testing literature (i.e., coverage-based and failure-based test objectives). Further, the feedback from domain experts indicates that the detected feature interaction failures represent real faults in their systems that were not previously identified based on analysis of the system features and their requirements. [less ▲]

Detailed reference viewed: 610 (75 UL)
Full Text
Peer Reviewed
See detailTesting Vision-Based Control Systems Using Learnable Evolutionary Algorithms
Ben Abdessalem (helali), Raja UL; Nejati, Shiva UL; Briand, Lionel UL et al

in Proceedings of the 40th International Conference on Software Engineering (ICSE 2018) (2018)

Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We ... [more ▼]

Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We propose an automated testing algorithm that builds on learnable evolutionary algorithms. These algorithms rely on machine learning or a combination of machine learning and Darwinian genetic operators to guide the generation of new solutions (test scenarios in our context). Our approach combines multiobjective population-based search algorithms and decision tree classification models to achieve the following goals: First, classification models guide the search-based generation of tests faster towards critical test scenarios (i.e., test scenarios leading to failures). Second, search algorithms refine classification models so that the models can accurately characterize critical regions (i.e., the regions of a test input space that are likely to contain most critical test scenarios). Our evaluation performed on an industrial automotive vision-based control system shows that: (1) Our algorithm outperforms a baseline evolutionary search algorithm and generates 78% more distinct, critical test scenarios compared to the baseline algorithm. (2) Our algorithm accurately characterizes critical regions of the system under test, thus identifying the conditions that likely to lead to system failures. [less ▲]

Detailed reference viewed: 862 (180 UL)
Full Text
Peer Reviewed
See detailTesting Advanced Driver Assistance Systems using Multi-objective Search and Neural Networks
Ben Abdessalem (helali), Raja UL; Nejati, Shiva UL; Briand, Lionel UL et al

in International Conference on Automated Software Engineering (ASE 2016) (2016)

Recent years have seen a proliferation of complex Advanced Driver Assistance Systems (ADAS), in particular, for use in autonomous cars. These systems consist of sensors and cameras as well as image ... [more ▼]

Recent years have seen a proliferation of complex Advanced Driver Assistance Systems (ADAS), in particular, for use in autonomous cars. These systems consist of sensors and cameras as well as image processing and decision support software components. They are meant to help drivers by providing proper warnings or by preventing dangerous situations. In this paper, we focus on the problem of design time testing of ADAS in a simulated environment. We provide a testing approach for ADAS by combining multi- objective search with surrogate models developed based on neural networks. We use multi-objective search to guide testing towards the most critical behaviors of ADAS. Surrogate modeling enables our testing approach to explore a larger part of the input search space within limited computational resources. We characterize the condition under which the multi-objective search algorithm behaves the same with and without surrogate modeling, thus showing the accuracy of our approach. We evaluate our approach by applying it to an industrial ADAS system. Our experiment shows that our approach automatically identifies test cases indicating critical ADAS behaviors. Further, we show that combining our search algorithm with surrogate modeling improves the quality of the generated test cases, especially under tight and realistic computational resources. [less ▲]

Detailed reference viewed: 652 (143 UL)