| Reference : Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and Many-Ob... |
| Scientific congresses, symposiums and conference proceedings : Paper published in a book | |||
| Engineering, computing & technology : Computer science | |||
| Security, Reliability and Trust | |||
| http://hdl.handle.net/10993/50091 | |||
| Efficient Online Testing for DNN-Enabled Systems using Surrogate-Assisted and Many-Objective Optimization | |
| English | |
Ul Haq, Fitash  [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV >] | |
| Shin, Donghwan [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV >] | |
| Briand, Lionel [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV >] | |
| May-2022 | |
| Proceedings of the 44th International Conference on Software Engineering (ICSE ’22) | |
| ACM | |
| 811-822 | |
| Yes | |
| No | |
| International | |
| New York, NY | |
| USA | |
| 44th International Conference on Software Engineering (ICSE ’22) | |
| from 21-05-2022 to 29-05-2022 | |
| [en] DNN testing ; self-driving cars ; online testing ; many-objective search ; surrogate-assisted optimization | |
| [en] With the recent advances of Deep Neural Networks (DNNs) in real-world
applications, such as Automated Driving Systems (ADS) for self-driving cars, ensuring the reliability and safety of such DNN- enabled Systems emerges as a fundamental topic in software testing. One of the essential testing phases of such DNN-enabled systems is online testing, where the system under test is embedded into a specific and often simulated application environment (e.g., a driving environment) and tested in a closed-loop mode in interaction with the environment. However, despite the importance of online testing for detecting safety violations, automatically generating new and diverse test data that lead to safety violations present the following challenges: (1) there can be many safety requirements to be considered at the same time, (2) running a high-fidelity simulator is often very computationally-intensive, and (3) the space of all possible test data that may trigger safety violations is too large to be exhaustively explored. In this paper, we address the challenges by proposing a novel approach, called SAMOTA (Surrogate-Assisted Many-Objective Testing Approach), extending existing many-objective search algorithms for test suite generation to efficiently utilize surrogate models that mimic the simulator, but are much less expensive to run. Empirical evaluation results on Pylot, an advanced ADS composed of multiple DNNs, using CARLA, a high-fidelity driving simulator, show that SAMOTA is significantly more effective and efficient at detecting unknown safety requirement violations than state-of-the-art many-objective test suite generation algorithms and random search. In other words, SAMOTA appears to be a key enabler technology for online testing in practice. | |
| Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Software Verification and Validation Lab (SVV Lab) | |
| Fonds National de la Recherche - FnR | |
| http://hdl.handle.net/10993/50091 | |
| 10.1145/3510003.3510188 | |
| H2020 ; 694277 - TUNE - Testing the Untestable: Model Testing of Complex Software-Intensive Systems | |
| FnR ; FNR14711346 > Fabrizio Pastore > FUNTASY > Functional Safety For Autonomous Systems > 01/08/2020 > 31/07/2023 > 2020 |
| File(s) associated to this reference | ||||||||||||||
|
Fulltext file(s):
| ||||||||||||||
All documents in ORBilu are protected by a user license.
