Paper published in a book (Scientific congresses, symposiums and conference proceedings)
Automatic Test Suite Generation for Key-Points Detection DNNs using Many-Objective Search (Experience Paper)
UL HAQ, Fitash; SHIN, Donghwan; BRIAND, Lionel et al.
2021In 2021 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)
Peer reviewed
 

Files


Full Text
main.pdf
Author postprint (2.07 MB)
author postprint
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
Key-point detection; deep neural network; software testing; many-objective search algorithm
Abstract :
[en] Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver's gaze detection and drowsiness detection in automated driving systems. With the recent advances of Deep Neural Networks (DNNs), Key-Points detection DNNs (KP-DNNs) have been increasingly employed for that purpose. Nevertheless, KP-DNN testing and validation have remained a challenging problem because KP-DNNs predict many independent key-points at the same time---where each individual key-point may be critical in the targeted application---and images can vary a great deal according to many factors. In this paper, we present an approach to automatically generate test data for KP-DNNs using many-objective search. In our experiments, focused on facial key-points detection DNNs developed for an industrial automotive application, we show that our approach can generate test suites to severely mispredict, on average, more than 93% of all key-points. In comparison, random search-based test data generation can only severely mispredict 41% of them. Many of these mispredictions, however, are not avoidable and should not therefore be considered failures. We also empirically compare state-of-the-art, many-objective search algorithms and their variants, tailored for test suite generation. Furthermore, we investigate and demonstrate how to learn specific conditions, based on image characteristics (e.g., head posture and skin color), that lead to severe mispredictions. Such conditions serve as a basis for risk analysis or DNN retraining.
Disciplines :
Computer science
Author, co-author :
UL HAQ, Fitash ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV
SHIN, Donghwan ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV
BRIAND, Lionel ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV
Stifter, Thomas;  IEE S.A.
Wang, Jun;  Post Luxembourg
External co-authors :
no
Language :
English
Title :
Automatic Test Suite Generation for Key-Points Detection DNNs using Many-Objective Search (Experience Paper)
Publication date :
July 2021
Event name :
INTERNATIONAL SYMPOSIUM ON SOFTWARE TESTING AND ANALYSIS
Event date :
from 11-07-2021 to 17-07-2021
Audience :
International
Main work title :
2021 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)
Pages :
91-102
Peer reviewed :
Peer reviewed
Focus Area :
Security, Reliability and Trust
European Projects :
H2020 - 694277 - TUNE - Testing the Untestable: Model Testing of Complex Software-Intensive Systems
FnR Project :
FNR14711346 - Functional Safety For Autonomous Systems, 2020 (01/08/2020-31/07/2023) - Fabrizio Pastore
Funders :
CE - Commission Européenne [BE]
Available on ORBilu :
since 30 April 2021

Statistics


Number of views
401 (96 by Unilu)
Number of downloads
331 (37 by Unilu)

Bibliography


Similar publications



Contact ORBilu