Article (Scientific journals)
Supporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learning
FAHMY, Hazem; PASTORE, Fabrizio; Bagherzadeh, Mojtaba et al.
2021In IEEE Transactions on Reliability, 70 (4), p. 1641-1657
Peer Reviewed verified by ORBi
 

Files


Full Text
HUDD.pdf
Author postprint (1.82 MB)
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
DNN Explanation; DNN Functional Safety Analysis; Debugging; Heatmaps; AI
Abstract :
[en] Deep neural networks (DNNs) are increasingly im- portant in safety-critical systems, for example in their perception layer to analyze images. Unfortunately, there is a lack of methods to ensure the functional safety of DNN-based components. We observe three major challenges with existing practices regarding DNNs in safety-critical systems: (1) scenarios that are underrepresented in the test set may lead to serious safety violation risks, but may, however, remain unnoticed; (2) char- acterizing such high-risk scenarios is critical for safety analysis; (3) retraining DNNs to address these risks is poorly supported when causes of violations are difficult to determine. To address these problems in the context of DNNs analyzing images, we propose HUDD, an approach that automatically supports the identification of root causes for DNN errors. HUDD identifies root causes by applying a clustering algorithm to heatmaps capturing the relevance of every DNN neuron on the DNN outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters. We evaluated HUDD with DNNs from the automotive domain. HUDD was able to identify all the distinct root causes of DNN errors, thus supporting safety analysis. Also, our retraining approach has shown to be more effective at improving DNN accuracy than existing approaches.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Software Verification and Validation Lab (SVV Lab)
ULHPC - University of Luxembourg: High Performance Computing
Disciplines :
Computer science
Author, co-author :
FAHMY, Hazem ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV
PASTORE, Fabrizio  ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV
Bagherzadeh, Mojtaba;  University of Ottawa
BRIAND, Lionel ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SVV
External co-authors :
yes
Language :
English
Title :
Supporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learning
Publication date :
May 2021
Journal title :
IEEE Transactions on Reliability
eISSN :
0018-9529
Publisher :
Institute of Electrical and Electronics Engineers, New-York, United States - New York
Special issue title :
Special Section on Quality Assurance of Machine Learning Systems
Volume :
70
Issue :
4
Pages :
1641-1657
Peer reviewed :
Peer Reviewed verified by ORBi
European Projects :
H2020 - 694277 - TUNE - Testing the Untestable: Model Testing of Complex Software-Intensive Systems
FnR Project :
FNR14711346 - Functional Safety For Autonomous Systems, 2020 (01/08/2020-31/07/2023) - Fabrizio Pastore
Name of the research project :
BRIDGES2020/IS/14711346/FUNTASY
Funders :
CE - Commission Européenne
European Union
Available on ORBilu :
since 19 April 2021

Statistics


Number of views
351 (70 by Unilu)
Number of downloads
130 (21 by Unilu)

Scopus citations®
 
26
Scopus citations®
without self-citations
17
OpenCitations
 
12
OpenAlex citations
 
25
WoS citations
 
21

publications
0
supporting
0
mentioning
0
contrasting
0
Smart Citations
0
0
0
0
Citing PublicationsSupportingMentioningContrasting
View Citations

See how this article has been cited at scite.ai

scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.

Bibliography


Similar publications



Sorry the service is unavailable at the moment. Please try again later.
Contact ORBilu