References of "Bagherzadeh, Mojtaba"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailReinforcement Learning for Test Case Prioritization
Bagherzadeh, Mojtaba; Kahani, Nafiseh; Briand, Lionel UL

in IEEE Transactions on Software Engineering (in press)

Continuous Integration (CI) context significantly reduces integration problems, speeds up development time, and shortens release time. However, it also introduces new challenges for quality assurance ... [more ▼]

Continuous Integration (CI) context significantly reduces integration problems, speeds up development time, and shortens release time. However, it also introduces new challenges for quality assurance activities, including regression testing, which is the focus of this work. Though various approaches for test case prioritization have shown to be very promising in the context of regression testing, specific techniques must be designed to deal with the dynamic nature and timing constraints of CI. Recently, Reinforcement Learning (RL) has shown great potential in various challenging scenarios that require continuous adaptation, such as game playing, real-time ads bidding, and recommender systems. Inspired by this line of work and building on initial efforts in supporting test case prioritization with RL techniques, we perform here a comprehensive investigation of RL-based test case prioritization in a CI context. To this end, taking test case prioritization as a ranking problem, we model the sequential interactions between the CI environment and a test case prioritization agent as an RL problem, using three alternative ranking models. We then rely on carefully selected and tailored state-of-the-art RL techniques to automatically and continuously learn a test case prioritization strategy, whose objective is to be as close as possible to the optimal one. Our extensive experimental analysis shows that the best RL solutions provide a significant accuracy improvement over previous RL-based work, with prioritization strategies getting close to being optimal, thus paving the way for using RL to prioritize test cases in a CI context. [less ▲]

Detailed reference viewed: 61 (1 UL)
Full Text
Peer Reviewed
See detailTest Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review
Pan, Rongqi; Bagherzadeh, Mojtaba; Ghaleb, Taher et al

in Empirical Software Engineering (in press)

Regression testing is an essential activity to assure that software code changes do not adversely a ect existing functionalities. With the wide adoption of Continuous Integration (CI) in software projects ... [more ▼]

Regression testing is an essential activity to assure that software code changes do not adversely a ect existing functionalities. With the wide adoption of Continuous Integration (CI) in software projects, which increases the frequency of running software builds, running all tests can be time-consuming and resource-intensive. To alleviate that problem, Test case Selection and Prioritiza- tion (TSP) techniques have been proposed to improve regression testing by selecting and prioritizing test cases in order to provide early feedback to developers. In recent years, researchers have relied on Machine Learning (ML) techniques to achieve e ective TSP (ML-based TSP). Such techniques help combine information about test cases, from partial and imperfect sources, into accurate prediction models. This work conducts a systematic literature review focused on ML-based TSP techniques, aiming to perform an in-depth analysis of the state of the art, thus gaining insights regarding fu- ture avenues of research. To that end, we analyze 29 primary studies published from 2006 to 2020, which have been identi ed through a systematic and documented process. This paper addresses ve research questions addressing variations in ML-based TSP techniques and feature sets for training and testing ML models, alternative metrics used for evaluating the techniques, the performance of techniques, and the reproducibility of the published studies. We summarize the results related to our research questions in a high-level summary that can be used as a taxonomy for classifying future TSP studies. [less ▲]

Detailed reference viewed: 62 (3 UL)
Full Text
Peer Reviewed
See detailSupporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learning
Fahmy, Hazem UL; Pastore, Fabrizio UL; Bagherzadeh, Mojtaba et al

in IEEE Transactions on Reliability (in press)

Deep neural networks (DNNs) are increasingly im- portant in safety-critical systems, for example in their perception layer to analyze images. Unfortunately, there is a lack of methods to ensure the ... [more ▼]

Deep neural networks (DNNs) are increasingly im- portant in safety-critical systems, for example in their perception layer to analyze images. Unfortunately, there is a lack of methods to ensure the functional safety of DNN-based components. We observe three major challenges with existing practices regarding DNNs in safety-critical systems: (1) scenarios that are underrepresented in the test set may lead to serious safety violation risks, but may, however, remain unnoticed; (2) char- acterizing such high-risk scenarios is critical for safety analysis; (3) retraining DNNs to address these risks is poorly supported when causes of violations are difficult to determine. To address these problems in the context of DNNs analyzing images, we propose HUDD, an approach that automatically supports the identification of root causes for DNN errors. HUDD identifies root causes by applying a clustering algorithm to heatmaps capturing the relevance of every DNN neuron on the DNN outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters. We evaluated HUDD with DNNs from the automotive domain. HUDD was able to identify all the distinct root causes of DNN errors, thus supporting safety analysis. Also, our retraining approach has shown to be more effective at improving DNN accuracy than existing approaches. [less ▲]

Detailed reference viewed: 91 (8 UL)