Paper published in a book (Scientific congresses, symposiums and conference proceedings)
PEELER: Learning to Effectively Predict Flakiness without Running Tests
Qin, Yihao; Wang, Shangwen; Liu, Kui et al.
2022In Proceedings of the 38th IEEE International Conference on Software Maintenance and Evolution
Peer reviewed
 

Files


Full Text
ICSME_2022_Flaky_test.pdf
Author preprint (729.66 kB)
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
Flaky tests; Deep learning; Program dependency
Abstract :
[en] —Regression testing is a widely adopted approach to expose change-induced bugs as well as to verify the correctness/robustness of code in modern software development settings. Unfortunately, the occurrence of flaky tests leads to a significant increase in the cost of regression testing and eventually reduces the productivity of developers (i.e., their ability to find and fix real problems). State-of-the-art approaches leverage dynamic test information obtained through expensive re-execution of test cases to effectively identify flaky tests. Towards accounting for scalability constraints, some recent approaches have built on static test case features, but fall short on effectiveness. In this paper, we introduce PEELER, a new fully static approach for predicting flaky tests through exploring a representation of test cases based on the data dependency relations. The predictor is then trained as a neural network based model, which achieves at the same time scalability (because it does not require any test execution), effectiveness (because it exploits relevant test dependency features), and practicality (because it can be applied in the wild to find new flaky tests). Experimental validation on 17,532 test cases from 21 Java projects shows that PEELER outperforms the state-of-the-art FlakeFlagger by around 20 percentage points: we catch 22% more flaky tests while yielding 51% less false positives. Finally, in a live study with projects in-the-wild, we reported to developers 21 flakiness cases, among which 12 have already been confirmed by developers as being indeed flaky.
Disciplines :
Computer science
Author, co-author :
Qin, Yihao;  National University of Defense Technology, China
Wang, Shangwen;  National University of Defense Technology, China
Liu, Kui;  Huawei Software Engineering Application Technology Lab, China
Lin, Bo;  National University of Defense Technology
Wu, Hongjun;  National University of Defense Technology, China
Li, Li;  Monash University, Australia
Mao, Xiaoguang;  National University of Defense Technology, China
Bissyande, Tegawendé François D Assise  ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > TruX
External co-authors :
yes
Language :
English
Title :
PEELER: Learning to Effectively Predict Flakiness without Running Tests
Publication date :
October 2022
Event name :
38th IEEE International Conference on Software Maintenance and Evolution
Event organizer :
IEEE
Event place :
Limassol, Cyprus
Event date :
from 02-10-2022 to 07-10-2022
Audience :
International
Main work title :
Proceedings of the 38th IEEE International Conference on Software Maintenance and Evolution
Pages :
1-12
Peer reviewed :
Peer reviewed
Focus Area :
Security, Reliability and Trust
European Projects :
H2020 - 949014 - NATURAL - Natural Program Repair
Funders :
CE - Commission Européenne [BE]
Available on ORBilu :
since 24 September 2022

Statistics


Number of views
139 (19 by Unilu)
Number of downloads
190 (15 by Unilu)

Scopus citations®
 
4
Scopus citations®
without self-citations
4

Bibliography


Similar publications



Contact ORBilu