Article (Scientific journals)
On the use of commit-relevant mutants
Ojdanic, Milos; Ma, Wei; Laurent, Thomas et al.
2022In Empirical Software Engineering, 27
Peer Reviewed verified by ORBi


Full Text
Publisher postprint (3.37 MB)

All documents in ORBilu are protected by a user license.

Send to


Keywords :
Mutation testing; Change-relevant mutants; Continuous integration; Regression testing
Abstract :
[en] Applying mutation testing to test subtle program changes, such as program patches or other small-scale code modifications, requires using mutants that capture the delta of the altered behaviours. To address this issue, we introduce the concept of commit-relevant mutants, which are the mutants that interact with the behaviours of the system affected by a particular commit. Therefore, commit-aware mutation testing, is a test assessment metric tailored to a specific commit. By analysing 83 commits from 25 projects involving 2,253,610 mutants in both C and Java, we identify the commit-relevant mutants and explore their relationship with other categories of mutants. Our results show that commit-relevant mutants represent a small subset of all mutants, which differs from the other classes of mutants (subsuming and hard-to-kill), and that the commit-relevant mutation score is weakly correlated with the traditional mutation score (Kendall/Pearson 0.15-0.4). Moreover, commit-aware mutation analysis provides insights about the testing of a commit, which can be more efficient than the classical mutation analysis; in our experiments, by analysing the same number of mutants, commit-aware mutants have better fault-revelation potential (30% higher chances of revealing commit-introducing faults) than traditional mutants. We also illustrate a possible application of commit-aware mutation testing as a metric to evaluate test case prioritisation.
Disciplines :
Computer science
Author, co-author :
Ojdanic, Milos   ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SerVal
Ma, Wei 
Laurent, Thomas
Titcheu Chekam, Thierry
Ventresque, Anthony
Papadakis, Mike ;  University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
 These authors have contributed equally to this work.
External co-authors :
Language :
Title :
On the use of commit-relevant mutants
Publication date :
30 May 2022
Journal title :
Empirical Software Engineering
Publisher :
Kluwer Academic Publishers, Netherlands
Volume :
Peer reviewed :
Peer Reviewed verified by ORBi
FnR Project :
FNR11686509 - Continuous Development With Mutation Analysis And Testing, 2017 (01/09/2018-31/08/2021) - Michail Papadakis
Available on ORBilu :
since 16 January 2023


Number of views
22 (3 by Unilu)
Number of downloads
8 (0 by Unilu)

Scopus citations®
Scopus citations®
without self-citations
WoS citations


Similar publications

Contact ORBilu