Reference : Are mutants really natural? A study on how “naturalness” helps mutant selection
Scientific congresses, symposiums and conference proceedings : Paper published in a journal
Engineering, computing & technology : Computer science
Security, Reliability and Trust
http://hdl.handle.net/10993/36854
Are mutants really natural? A study on how “naturalness” helps mutant selection
English
Jimenez, Matthieu mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Titcheu Chekam, Thierry mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Cordy, Maxime mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Papadakis, Mike mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Computer Science and Communications Research Unit (CSC) >]
Kintis, Marinos mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Le Traon, Yves mailto [University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC) >]
Harman, Mark mailto [University College London - UCL]
11-Oct-2018
12th International Symposium on 
 Empirical Software Engineering and Measurement (ESEM'18)
Yes
International
12th International Symposium on 
 Empirical Software Engineering and Measurement (ESEM'18)
from 11-10-18 to 13-10-18
Oulu
Finlans
[en] Mutation testing ; Fault Revelation ; Language Models
[en] Background: Code is repetitive and predictable in a way that is similar to the natural language. This means that code is ``natural'' and this ``naturalness'' can be captured by natural language modelling techniques.
Such models promise to capture the program semantics and identify source code parts that `smell', i.e., they are strange, badly written and are generally error-prone (likely to be defective).
Aims: We investigate the use of natural language modelling techniques in mutation testing (a testing technique that uses artificial faults). We thus, seek to identify how well artificial faults simulate real ones and ultimately understand how natural the artificial faults can be. %We investigate this question in a fault revelation perspective.
Our intuition is that natural mutants, i.e., mutants that are predictable (follow the implicit coding norms of developers), are semantically useful and generally valuable (to testers). We also expect that mutants located on unnatural code locations (which are generally linked with error-proneness) to be of higher value than those located on natural code locations.
Method:
Based on this idea, we propose mutant selection strategies that rank mutants according to a) their naturalness (naturalness of the mutated code), b) the naturalness of their locations (naturalness of the original program statements) and c) their impact on the naturalness of the code that they apply to (naturalness differences between original and mutated statements). We empirically evaluate these issues on a benchmark set of 5 open-source projects, involving more than 100k mutants and 230 real faults. Based on the fault set we estimate the utility (i.e. capability to reveal faults) of mutants selected on the basis of their naturalness, and compare it against the utility of randomly selected mutants.
Results:
Our analysis shows that there is no link between naturalness and the fault revelation utility of mutants. We also demonstrate that the naturalness-based mutant selection performs similar (slightly worse) to the random mutant selection.
Conclusions:
Our findings are negative
but we consider them interesting as they confute a strong intuition, i.e., fault revelation is independent of the mutants' naturalness.
http://hdl.handle.net/10993/36854

File(s) associated to this reference

Fulltext file(s):

FileCommentaryVersionSizeAccess
Open access
fse18.pdfAuthor preprint6.15 MBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.