Reference : You Cannot Fix What You Cannot Find! An Investigation of Fault Localization Bias in B...
Scientific congresses, symposiums and conference proceedings : Paper published in a book
Engineering, computing & technology : Computer science
Computational Sciences
http://hdl.handle.net/10993/38442
You Cannot Fix What You Cannot Find! An Investigation of Fault Localization Bias in Benchmarking Automated Program Repair Systems
English
Liu, Kui mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Koyuncu, Anil mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Bissyande, Tegawendé François D Assise mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Kim, Dongsun []
Klein, Jacques mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Computer Science and Communications Research Unit (CSC) >]
Le Traon, Yves mailto [University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC) >]
24-Apr-2019
The 12th IEEE International Conference on Software Testing, Verification and Validation (ICST-2019)
IEEE
Yes
Xi'an
China
The 12th IEEE International Conference on Software Testing, Verification and Validation
from 22-04-2019 to 27-04-2019
Xi'an
China
[en] Automated Program Repair ; Spectrum-based Fault Localization ; Benchmarking ; Empirical Assessment ; Bias
[en] Properly benchmarking Automated Program Repair (APR) systems should contribute to the development and adoption of the research outputs by practitioners. To that end, the research community must ensure that it reaches significant milestones by reliably comparing state-of-the-art tools for a better understanding of their strengths and weaknesses. In this work, we identify and investigate a practical bias caused by the fault localization (FL) step in a repair pipeline. We propose to highlight the different fault localization configurations used in the literature, and their impact on APR systems when applied to the Defects4J benchmark. Then, we explore the performance variations that can be achieved by "tweaking'' the FL step. Eventually, we expect to create a new momentum for (1) full disclosure of APR experimental procedures with respect to FL, (2) realistic expectations of repairing bugs in Defects4J, as well as (3) reliable performance comparison among the state-of-the-art APR systems, and against the baseline performance results of our thoroughly assessed kPAR repair tool. Our main findings include: (a) only a subset of Defects4J bugs can be currently localized by commonly-used FL techniques; (b) current practice of comparing state-of-the-art APR systems (i.e., counting the number of fixed bugs) is potentially misleading due to the bias of FL configurations; and (c) APR authors do not properly qualify their performance achievement with respect to the different tuning parameters implemented in APR systems.
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Security Design and Validation Research Group (SerVal)
Fonds National de la Recherche - FnR
Researchers ; Professionals ; Students ; General public
http://hdl.handle.net/10993/38442
FnR ; FNR10449467 > Tegawend� Fran�ois D'Assise Bissyand� > RECOMMEND > Automatic Bug Fix Recommendation: Improving Software Repair and Reducing Time-to-Fix Delays in Software Development Projects > 01/02/2016 > 31/01/2019 > 2015

File(s) associated to this reference

Fulltext file(s):

FileCommentaryVersionSizeAccess
Open access
icst-2019.pdfPublisher postprint904.33 kBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.