Reference : On the Evolution of Keyword-Driven Test Suites
Scientific congresses, symposiums and conference proceedings : Paper published in a book
Engineering, computing & technology : Computer science
Security, Reliability and Trust
http://hdl.handle.net/10993/38894
On the Evolution of Keyword-Driven Test Suites
English
Rwemalika, Renaud mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Kintis, Marinos mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Papadakis, Mike mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Computer Science and Communications Research Unit (CSC) >]
Le Traon, Yves mailto [University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC) >]
Lorrach, Pierre mailto []
2019
12th IEEE International Conference on Software Testing, Verification and Validation
Yes
International
12th IEEE International Conference on Software Testing, Verification and Validation
from 22-04-2019 to 27-04-2019
Xi'an
China
[en] keyword-driven testing ; acceptance testing ; end- to-end testing ; test code evolution ; test clones
[en] Many companies rely on software testing to verify that their software products meet their requirements. However, test quality and, in particular, the quality of end-to-end testing is relatively hard to achieve. The problem becomes challenging when software evolves, as end-to-end test suites need to adapt and conform to the evolved software. Unfortunately, end-to-end tests are particularly fragile as any change in the application interface, e.g., application flow, location or name of graphical user interface elements, necessitates a change in the tests. This paper presents an industrial case study on the evolution of Keyword-Driven test suites, also known as Keyword-Driven Testing (KDT). Our aim is to demonstrate the problem of test maintenance, identify the benefits of Keyword-Driven Testing and overall improve the understanding of test code evolution (at the acceptance testing level). This information will support the development of automatic techniques, such as test refactoring and repair, and will motivate future research. To this end, we identify, collect and analyze test code changes across the evolution of industrial KDT test suites for a period of eight months. We show that the problem of test maintenance is largely due to test fragility (most commonly-performed changes are due to locator and synchronization issues) and test clones (over 30% of keywords are duplicated). We also show that the better test design of KDT test suites has the potential for drastically reducing (approximately 70%) the number of test code changes required to support software evolution. To further validate our results, we interview testers from BGL BNP Paribas and report their perceptions on the advantages and challenges of keyword-driven testing.
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Security Design and Validation Research Group (SerVal)
Luxembourg National Research Fund ; Alphonse Weicker Foundation
Researchers ; Professionals
http://hdl.handle.net/10993/38894

File(s) associated to this reference

Fulltext file(s):

FileCommentaryVersionSizeAccess
Open access
icst2019.pdfPublisher postprint441.9 kBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.