On the Maintenance of System User Interactive Tests

Doctoral thesis (2021)

Many companies rely on software testing to verify that their software products meet their requirements. As such, software testing is a keystone of the quality process. It offers means to assess that both ... [more ▼]

Many companies rely on software testing to verify that their software products meet their requirements. As such, software testing is a keystone of the quality process. It offers means to assess that both functional and technical requirements are met. However, scaling the number of tests while preserving their quality poses challenges, just as in any software artifact. This is particularly true in the case of complex tests interacting with the SUT through its Graphical User Interface (GUI), i.e. System User Interactive Test (SUIT). The problem becomes challenging because as the System Under Test (SUT) evolves, the SUIT suites need to adapt and conform to the evolved software. Indeed, because the user interface is a part of the system that tends to undergo rapid evolutions, SUITs are particularly prompt to break. As the first contribution of this dissertation, we aim to demonstrate the problem of test maintenance and overall improve the understanding of SUIT scripts evolution. To that end, we identify, collect and analyze test code changes across the evolution of an industrial test suite. We show that the problem of test maintenance is largely due to test fragility (most commonly performed changes are due to locator and synchronization issues) and bad practices (over 30% of keywords are duplicated). To further investigate the question of bad test code practices such as test clones, we perform a multivocal study to identify which bad practices, i.e. SUIT smells, are already studied in both industry and academia. This process yields a catalog of 35 test code smells. For 16 of them, we derive metrics to analyze their diffusion across tests as well as potential refactoring actions removing the test code smells. Through an empirical study including both industrial and open-source test suites, we show that test code smells are largely present in SUITs, potentially contributing to SUIT fragility and hindering the maintenance process. Interestingly, refactoring actions remain rare during the lifespan of the tests. Yet, symptoms tend to disappear as old tests are discarded and new ones are introduced. However, during the analysis of SUIT smells, we observe that bad practices impacting locators do not appear often in the test code. This observation contrasts with the analysis of the evolution of the SUIT presented in our first empirical study. This apparent contradiction arises from the limitation of property-based locators which rely on the structure of the representation layer of the SUT. Indeed, such approaches are sensitive to internal iso-functional changes occurring during the normal evolution of the SUT. To account for this limitation, we introduce a new way of expressing locators, HPath. Instead of relying on the internal representation of the presentation layer, HPath relies on the rendered characteristics of the GUI. Our results suggest that despite what is presented in the literature on locators, expressions relying on a smaller number of GUI elements to fully qualify a target do not necessarily lead to more robust locators. On the contrary, the choice of the GUI element properties seems to play a stronger role in the robustness to change. Overall, this dissertation provides insights into how SUITs evolve and shows that SUIT fragility plays a major role in the associated maintenance effort. It also proposes techniques that effectively facilitate SUIT maintenance with the early detection of sub-optimal patterns and the introduction of a locator representation more robust to SUT evolution. [less ▲]

Pandemic Simulation and Forecasting of exit strategies:Convergence of Machine Learning and EpidemiologicalModels

Report (2020)

The COVID-19 pandemic has created a public health emergency unprecedented in this century. The lack ofaccurate knowledge regarding the outcomes of the virus has made it challenging for policymakers to ... [more ▼]

The COVID-19 pandemic has created a public health emergency unprecedented in this century. The lack ofaccurate knowledge regarding the outcomes of the virus has made it challenging for policymakers to decideon appropriate countermeasures to mitigate its impact on society, in particular the public health and the veryhealthcare system.While the mitigation strategies (including the lockdown) are getting lifted, understanding the current im-pacts of the outbreak remains challenging. This impedes any analysis and scheduling of measures requiredfor the different countries to recover from the pandemic without risking a new outbreak.Therefore, we propose a novel approach to build realistic data-driven pandemic simulation and forecastingmodels to support policymakers. Our models allow the investigation of mitigation/recovery measures andtheir impact. Thereby, they enable appropriate planning of those measures, with the aim to optimize theirsocietal benefits.Our approach relies on a combination of machine learning and classical epidemiological models, circum-venting the respective limitations of these techniques to allow a policy-making based on established knowl-edge, yet driven by factual data, and tailored to each country’s specific context. [less ▲]

On the Evolution of Keyword-Driven Test Suites

in 12th IEEE International Conference on Software Testing, Verification and Validation (2019)

Many companies rely on software testing to verify that their software products meet their requirements. However, test quality and, in particular, the quality of end-to-end testing is relatively hard to ... [more ▼]

Many companies rely on software testing to verify that their software products meet their requirements. However, test quality and, in particular, the quality of end-to-end testing is relatively hard to achieve. The problem becomes challenging when software evolves, as end-to-end test suites need to adapt and conform to the evolved software. Unfortunately, end-to-end tests are particularly fragile as any change in the application interface, e.g., application flow, location or name of graphical user interface elements, necessitates a change in the tests. This paper presents an industrial case study on the evolution of Keyword-Driven test suites, also known as Keyword-Driven Testing (KDT). Our aim is to demonstrate the problem of test maintenance, identify the benefits of Keyword-Driven Testing and overall improve the understanding of test code evolution (at the acceptance testing level). This information will support the development of automatic techniques, such as test refactoring and repair, and will motivate future research. To this end, we identify, collect and analyze test code changes across the evolution of industrial KDT test suites for a period of eight months. We show that the problem of test maintenance is largely due to test fragility (most commonly-performed changes are due to locator and synchronization issues) and test clones (over 30% of keywords are duplicated). We also show that the better test design of KDT test suites has the potential for drastically reducing (approximately 70%) the number of test code changes required to support software evolution. To further validate our results, we interview testers from BGL BNP Paribas and report their perceptions on the advantages and challenges of keyword-driven testing. [less ▲]

An Empirical Study on Vulnerability Prediction of Open-Source Software Releases

in Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (2019)