Paper published in a book (Scientific congresses, symposiums and conference proceedings)
Comparing automated visual GUI testing tools: an industrial case study
Garousi, Vahid; Afzal, Wasif; Çağlar, Adem et al.
2017In Proceedings of International Workshop on Automated Software Testing (A-TEST)
Peer reviewed
 

Files


Full Text
ATEST-Comparing tools-camera ready-Latex-for web.pdf
Author preprint (959.48 kB)
Request a copy

All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
Graphical User Interface (GUI) Testing; Visual GUI Testing; Automated Testing Tools; Empirical Evaluation; Industrial Case Study
Abstract :
[en] Visual GUI testing (VGT) is a tool-driven technique, which uses image recognition for interaction and assertion of the behaviour of system under test. Motivated by a real industrial need, in the context of a large Turkish software and systems company providing solutions in the areas of defense and IT sector, we systematically planned and applied a VGT project in this industrial context. The goal of the initial phase of the project was to empirically evaluate two well-known VGT tools (Sikuli and JAutomate) to help the company select the best tool for a given testing project. Our results show that both two tools suffer from similar test ‘Replay’ problems such as the inability to find smaller-sized images. The repeatability of test executions was better for JAutomate in case of one of the two software under test (SUT) while it was comparable for the other. In terms of test development effort, for both tools, there were high correlations with number of steps in test suites, however the effort is reduced if test code is reused. The study has already provided benefits to the test engineers and managers in the company by increasing the know-how in the company w.r.t. VGT, and by identifying the challenges and their workarounds in using the tools. The industrial case study in this paper intends to add to the body of evidence in VGT and help other researchers and practitioners.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Software Verification and Validation Lab (SVV Lab)
Disciplines :
Computer science
Author, co-author :
Garousi, Vahid ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Afzal, Wasif
Çağlar, Adem
Işık, İhsan Berk
Baydan, Berker
Çaylak, Seçkin
Boyraz, Ahmet Zeki
Yolaçan, Burak
Herkiloğlu, Kadir
External co-authors :
yes
Language :
English
Title :
Comparing automated visual GUI testing tools: an industrial case study
Publication date :
2017
Event name :
International Workshop on Automated Software Testing (A-TEST)
Event place :
Paderborn, Germany
Event date :
September 2017
Audience :
International
Main work title :
Proceedings of International Workshop on Automated Software Testing (A-TEST)
Peer reviewed :
Peer reviewed
Available on ORBilu :
since 04 August 2017

Statistics


Number of views
163 (9 by Unilu)
Number of downloads
2 (2 by Unilu)

Scopus citations®
 
19
Scopus citations®
without self-citations
13
OpenCitations
 
13

Bibliography


Similar publications



Contact ORBilu