Reference : Comparing automated visual GUI testing tools: an industrial case study
Scientific congresses, symposiums and conference proceedings : Paper published in a book
Engineering, computing & technology : Computer science
Comparing automated visual GUI testing tools: an industrial case study
Garousi, Vahid mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Afzal, Wasif mailto []
Çağlar, Adem mailto []
Işık, İhsan Berk mailto []
Baydan, Berker mailto []
Çaylak, Seçkin mailto []
Boyraz, Ahmet Zeki mailto []
Yolaçan, Burak mailto []
Herkiloğlu, Kadir mailto []
In press
Proceedings of International Workshop on Automated Software Testing (A-TEST)
International Workshop on Automated Software Testing (A-TEST)
September 2017
[en] Graphical User Interface (GUI) Testing ; Visual GUI Testing ; Automated Testing Tools ; Empirical Evaluation ; Industrial Case Study
[en] Visual GUI testing (VGT) is a tool-driven technique, which uses image recognition for interaction and assertion of the behaviour of system under test. Motivated by a real industrial need, in the context of a large Turkish software and systems company providing solutions in the areas of defense and IT sector, we systematically planned and applied a VGT project in this industrial context. The goal of the initial phase of the project was to empirically evaluate two well-known VGT tools (Sikuli and JAutomate) to help the company select the best tool for a given testing project. Our results show that both two tools suffer from similar test ‘Replay’ problems such as the inability to find smaller-sized images. The repeatability of test executions was better for JAutomate in case of one of the two software under test (SUT) while it was comparable for the other. In terms of test development effort, for both tools, there were high correlations with number of steps in test suites, however the effort is reduced if test code is reused. The study has already provided benefits to the test engineers and managers in the company by increasing the know-how in the company w.r.t. VGT, and by identifying the challenges and their workarounds in using the tools. The industrial case study in this paper intends to add to the body of evidence in VGT and help other researchers and practitioners.
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Software Verification and Validation Lab (SVV Lab)

File(s) associated to this reference

Fulltext file(s):

Limited access
ATEST-Comparing tools-camera ready-Latex-for web.pdfAuthor preprint936.99 kBRequest a copy

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.