References of "Garousi, Vahid 50026857"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailWhat we know about testing embedded software
Garousi, Vahid UL; Felderer, Michael; Karapıçak, Çağrı Murat et al

in IEEE Software (in press)

To cost-effectively test embedded software, practitioners and researchers have proposed many test techniques, approaches, tools, and frameworks. However, obtaining an overview of the state of the art and ... [more ▼]

To cost-effectively test embedded software, practitioners and researchers have proposed many test techniques, approaches, tools, and frameworks. However, obtaining an overview of the state of the art and state of the practice in this area is challenging for practitioners or new researchers. In addition, owing to an inadequate overview of what already exists in this area, some companies often reinvent the wheel by designing a test approach that’s new to them but already exists. To address these problems, the authors conducted a systematic literature review of this area that covered the testing topics, testing activities, test artifacts, and industries on which the studies focused. The results can benefit both practitioners and researchers by serving as an index to the vast body of knowledge in this important, fast-growing area. [less ▲]

Detailed reference viewed: 41 (1 UL)
Full Text
Peer Reviewed
See detailMulti-objective regression test selection in practice: an empirical study in the defense software industry
Özkan, Ramazan; Garousi, Vahid UL; Betin-Can, Aysu

in Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) (in press)

Executing an entire regression test suite after every code change is often costly in large software projects. To address this challenge, various regression test selection techniques have been proposed in ... [more ▼]

Executing an entire regression test suite after every code change is often costly in large software projects. To address this challenge, various regression test selection techniques have been proposed in the literature. One of those approaches is the Multi-Objective Regression Test Optimization (MORTO) approach, which is applied when there are multiple objectives during regression testing (e.g., minimizing the number of test cases and maximizing test coverage). This paper reports an action research type of study which was conducted to improve regression test-selection practices of a safety-critical industrial software in the defence domain based on the MORTO approach. The problem is formulated and solved by converting the multi-objective genetic-algorithm (GA) problem into a custom-built scalarized single-objective GA. The empirical results demonstrate that this approach yields a more efficient test suite (in terms of testing cost and benefits) compared to the old (manual) test-selection approach and another approach from the literature, i.e., the selective requirement coverage-based approach. Based on the context and need of the company, we developed the GA-based solution on a set of five cost objectives and four benefit objectives for regression testing while providing full coverage of the affected (changed) requirements. Since our proposed approach has been beneficial in the industrial context, it is currently in active use in the company. [less ▲]

Detailed reference viewed: 16 (1 UL)
Full Text
Peer Reviewed
See detailIntroducing automated GUI testing and observing its benefits: an industrial case study in the context of law-practice management software
Garousi, Vahid UL; Yıldırım, Erdem

in Proceedings of ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) (in press)

Motivated by a real-world industrial need in the context of a large IT solutions company based in Turkey, the authors and their colleagues developed and introduced automated test suites for GUI testing of ... [more ▼]

Motivated by a real-world industrial need in the context of a large IT solutions company based in Turkey, the authors and their colleagues developed and introduced automated test suites for GUI testing of two large-scale law-practice management software (comprising of 414 and 105 KLOC). We report in this paper our experience in developing and introducing a set of large automated test suites (more than 50 KLOC in total), using best practices in state-of-the art and –practice, and to report its observed benefits by conducting cost-benefit analysis in the specific industrial context. The project was conducted based on the principles of case-study and “action research” in which the real industrial needs drove the research. Among the best practices that we used are the followings: (1) the page-object test pattern, (2) modularity in test code, (3) creating test-specific libraries, and (4) using systematic guidelines to decide when and what (test cases) to automate. To assess the cost-benefit and Return On Investment (ROI) of test automation, we followed a hybrid measurement approach to assess both the quantitative and qualitative (intangible) benefits of test automation. The empirical findings showed that the automated GUI testing approach has indeed benefitted the test and QA team in the company under study and automation has been highly welcome by the test engineers. By serving as a success story and experience report in development and introduction of automated test suites in an industrial setting, this paper adds to the body of evidence in this area and it aims at sharing both technical (e.g., using automated test patterns) and process aspects (e.g., test process improvement) of our project with other practitioners and researchers with the hope of encouraging more industry-academia collaborations in test automation. [less ▲]

Detailed reference viewed: 17 (3 UL)
Full Text
Peer Reviewed
See detailComparing automated visual GUI testing tools: an industrial case study
Garousi, Vahid UL; Afzal, Wasif; Çağlar, Adem et al

in Proceedings of International Workshop on Automated Software Testing (A-TEST) (in press)

Visual GUI testing (VGT) is a tool-driven technique, which uses image recognition for interaction and assertion of the behaviour of system under test. Motivated by a real industrial need, in the context ... [more ▼]

Visual GUI testing (VGT) is a tool-driven technique, which uses image recognition for interaction and assertion of the behaviour of system under test. Motivated by a real industrial need, in the context of a large Turkish software and systems company providing solutions in the areas of defense and IT sector, we systematically planned and applied a VGT project in this industrial context. The goal of the initial phase of the project was to empirically evaluate two well-known VGT tools (Sikuli and JAutomate) to help the company select the best tool for a given testing project. Our results show that both two tools suffer from similar test ‘Replay’ problems such as the inability to find smaller-sized images. The repeatability of test executions was better for JAutomate in case of one of the two software under test (SUT) while it was comparable for the other. In terms of test development effort, for both tools, there were high correlations with number of steps in test suites, however the effort is reduced if test code is reused. The study has already provided benefits to the test engineers and managers in the company by increasing the know-how in the company w.r.t. VGT, and by identifying the challenges and their workarounds in using the tools. The industrial case study in this paper intends to add to the body of evidence in VGT and help other researchers and practitioners. [less ▲]

Detailed reference viewed: 29 (3 UL)
Full Text
Peer Reviewed
See detailProceedings of International Conference on Software and Systems Process (ICSSP)
Kuhrmann, Marco; Diebold, Philipp; Münch, Jürgen et al

in Hybrid software and system development in practice: Waterfall, scrum, and beyond (2017, July)

Software and system development faces numerous challenges of rapidly changing markets. To address such challenges, companies and projects design and adopt specific development approaches by combining well ... [more ▼]

Software and system development faces numerous challenges of rapidly changing markets. To address such challenges, companies and projects design and adopt specific development approaches by combining well-structured comprehensive methods and flexible agile practices. Yet, the number of methods and practices is large, and available studies argue that the actual process composition is carried out in a fairly ad-hoc manner. The present paper reports on a survey on hybrid software development approaches. We study which approaches are used in practice, how different approaches are combined, and what contextual factors influence the use and combination of hybrid software development approaches. Our results from 69 study participants show a variety of development approaches used and combined in practice. We show that most combinations follow a pattern in which a traditional process model serves as framework in which several fine-grained (agile) practices are plugged in. We further show that hybrid software development approaches are independent from the company size and external triggers. We conclude that such approaches are the results of a natural process evolution, which is mainly driven by experience, learning, and pragmatism. [less ▲]

Detailed reference viewed: 35 (11 UL)
Full Text
Peer Reviewed
See detailChoosing the Right Test Automation Tool: a Grey Literature Review of Practitioner Sources
Raulamo-Jurvanen, Päivi; Mäntylä, Mika; Garousi, Vahid UL

in Choosing the Right Test Automation Tool: a Grey Literature Review of Practitioner Sources (2017, June 15)

Background: Choosing the right software test automation tool is not trivial, and recent industrial surveys indicate lack of right tools as the main obstacle to test automation. Aim: In this paper, we ... [more ▼]

Background: Choosing the right software test automation tool is not trivial, and recent industrial surveys indicate lack of right tools as the main obstacle to test automation. Aim: In this paper, we study how practitioners tackle the problem of choosing the right test automation tool. Method: We synthesize the “voice” of the practitioners with a grey literature review originating from 53 different companies. The industry experts behind the sources had roles such as “Software Test Automation Architect”, and “Principal Software Engineer”. Results: Common consensus about the important criteria exists but those are not applied systematically. We summarize the scattered steps from individual sources by presenting a comprehensive process for tool evaluation with 12 steps and a total of 14 different criteria for choosing the right tool. Conclusions: The practitioners tend to have general interest in and be influenced by related grey literature as about 78% of our sources had at least 20 backlinks (a reference comparable to a citation) while the variation was between 3 and 759 backlinks. There is a plethora of different software testing tools available, yet the practitioners seem to prefer and adopt the widely known and used tools. The study helps to identify the potential pitfalls of existing processes and opportunities for comprehensive tool evaluation. [less ▲]

Detailed reference viewed: 52 (4 UL)
Full Text
Peer Reviewed
See detailIndustry-academia collaborations in software engineering: an empirical analysis of challenges, patterns and anti-patterns in research projects
Garousi, Vahid UL; Felderer, Michael; Fernandes, João et al

in Proceedings of International Conference on Evaluation and Assessment in Software Engineering (EASE) (2017, June 15)

Research collaboration between industry and academia supports improvement and innovation in industry and helps to ensure industrial relevance in academic research. However, many researchers and ... [more ▼]

Research collaboration between industry and academia supports improvement and innovation in industry and helps to ensure industrial relevance in academic research. However, many researchers and practitioners believe that the level of joint industry-academia collaboration (IAC) in software engineering (SE) research is still relatively low, compared to the amount of activity in each of the two communities. The goal of the empirical study reported in this paper is to exploratory characterize the state of IAC with respect to a set of challenges, patterns and anti-patterns identified by a recent Systematic Literature Review study. To address the above goal, we gathered the opinions of researchers and practitioners w.r.t. their experiences in IAC projects. Our dataset includes 47 opinion data points related to a large set of projects conducted in 10 different countries. We aim to contribute to the body of evidence in the area of IAC, for the benefit of researchers and practitioners in conducting future successful IAC projects in SE. As an output, the study presents a set of empirical findings and evidence-based recommendations to increase the success of IAC projects. [less ▲]

Detailed reference viewed: 16 (0 UL)
Full Text
Peer Reviewed
See detailWhat industry wants from academia in software testing? Hearing practitioners’ opinions
Garousi, Vahid UL; Felderer, Michael; Kuhrmann, Marco et al

in Proceedings of International Conference on Evaluation and Assessment in Software Engineering (EASE) (2017, June 15)

The level of industry-academia collaboration (IAC) in software engineering in general and in software testing in particular is quite low. Many researchers and practitioners are not collaborating with the ... [more ▼]

The level of industry-academia collaboration (IAC) in software engineering in general and in software testing in particular is quite low. Many researchers and practitioners are not collaborating with the “other side” to solve industrial problems. To shed light on the above issue and to characterize precisely what industry wants from academia in software testing, we solicited practitioners’ opinions on their challenges in different testing activities and also the particularly relevant topics that they want the research community to work on. This short paper aims to draw the community’s attention to the important issue of strengthening IAC with the hope of more IAC in software testing in the areas of most importance to the industry. [less ▲]

Detailed reference viewed: 17 (3 UL)
Full Text
Peer Reviewed
See detailExperience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering
Garousi, Vahid UL; Felderer, Michael

in Proceedings of International Conference on Evaluation and Assessment in Software Engineering (EASE) (2017, June)

To systematically collect evidence and to structure a given area in software engineering (SE), Systematic Literature Reviews (SLR) and Systematic Mapping (SM) studies have become common. Data extraction ... [more ▼]

To systematically collect evidence and to structure a given area in software engineering (SE), Systematic Literature Reviews (SLR) and Systematic Mapping (SM) studies have become common. Data extraction is one of the main phases (activities) when conducting an SM or an SLR, whose objective is to extract required data from the primary studies and to accurately record the information researchers need to answer the questions of the SM/SLR study. Based on experience in a large number of SM/SLR studies, we and many other researchers have found the data extraction in SLRs to be time consuming and error-prone, thus raising the real need for heuristics and guidelines for effective and efficient data extraction in these studies, especially to be learnt by junior and young researchers. As a ‘guideline’ paper, this paper contributes a synthesized list of challenges usually faced during SLRs’ data extraction phase and the corresponding solutions (guidelines). For our synthesis, we consider two data sources: (1) the pool of 16 SLR studies in which the authors have been involved in, as well as (2) a review of challenges and guidelines in the existing literature. Our experience in utilizing the presented guidelines in the near past have helped our junior colleagues to conduct data extractions more effectively and efficiently. [less ▲]

Detailed reference viewed: 40 (7 UL)
Full Text
Peer Reviewed
See detailCross-factor analysis of software modeling practices versus practitioner demographics in the embedded software industry
Akdur, Deniz; Garousi, Vahid UL; Demirörs, Onur

in 6th Mediterranean Conference on Embedded Computing (MECO) (2017)

Software-intensive embedded systems have evolved to be essential tools of our lives. To cope with growing complexities in embedded software industry, software modeling and model-driven engineering (MDE ... [more ▼]

Software-intensive embedded systems have evolved to be essential tools of our lives. To cope with growing complexities in embedded software industry, software modeling and model-driven engineering (MDE) have become popular for design, development and testing of these systems. However, the usage of models in embedded software industry and the relevant practices usually vary since challenges, requirements and purposes differ among systems as well as among sectors. To understand the state-of-the-practice of software modeling in embedded systems industry, we designed and conducted a world-wide survey, which accepted 627 responses from 27 different countries. The goal of this paper is to present our work built on this survey to better understand and characterize the cross-factor analysis of software modeling versus practitioner demographics, e.g., degree of using software modeling versus target sectors of the products. [less ▲]

Detailed reference viewed: 82 (20 UL)