Article (Scientific journals)
Automatic Generation of Acceptance Test Cases from Use Case Specifications: an NLP-based Approach
WANG, Chunhui; PASTORE, Fabrizio; Göknil, Arda et al.
2022In IEEE Transactions on Software Engineering, 48 (2), p. 585 - 616
Peer reviewed
 

Files


Full Text
UMTG_TSE_2020.pdf
Author postprint (2.46 MB)
Download

0098-5589 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.


All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
System Test Case Generation; Use Case Specifications; Semantic Role Labeling; Natural Language Processing; Acceptance Testing
Abstract :
[en] Acceptance testing is a validation activity performed to ensure the conformance of software systems with respect to their functional requirements. In safety critical systems, it plays a crucial role since it is enforced by software standards, which mandate that each requirement be validated by such testing in a clearly traceable manner. Test engineers need to identify all the representative test execution scenarios from requirements, determine the runtime conditions that trigger these scenarios, and finally provide the input data that satisfy these conditions. Given that requirements specifications are typically large and often provided in natural language (e.g., use case specifications), the generation of acceptance test cases tends to be expensive and error-prone. In this paper, we present Use Case Modeling for System-level, Acceptance Tests Generation (UMTG), an approach that supports the generation of executable, system-level, acceptance test cases from requirements specifications in natural language, with the goal of reducing the manual effort required to generate test cases and ensuring requirements coverage. More specifically, UMTG automates the generation of acceptance test cases based on use case specifications and a domain model for the system under test, which are commonly produced in many development environments. Unlike existing approaches, it does not impose strong restrictions on the expressiveness of use case specifications. We rely on recent advances in natural language processing to automatically identify test scenarios and to generate formal constraints that capture conditions triggering the execution of the scenarios, thus enabling the generation of test data. In two industrial case studies, UMTG automatically and correctly translated 95% of the use case specification steps into formal constraints required for test data generation; furthermore, it generated test cases that exercise not only all the test scenarios manually implemented by experts, but also some critical scenarios not previously considered.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Software Verification and Validation Lab (SVV Lab)
Disciplines :
Computer science
Author, co-author :
WANG, Chunhui ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
PASTORE, Fabrizio  ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Göknil, Arda ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
BRIAND, Lionel ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
External co-authors :
no
Language :
English
Title :
Automatic Generation of Acceptance Test Cases from Use Case Specifications: an NLP-based Approach
Publication date :
01 February 2022
Journal title :
IEEE Transactions on Software Engineering
ISSN :
0098-5589
Publisher :
Institute of Electrical and Electronics Engineers, New York, United States - New York
Volume :
48
Issue :
2
Pages :
585 - 616
Peer reviewed :
Peer reviewed
Focus Area :
Security, Reliability and Trust
European Projects :
H2020 - 694277 - TUNE - Testing the Untestable: Model Testing of Complex Software-Intensive Systems
Funders :
CE - Commission Européenne
European Union
Available on ORBilu :
since 02 June 2020

Statistics


Number of views
328 (59 by Unilu)
Number of downloads
469 (26 by Unilu)

Scopus citations®
 
38
Scopus citations®
without self-citations
38
OpenAlex citations
 
51
WoS citations
 
26

Bibliography


Similar publications



Contact ORBilu