Results 1-20 of 26.
fabrizio pastore

Bookmark and Share    
Full Text
Peer Reviewed
See detailSupporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learning
Fahmy, Hazem UL; Pastore, Fabrizio UL; Bagherzadeh, Mojtaba et al

in IEEE Transactions on Reliability (in press)

Deep neural networks (DNNs) are increasingly im- portant in safety-critical systems, for example in their perception layer to analyze images. Unfortunately, there is a lack of methods to ensure the ... [more ▼]

Deep neural networks (DNNs) are increasingly im- portant in safety-critical systems, for example in their perception layer to analyze images. Unfortunately, there is a lack of methods to ensure the functional safety of DNN-based components. We observe three major challenges with existing practices regarding DNNs in safety-critical systems: (1) scenarios that are underrepresented in the test set may lead to serious safety violation risks, but may, however, remain unnoticed; (2) char- acterizing such high-risk scenarios is critical for safety analysis; (3) retraining DNNs to address these risks is poorly supported when causes of violations are difficult to determine. To address these problems in the context of DNNs analyzing images, we propose HUDD, an approach that automatically supports the identification of root causes for DNN errors. HUDD identifies root causes by applying a clustering algorithm to heatmaps capturing the relevance of every DNN neuron on the DNN outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters. We evaluated HUDD with DNNs from the automotive domain. HUDD was able to identify all the distinct root causes of DNN errors, thus supporting safety analysis. Also, our retraining approach has shown to be more effective at improving DNN accuracy than existing approaches. [less ▲]

Detailed reference viewed: 34 (1 UL)
Full Text
Peer Reviewed
See detailAutomatic Test Suite Generation for Key-points Detection DNNs Using Many-Objective Search
Ul Haq, Fitash UL; Shin, Donghwan UL; Briand, Lionel UL et al

in 2021 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) (in press)

Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver's gaze detection and drowsiness ... [more ▼]

Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver's gaze detection and drowsiness detection in automated driving systems. With the recent advances of Deep Neural Networks (DNNs), Key-Points detection DNNs (KP-DNNs) have been increasingly employed for that purpose. Nevertheless, KP-DNN testing and validation have remained a challenging problem because KP-DNNs predict many independent key-points at the same time---where each individual key-point may be critical in the targeted application---and images can vary a great deal according to many factors. In this paper, we present an approach to automatically generate test data for KP-DNNs using many-objective search. In our experiments, focused on facial key-points detection DNNs developed for an industrial automotive application, we show that our approach can generate test suites to severely mispredict, on average, more than 93% of all key-points. In comparison, random search-based test data generation can only severely mispredict 41% of them. Many of these mispredictions, however, are not avoidable and should not therefore be considered failures. We also empirically compare state-of-the-art, many-objective search algorithms and their variants, tailored for test suite generation. Furthermore, we investigate and demonstrate how to learn specific conditions, based on image characteristics (e.g., head posture and skin color), that lead to severe mispredictions. Such conditions serve as a basis for risk analysis or DNN retraining. [less ▲]

Detailed reference viewed: 21 (3 UL)
Full Text
Peer Reviewed
See detailCan Offline Testing of Deep Neural Networks Replace Their Online Testing?
Ul Haq, Fitash UL; Shin, Donghwan UL; Nejati, Shiva UL et al

in Empirical Software Engineering (in press)

We distinguish two general modes of testing for Deep Neural Networks (DNNs): Offline testing where DNNs are tested as individual units based on test datasets obtained without involving the DNNs under test ... [more ▼]

We distinguish two general modes of testing for Deep Neural Networks (DNNs): Offline testing where DNNs are tested as individual units based on test datasets obtained without involving the DNNs under test, and online testing where DNNs are embedded into a specific application environment and tested in a closed-loop mode in interaction with the application environment. Typically, DNNs are subjected to both types of testing during their development life cycle where offline testing is applied immediately after DNN training and online testing follows after offline testing and once a DNN is deployed within a specific application environment. In this paper, we study the relationship between offline and online testing. Our goal is to determine how offline testing and online testing differ or complement one another and if offline testing results can be used to help reduce the cost of online testing? Though these questions are generally relevant to all autonomous systems, we study them in the context of automated driving systems where, as study subjects, we use DNNs automating end-to-end controls of steering functions of self-driving vehicles. Our results show that offline testing is less effective than online testing as many safety violations identified by online testing could not be identified by offline testing, while large prediction errors generated by offline testing always led to severe safety violations detectable by online testing. Further, we cannot exploit offline testing results to reduce the cost of online testing in practice since we are not able to identify specific situations where offline testing could be as accurate as online testing in identifying safety requirement violations. [less ▲]

Detailed reference viewed: 25 (6 UL)
Full Text
Peer Reviewed
See detailTkT: Automatic Inference of Timed and Extended Pushdown Automata
Pastore, Fabrizio UL; Micucci, Daniela; Guzman, Michell et al

in IEEE Transactions on Software Engineering (2020)

To mitigate the cost of manually producing and maintaining models capturing software specifications, specification mining techniques can be exploited to automatically derive up-to-date models that ... [more ▼]

To mitigate the cost of manually producing and maintaining models capturing software specifications, specification mining techniques can be exploited to automatically derive up-to-date models that faithfully represent the behavior of software systems. So far, specification mining solutions focused on extracting information about the functional behavior of the system, especially in the form of models that represent the ordering of the operations. Well-known examples are finite state models capturing the usage protocol of software interfaces and temporal rules specifying relations among system events. Although the functional behavior of a software system is a primary aspect of concern, there are several other non-functional characteristics that must be typically addressed jointly with the functional behavior of a software system. Efficiency is one of the most relevant characteristics. In fact, an application delivering the right functionalities inefficiently has a big chance to not satisfy the expectation of its users. Interestingly, the timing behavior is strongly dependent on the functional behavior of a software system. For instance, the timing of an operation depends on the functional complexity and size of the computation that is performed. Consequently, models that combine the functional and timing behaviors, as well as their dependencies, are extremely important to precisely reason on the behavior of software systems. In this paper, we address the challenge of generating models that capture both the functional and timing behavior of a software system from execution traces. The result is the Timed k-Tail (TkT) specification mining technique, which can mine finite state models that capture such an interplay: the functional behavior is represented by the possible order of the events accepted by the transitions, while the timing behavior is represented through clocks and clock constraints of different nature associated with transitions. Our empirical evaluation with several libraries and applications show that TkT can generate accurate models, capable of supporting the identification of timing anomalies due to overloaded environment and performance faults. Furthermore, our study shows that TkT outperforms state-of-the-art techniques in terms of scalability and accuracy of the mined models. [less ▲]

Detailed reference viewed: 66 (4 UL)
Full Text
Peer Reviewed
See detailAutomatic Generation of Acceptance Test Cases from Use Case Specifications: an NLP-based Approach
Wang, Chunhui UL; Pastore, Fabrizio UL; Göknil, Arda UL et al

in IEEE Transactions on Software Engineering (2020)

Acceptance testing is a validation activity performed to ensure the conformance of software systems with respect to their functional requirements. In safety critical systems, it plays a crucial role since ... [more ▼]

Acceptance testing is a validation activity performed to ensure the conformance of software systems with respect to their functional requirements. In safety critical systems, it plays a crucial role since it is enforced by software standards, which mandate that each requirement be validated by such testing in a clearly traceable manner. Test engineers need to identify all the representative test execution scenarios from requirements, determine the runtime conditions that trigger these scenarios, and finally provide the input data that satisfy these conditions. Given that requirements specifications are typically large and often provided in natural language (e.g., use case specifications), the generation of acceptance test cases tends to be expensive and error-prone. In this paper, we present Use Case Modeling for System-level, Acceptance Tests Generation (UMTG), an approach that supports the generation of executable, system-level, acceptance test cases from requirements specifications in natural language, with the goal of reducing the manual effort required to generate test cases and ensuring requirements coverage. More specifically, UMTG automates the generation of acceptance test cases based on use case specifications and a domain model for the system under test, which are commonly produced in many development environments. Unlike existing approaches, it does not impose strong restrictions on the expressiveness of use case specifications. We rely on recent advances in natural language processing to automatically identify test scenarios and to generate formal constraints that capture conditions triggering the execution of the scenarios, thus enabling the generation of test data. In two industrial case studies, UMTG automatically and correctly translated 95% of the use case specification steps into formal constraints required for test data generation; furthermore, it generated test cases that exercise not only all the test scenarios manually implemented by experts, but also some critical scenarios not previously considered. [less ▲]

Detailed reference viewed: 108 (18 UL)
Full Text
Peer Reviewed
See detailMetamorphic Security Testing for Web Systems
Mai, Xuan Phu UL; Pastore, Fabrizio UL; Goknil, Arda et al

in IEEE International Conference on Software Testing, Verification and Validation (ICST) 2020 (2020, March)

Security testing verifies that the data and the resources of software systems are protected from attackers. Unfortunately, it suffers from the oracle problem, which refers to the challenge, given an input ... [more ▼]

Security testing verifies that the data and the resources of software systems are protected from attackers. Unfortunately, it suffers from the oracle problem, which refers to the challenge, given an input for a system, of distinguishing correct from incorrect behavior. In many situations where potential vulnerabilities are tested, a test oracle may not exist, or it might be impractical due to the many inputs for which specific oracles have to be defined. In this paper, we propose a metamorphic testing approach that alleviates the oracle problem in security testing. It enables engineers to specify metamorphic relations (MRs) that capture security properties of the system. Such MRs are then used to automate testing and detect vulnerabilities. We provide a catalog of 22 system-agnostic MRs to automate security testing in Web systems. Our approach targets 39% of the OWASP security testing activities not automated by state-of-the-art techniques. It automatically detected 10 out of 12 vulnerabilities affecting two widely used systems, one commercial and the other open source (Jenkins). [less ▲]

Detailed reference viewed: 180 (25 UL)
Full Text
Peer Reviewed
See detailSMRL: A Metamorphic Security Testing Tool for Web Systems
Mai, Xuan Phu UL; Göknil, Arda; Pastore, Fabrizio UL et al

in 2020 IEEE/ACM 42nd International Conference on Software Engineering (2020)

We present a metamorphic testing tool that alleviates the oracle problem in security testing. The tool enables engineers to specify metamorphic relations that capture security properties of Web systems ... [more ▼]

We present a metamorphic testing tool that alleviates the oracle problem in security testing. The tool enables engineers to specify metamorphic relations that capture security properties of Web systems. It automatically tests Web systems to detect vulnerabilities based on those relations. We provide a domain-specific language accompanied by an Eclipse editor to facilitate the specification of metamorphic relations. The tool automatically collects the input data and transforms the metamorphic relations into executable Java code in order to automatically perform security testing based on the collected data. The tool has been successfully evaluated on a commercial system and a leading open source system (Jenkins). Demo video: https://youtu.be/9kx6u9LsGxs. [less ▲]

Detailed reference viewed: 163 (22 UL)
Full Text
Peer Reviewed
See detailCPSDebug: a tool for explanation of failures in cyber-physical systems
Bartocci, Ezio; Manjunath, Niveditha; Mariani, Leonardo et al

in CPSDebug: a tool for explanation of failures in cyber-physical systems (2020)

Debugging Cyber-Physical System models is often challenging, as it requires identifying a potentially long, complex and heterogenous combination of events that resulted in a violation of the expected ... [more ▼]

Debugging Cyber-Physical System models is often challenging, as it requires identifying a potentially long, complex and heterogenous combination of events that resulted in a violation of the expected behavior of the system. In this paper we present CPSDebug, a tool for supporting designers in the debugging of failures in MAT- LAB Simulink/Stateflow models. CPSDebug implements a gray-box approach that combines testing, specification mining, and failure analysis to identify the causes of failures and explain their propagation in time and space. The evaluation of the tool, based on multiple usage scenarios and faults and direct feedback from engineers, shows that CPSDebug can effectively aid engineers during debugging tasks. [less ▲]

Detailed reference viewed: 140 (1 UL)
Full Text
Peer Reviewed
See detailAutomating System Test Case Classification and Prioritization for Use Case-Driven Testing in Product Lines
Hajri, Ines UL; Göknil, Arda UL; Pastore, Fabrizio UL et al

in Empirical Software Engineering (2020), 25(5), 37113769

Product Line Engineering (PLE) is a crucial practice in many software development environments where software systems are complex and developed for multiple customers with varying needs. At the same time ... [more ▼]

Product Line Engineering (PLE) is a crucial practice in many software development environments where software systems are complex and developed for multiple customers with varying needs. At the same time, many development processes are use case-driven and this strongly influences their requirements engineering and system testing practices. In this paper, we propose, apply, and assess an automated system test case classification and prioritization approach specifically targeting system testing in the context of use case-driven development of product families. Our approach provides: (i) automated support to classify, for a new product in a product family, relevant and valid system test cases associated with previous products, and (ii) automated prioritization of system test cases using multiple risk factors such as fault-proneness of requirements and requirements volatility in a product family. Our evaluation was performed in the context of an industrial product family in the automotive domain. Results provide empirical evidence that we propose a practical and beneficial way to classify and prioritize system test cases for industrial product lines. [less ▲]

Detailed reference viewed: 171 (14 UL)
Full Text
Peer Reviewed
See detailBridging the Gap between Requirements Modeling and Behavior-driven Development
Alferez, Mauricio UL; Pastore, Fabrizio UL; Sabetzadeh, Mehrdad UL et al

in Proceedings of 22nd IEEE / ACM International Conference on Model Driven Engineering Languages and Systems (MODELS) (2019, September)

Acceptance criteria (AC) are implementation agnostic conditions that a system must meet to be consistent with its requirements and be accepted by its stakeholders. Each acceptance criterion is typically ... [more ▼]

Acceptance criteria (AC) are implementation agnostic conditions that a system must meet to be consistent with its requirements and be accepted by its stakeholders. Each acceptance criterion is typically expressed as a natural-language statement with a clear pass or fail outcome. Writing AC is a tedious and error-prone activity, especially when the requirements specifications evolve and there are different analysts and testing teams involved. Analysts and testers must iterate multiple times to ensure that AC are understandable and feasible, and accurately address the most important requirements and workflows of the system being developed. In many cases, analysts express requirements through models, along with natural language, typically in some variant of the UML. AC must then be derived by developers and testers from such models. In this paper, we bridge the gap between requirements models and AC by providing a UML-based modeling methodology and an automated solution to generate AC. We target AC in the form of Behavioral Specifications in the context of Behavioral-Driven Development (BDD), a widely used agile practice in many application domains. More specially we target the well-known Gherkin language to express AC, which then can be used to generate executable test cases. We evaluate our modeling methodology and AC generation solution through an industrial case study in the financial domain. Our results suggest that (1) our methodology is feasible to apply in practice, and (2) the additional modeling effort required by our methodology is outweighed by the benefits the methodology brings in terms of automated and systematic AC generation and improved model precision. [less ▲]

Detailed reference viewed: 541 (121 UL)
Full Text
Peer Reviewed
See detailMCP: A Security Testing Tool Driven by Requirements
Mai, Xuan Phu UL; Pastore, Fabrizio UL; Goknil, Arda et al

in 2019 IEEE/ACM 41st International Conference on Software Engineering (2019, May)

We present MCP, a tool for automatically generating executable security test cases from misuse case specifications in natural language (i.e., use case specifications capturing the behavior of malicious ... [more ▼]

We present MCP, a tool for automatically generating executable security test cases from misuse case specifications in natural language (i.e., use case specifications capturing the behavior of malicious users). MCP relies on Natural Language Processing (NLP), a restricted form of misuse case specifications, and a test driver API implementing basic utility functions for security testing. NLP is used to identify the activities performed by the malicious user and the control flow of misuse case specifications. MCP matches the malicious user’s activities to the methods of the provided test driver API in order to generate executable security test cases that perform the activities described in the misuse case specifications. MCP has been successfully evaluated on an industrial case study. [less ▲]

Detailed reference viewed: 215 (32 UL)
Full Text
Peer Reviewed
See detailOracles for Testing Software Timeliness with Uncertainty
Wang, Chunhui UL; Pastore, Fabrizio UL; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (2019), 28(1),

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time ... [more ▼]

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time uncertainty leads to non-determinism. For example, time-triggered operations may either generate different valid outputs across different executions, or experience failures (e.g., results not being generated in the expected time window) that occur only occasionally over many executions. For these reasons, time uncertainty makes the generation of effective test oracles for timing requirements a challenging task. To address the above challenge, we propose STUIOS (Stochastic Testing with Unique Input Output Sequences), an approach for the automated generation of stochastic oracles that verify the capability of a software system to fulfill timing constraints in the presence of time uncertainty. Such stochastic oracles entail the statistical analysis of repeated test case executions based on test output probabilities predicted by means of statistical model checking. Results from two industrial case studies in the automotive domain demonstrate that this approach improves the fault detection effectiveness of tests suites derived from timed automata, compared to traditional approaches. [less ▲]

Detailed reference viewed: 370 (66 UL)
Full Text
Peer Reviewed
See detailSoftware Engineering Research and Industry: A Symbiotic Relationship to Foster Impact
Basili, Victor; Briand, Lionel UL; Bianculli, Domenico UL et al

in IEEE Software (2018), 35(5), 44-49

Software engineering is not only an increasingly challenging endeavor that goes beyond the intellectual capabilities of any single individual engineer, but is also an intensely human one. Tools and ... [more ▼]

Software engineering is not only an increasingly challenging endeavor that goes beyond the intellectual capabilities of any single individual engineer, but is also an intensely human one. Tools and methods to develop software are employed by engineers of varied backgrounds within a large variety of organizations and application domains. As a result, the variation in challenges and practices in system requirements, architecture, and quality assurance is staggering. Human, domain and organizational factors define the context within which software engineering methodologies and technologies are to be applied and therefore the context that research needs to account for, if it is to be impactful. This paper provides an assessment of the current challenges faced by software engineering research in achieving its potential, a description of the root causes of such challenges, and a proposal for the field to move forward and become more impactful through collaborative research and innovation between public research and industry. [less ▲]

Detailed reference viewed: 276 (39 UL)
Full Text
Peer Reviewed
See detailAutomated Generation of Constraints from Use Case Specifications to Support System Testing
Wang, Chunhui UL; Pastore, Fabrizio UL; Briand, Lionel UL

in Proceedings of the 11th IEEE Conference on Software Testing, Validation and Verification (2018)

System testing plays a crucial role in safety-critical domains, e.g., automotive, where system test cases are used to demonstrate the compliance of software with its functional and safety requirements ... [more ▼]

System testing plays a crucial role in safety-critical domains, e.g., automotive, where system test cases are used to demonstrate the compliance of software with its functional and safety requirements. Unfortunately, since requirements are typically written in natural language, significant engineering effort is required to derive test cases from requirements. In such a context, automated support for generating system test cases from requirements specifications written in natural language would be highly beneficial. Unfortunately, existing approaches have limited applicability. For example, some of them require that software engineers provide formal specifications that capture some of the software behavior described using natural language. The effort needed to define such specifications is usually a significant deterrent for software developers. This paper proposes an approach, OCLgen, which largely automates the generation of the additional formal specifications required by an existing test generation approach named UMTG. More specifically, OCLgen relies on semantic analysis techniques to automatically derive the pre- and post-conditions of the activities described in use case specifications. The generated conditions are used by UMTG to identify the test inputs that cover all the use case scenarios described in use case specifications. In practice, the proposed approach enables the automated generation of test cases from use case specifications while avoiding most of the additional modeling effort required by UMTG. Results from an industrial case study show that the approach can automatically and correctly generate more than 75% of the pre- and post-conditions characterizing the activities described in use case specifications. [less ▲]

Detailed reference viewed: 505 (68 UL)
Full Text
Peer Reviewed
See detailA Natural Language Programming Approach for Requirements-based Security Testing
Mai, Xuan Phu UL; Pastore, Fabrizio UL; Göknil, Arda UL et al

in 29th IEEE International Symposium on Software Reliability Engineering (ISSRE 2018) (2018)

To facilitate communication among stakeholders, software security requirements are typically written in natural language and capture both positive requirements (i.e., what the system is supposed to do to ... [more ▼]

To facilitate communication among stakeholders, software security requirements are typically written in natural language and capture both positive requirements (i.e., what the system is supposed to do to ensure security) and negative requirements (i.e., undesirable behavior undermining security). In this paper, we tackle the problem of automatically generat- ing executable security test cases from security requirements in natural language (NL). More precisely, since existing approaches for the generation of test cases from NL requirements verify only positive requirements, we focus on the problem of generating test cases from negative requirements. We propose, apply and assess Misuse Case Programming (MCP), an approach that automatically generates security test cases from misuse case specifications (i.e., use case specifications capturing the behavior of malicious users). MCP relies on natural language processing techniques to extract the concepts (e.g., inputs and activities) appearing in requirements specifications and generates executable test cases by matching the extracted concepts to the members of a provided test driver API. MCP has been evaluated in an industrial case study, which provides initial evidence of the feasibility and benefits of the approach. [less ▲]

Detailed reference viewed: 535 (53 UL)
Full Text
Peer Reviewed
See detailModeling Security and Privacy Requirements: a Use Case-Driven Approach
Mai, Xuan Phu UL; Göknil, Arda UL; Shar, Lwin Khin et al

in Information and Software Technology (2018), 100

Context: Modern internet-based services, ranging from food-delivery to home-caring, leverage the availability of multiple programmable devices to provide handy services tailored to end-user needs. These ... [more ▼]

Context: Modern internet-based services, ranging from food-delivery to home-caring, leverage the availability of multiple programmable devices to provide handy services tailored to end-user needs. These services are delivered through an ecosystem of device-specific software components and interfaces (e.g., mobile and wearable device applications). Since they often handle private information (e.g., location and health status), their security and privacy requirements are of crucial importance. Defining and analyzing those requirements is a significant challenge due to the multiple types of software components and devices integrated into software ecosystems. Each software component presents peculiarities that often depend on the context and the devices the component interact with, and that must be considered when dealing with security and privacy requirements. Objective: In this paper, we propose, apply, and assess a modeling method that supports the specification of security and privacy requirements in a structured and analyzable form. Our motivation is that, in many contexts, use cases are common practice for the elicitation of functional requirements and should also be adapted for describing security requirements. Method: We integrate an existing approach for modeling security and privacy requirements in terms of security threats, their mitigations, and their relations to use cases in a misuse case diagram. We introduce new security-related templates, i.e., a mitigation template and a misuse case template for specifying mitigation schemes and misuse case specifications in a structured and analyzable manner. Natural language processing can then be used to automatically report inconsistencies among artifacts and between the templates and specifications. Results: We successfully applied our approach to an industrial healthcare project and report lessons learned and results from structured interviews with engineers. Conclusion: Since our approach supports the precise specification and analysis of security threats, threat scenarios and their mitigations, it also supports decision making and the analysis of compliance to standards. [less ▲]

Detailed reference viewed: 311 (38 UL)
Full Text
Peer Reviewed
See detailThe Case for Context-Driven Software Engineering Research
Briand, Lionel UL; Bianculli, Domenico UL; Nejati, Shiva UL et al

in IEEE Software (2017), 34(5), 72-75

Detailed reference viewed: 372 (32 UL)
Full Text
See detailAutomated requirements-driven testing of embedded systems based on use case specifications and timed automata
Wang, Chunhui UL

Doctoral thesis (2017)

The complexity of embedded software in safety-critical domains, such as automotive and avionics, has significantly increased over the years. For most embedded systems, standards require system testing to ... [more ▼]

The complexity of embedded software in safety-critical domains, such as automotive and avionics, has significantly increased over the years. For most embedded systems, standards require system testing to explicitly demonstrate that the software meets its functional and safety requirements. In these domains, system test cases are often manually derived from functional requirements in natural language plus other design artefacts, like UML statecharts. The definition of system test cases is therefore time-consuming and error-prone, especially given the quickly rising complexity of embedded systems. The benefits of automatic test generation are widely acknowledged today but existing approaches often require behavioural models that tend to be complex and expensive to produce, and are thus often not part of development practice. The work proposed in this dissertation focusses on the automated generation of test cases for testing the compliance between software and its functional and timing requirements. This dissertation is inspired by contexts where functional and timing requirements are expressed by means of use case specifications and timing automata, respectively. This is the development context of our industrial partner, IEE, an automotive company located in Luxembourg, who provided the case study used to validate the approach and tool described in this dissertation. This dissertation presents five main contributions: (1) A set of guidelines for the definition of functional and timing requirements to enable the automated generation of system test cases. (2) A technique for the automated generation of functional test cases from requirements elicited in the form of use case specifications following a prescribed template and natural-language restrictions. (3) A technique that reuses the automatically generated functional test cases to generate timeliness test cases from minimal models of the timing requirements of the system. (4) A technique for the automated generation of oracles for non-deterministic systems whose specifications are expressed by means of timed automata. In the context of this dissertation, automated oracles for non-deterministic systems are necessary to evaluate the results of the generated timeliness test cases. (5) The evaluation of the applicability and effectiveness of the proposed guidelines and techniques on an industrial case study, a representative automotive embedded system developed by IEE. [less ▲]

Detailed reference viewed: 142 (29 UL)
Full Text
Peer Reviewed
See detailAugmenting Field Data for Testing Systems Subject to Incremental Requirements Changes
Di Nardo, Daniel; Pastore, Fabrizio; Briand, Lionel UL

in ACM Transactions on Software Engineering & Methodology (2017), 26(1), 1-40

Detailed reference viewed: 312 (70 UL)
Full Text
Peer Reviewed
See detailSystem Testing of Timing Requirements based on Use Cases and Timed Automata
Wang, Chunhui UL; Pastore, Fabrizio UL; Briand, Lionel UL

in 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017), Tokyo 13-18 March 2017 (2017, March 13)

In the context of use-case centric development and requirements-driven testing, this paper addresses the problem of automatically deriving system test cases to verify timing requirements. Inspired by ... [more ▼]

In the context of use-case centric development and requirements-driven testing, this paper addresses the problem of automatically deriving system test cases to verify timing requirements. Inspired by engineering practice in an automotive software development context, we rely on an analyzable form of use case specifications and augment such functional descriptions with timed automata, capturing timing requirements, following a methodology aiming at minimizing modeling overhead. We automate the generation of executable test cases using a test strategy based on maximizing test suite diversity and building over the UPPAAL model checker. Initial empirical results based on an industrial case study provide evidence of the effectiveness of the approach. [less ▲]

Detailed reference viewed: 467 (64 UL)