Automated, Requirements-based Security Testing of Web-oriented Software Systems

Doctoral thesis (2020)

Motivation and Context. Modern Internet-based services (e.g., home-banking, personal-training, healthcare) are delivered through Web-oriented software systems which run on multiple and different devices ... [more ▼]

Motivation and Context. Modern Internet-based services (e.g., home-banking, personal-training, healthcare) are delivered through Web-oriented software systems which run on multiple and different devices including computers, mobile devices, wearable devices, and smart TVs. They manage and exchange users’ personal data such as credit reports, locations, and health status. Therefore, the security of the system and its data are of crucial importance. Unfortunately, from security requirements elicitation to security testing, there are a number of challenges to be addressed to ensure the security of Web-oriented software systems. First, existing practices for capturing security requirements do not rely on templates that ensure the specification of requirements in a precise, structured, and unambiguous manner. Second, security testing is usually performed either manually or is only partially automated. Most of existing security testing automation approaches focus only on specific vulnerabilities (e.g., buffer overflow, code injection). In addition, they suffer from the oracle problem, i.e., they cannot determine that the software does not meet its security requirements, except when it leads to denial of service or crashes. For this reason, security test automation is usually partial and only addresses the generation of inputs and not the verification of outputs. Though, in principle, solutions for the automated verification of functional requirements might be adopted to automatically verify security requirements, a number of concerns remain to be addressed. First, there is a lack of studies that demonstrate their applicability, in the context of security testing. Second, the oracle problem remains an open problem in many aspects of software testing research, not only security testing. In the context of functional testing, metamorphic testing has shown to be a viable solution to address the oracle problem; however, it has never been studied in the context of security testing. Contributions. In this dissertation, we propose a set of approaches to address the above-mentioned challenges. (1) To model security requirements in a structured and analyzable manner, we propose a use case modeling approach that relies on a restricted natural language and a template already validated in the context of functional testing. It introduces the concepts of security use case specifications (i.e., what the system is supposed to do) and misuse case specifications (i.e., malicious user behaviours that the system is supposed to prevent). Moreover, we propose a template for capturing guidelines for the mitigation of security threats. (2) To verify that systems meet their security requirements, we propose an approach to automatically generate security test cases from misuse use case specifications. More precisely, we propose a natural language programming solution that automatically generates executable security test cases and test inputs from misuse case specifications in natural language. (3) To address the oracle problem, we propose a metamorphic testing solution for Web-oriented software systems. The solution relies on a predefined set of metamorphic relations that capture (a) how an attacker likely alters a valid input to exploit a vulnerable system and (b) how the output of the system should change as a result of the attack if the system meet its security requirements. Our solution relies on Web-crawlers to automatically identify the valid inputs to be used for testing. (4) We identify a set of testability guidelines to facilitate the adoption of the proposed approaches in software projects. The identified guidelines indicate (a) which types of vulnerabilities can be addressed through the solutions proposed in this dissertation and (b) which design solutions should be integrated into the system to enable effective test automation. [less ▲]

SMRL: A Metamorphic Security Testing Tool for Web Systems

in 2020 IEEE/ACM 42nd International Conference on Software Engineering (2020)

We present a metamorphic testing tool that alleviates the oracle problem in security testing. The tool enables engineers to specify metamorphic relations that capture security properties of Web systems ... [more ▼]

We present a metamorphic testing tool that alleviates the oracle problem in security testing. The tool enables engineers to specify metamorphic relations that capture security properties of Web systems. It automatically tests Web systems to detect vulnerabilities based on those relations. We provide a domain-specific language accompanied by an Eclipse editor to facilitate the specification of metamorphic relations. The tool automatically collects the input data and transforms the metamorphic relations into executable Java code in order to automatically perform security testing based on the collected data. The tool has been successfully evaluated on a commercial system and a leading open source system (Jenkins). Demo video: https://youtu.be/9kx6u9LsGxs. [less ▲]

A Natural Language Programming Approach for Requirements-based Security Testing

in 29th IEEE International Symposium on Software Reliability Engineering (ISSRE 2018) (2018)

To facilitate communication among stakeholders, software security requirements are typically written in natural language and capture both positive requirements (i.e., what the system is supposed to do to ... [more ▼]

To facilitate communication among stakeholders, software security requirements are typically written in natural language and capture both positive requirements (i.e., what the system is supposed to do to ensure security) and negative requirements (i.e., undesirable behavior undermining security). In this paper, we tackle the problem of automatically generat- ing executable security test cases from security requirements in natural language (NL). More precisely, since existing approaches for the generation of test cases from NL requirements verify only positive requirements, we focus on the problem of generating test cases from negative requirements. We propose, apply and assess Misuse Case Programming (MCP), an approach that automatically generates security test cases from misuse case specifications (i.e., use case specifications capturing the behavior of malicious users). MCP relies on natural language processing techniques to extract the concepts (e.g., inputs and activities) appearing in requirements specifications and generates executable test cases by matching the extracted concepts to the members of a provided test driver API. MCP has been evaluated in an industrial case study, which provides initial evidence of the feasibility and benefits of the approach. [less ▲]

Modeling Security and Privacy Requirements for Mobile Applications: a Use Case-driven Approach

Report (2017)

Defining and addressing security and privacy requirements in mobile apps is a significant challenge due to the high level of transparency regarding users' (private) information. In this paper, we propose ... [more ▼]

Defining and addressing security and privacy requirements in mobile apps is a significant challenge due to the high level of transparency regarding users' (private) information. In this paper, we propose, apply, and assess a modeling method that supports the specification of security and privacy requirements of mobile apps in a structured and analyzable form. Our motivation is that, in many contexts including mobile app development, use cases are common practice for the elicitation and analysis of functional requirements and should also be adapted for describing security requirements. We integrate and adapt an existing approach for modeling security and privacy requirements in terms of security threats, their mitigations, and their relations to use cases in a misuse case diagram. We introduce new security-related templates, i.e., a mitigation template and a misuse case template for specifying mitigation schemes and misuse case specifications in a structured and analyzable manner. Natural language processing can then be used to automatically detect and report inconsistencies among artifacts and between the templates and specifications. Since our approach supports stakeholders in precisely specifying and checking security threats, threat scenarios and their mitigations, it is expected to help with decision making and compliance with standards for improving security. We successfully applied our approach to industrial mobile apps and report lessons learned and results from structured interviews with engineers. [less ▲]