Model-Based Testing of Obligations

Obligations are mandatory actions that users must perform, addressing access control requirements. To ensure that such obligations are implemented correctly, an automated and systematic testing approach is often recommended. One such approach is Model-Based Testing (MBT) that allows defining cost-effective testing strategies to support rigorous testing via automation. In this paper, we present MBT for obligations by extending the Unified Modeling Language (UML) via a profile called the Obligations Profile. Based on the profile, we define a modeling methodology utilizing the concepts of Obligations Class Diagrams (OCDs) and Obligations State Machines (OSMs), which are standard UML Class Diagrams and UML State Machines with stereotypes from the Obligations Profile. Our methodology, using OCDs and OSMs, is automatically enforced by the validation of constraints defined in the profile. To assess the completeness and applicability of the profile and methodology, we modeled 47 obligations from four different systems. The results of our case study show that we successfully modeled all the obligations and used 75% of the stereotypes that we defined in the profile. In addition, using OCDs and OSMs, we automatically generate executable test cases using a standard state machine structural coverage criterion and common test data generation strategies. The effectiveness of generated test cases is assessed using mutation analysis on two systems, using mutation operators specifically designed for obligation faults. Test case execution killed 75% of the mutants and a careful analysis further suggests that more sophisticated testing strategies must be defined to further improve testing effectiveness.