References of "Briand, Lionel 50001049"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailA Query System for Extracting Requirements-related Information from Legal Texts
Sleimi, Amin UL; Ceci, Marcello UL; Sannier, Nicolas UL et al

in Proceedings of the 27th IEEE International Requirements Engineering Conference (RE'19), pp. 319-329 (2019)

Searching legal texts for relevant information is a complex and expensive activity. The search solutions offered by present-day legal portals are targeted primarily at legal professionals. These solutions ... [more ▼]

Searching legal texts for relevant information is a complex and expensive activity. The search solutions offered by present-day legal portals are targeted primarily at legal professionals. These solutions are not adequate for requirements analysts whose objective is to extract domain knowledge including stakeholders, rights and duties, and business processes that are relevant to legal requirements. Semantic Web technologies now enable smart search capabilities and can be exploited to help requirements analysts in elaborating legal requirements. In our previous work, we developed an automated framework for extracting semantic metadata from legal texts. In this paper, we investigate the use of our metadata extraction framework as an enabler for smart legal search with a focus on requirements engineering activities. We report on our industrial experience helping the Government of Luxembourg provide an advanced search facility over Luxembourg’s Income Tax Law. The experience shows that semantic legal metadata can be successfully exploited for answering requirements engineering-related legal queries. Our results also suggest that our conceptualization of semantic legal metadata can be further improved with new information elements and relations. [less ▲]

Detailed reference viewed: 465 (44 UL)
Full Text
Peer Reviewed
See detailA Machine Learning-Based Approach for Demarcating Requirements in Textual Specifications
Abualhaija, Sallam UL; Arora, Chetan UL; Sabetzadeh, Mehrdad UL et al

in 27th IEEE International Requirements Engineering Conference (RE'19) (2019)

A simple but important task during the analysis of a textual requirements specification is to determine which statements in the specification represent requirements. In principle, by following suitable ... [more ▼]

A simple but important task during the analysis of a textual requirements specification is to determine which statements in the specification represent requirements. In principle, by following suitable writing and markup conventions, one can provide an immediate and unequivocal demarcation of requirements at the time a specification is being developed. However, neither the presence nor a fully accurate enforcement of such conventions is guaranteed. The result is that, in many practical situations, analysts end up resorting to after-the-fact reviews for sifting requirements from other material in a requirements specification. This is both tedious and time-consuming. We propose an automated approach for demarcating requirements in free-form requirements specifications. The approach, which is based on machine learning, can be applied to a wide variety of specifications in different domains and with different writing styles. We train and evaluate our approach over an independently labeled dataset comprised of 30 industrial requirements specifications. Over this dataset, our approach yields an average precision of 81.2% and an average recall of 95.7%. Compared to simple baselines that demarcate requirements based on the presence of modal verbs and identifiers, our approach leads to an average gain of 16.4% in precision and 25.5% in recall. [less ▲]

Detailed reference viewed: 634 (64 UL)
Full Text
Peer Reviewed
See detailDecision Support for Security-Control Identification Using Machine Learning
Bettaieb, Seifeddine UL; Shin, Seung Yeob UL; Sabetzadeh, Mehrdad UL et al

in International Working Conference on Requirements Engineering: Foundation for Software Quality, Essen 18-21 March 2019 (2019)

[Context & Motivation] In many domains such as healthcare and banking, IT systems need to fulfill various requirements related to security. The elaboration of security requirements for a given system is ... [more ▼]

[Context & Motivation] In many domains such as healthcare and banking, IT systems need to fulfill various requirements related to security. The elaboration of security requirements for a given system is in part guided by the controls envisaged by the applicable security standards and best practices. [Problem] An important difficulty that analysts have to contend with during security requirements elaboration is sifting through a large number of security controls and determining which ones have a bearing on the security requirements for a given system. This challenge is often exacerbated by the scarce security expertise available in most organizations. [Principal ideas/results] In this paper, we develop automated decision support for the identification of security controls that are relevant to a specific system in a particular context. Our approach, which is based on machine learning, leverages historical data from security assessments performed over past systems in order to recommend security controls for a new system. We operationalize and empirically evaluate our approach using real historical data from the banking domain. Our results show that, when one excludes security controls that are rare in the historical data, our approach has an average recall of ≈ 95% and average precision of ≈ 67%. [Contribution] The high recall – indicating only a few relevant security controls are missed – combined with the reasonable level of precision – indicating that the effort required to confirm recommendations is not excessive – suggests that our approach is a useful aid to analysts for more efficiently identifying the relevant security controls, and also for decreasing the likelihood that important controls would be overlooked. [less ▲]

Detailed reference viewed: 319 (67 UL)
Full Text
Peer Reviewed
See detailOracles for Testing Software Timeliness with Uncertainty
Wang, Chunhui UL; Pastore, Fabrizio UL; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (2019), 28(1),

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time ... [more ▼]

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time uncertainty leads to non-determinism. For example, time-triggered operations may either generate different valid outputs across different executions, or experience failures (e.g., results not being generated in the expected time window) that occur only occasionally over many executions. For these reasons, time uncertainty makes the generation of effective test oracles for timing requirements a challenging task. To address the above challenge, we propose STUIOS (Stochastic Testing with Unique Input Output Sequences), an approach for the automated generation of stochastic oracles that verify the capability of a software system to fulfill timing constraints in the presence of time uncertainty. Such stochastic oracles entail the statistical analysis of repeated test case executions based on test output probabilities predicted by means of statistical model checking. Results from two industrial case studies in the automotive domain demonstrate that this approach improves the fault detection effectiveness of tests suites derived from timed automata, compared to traditional approaches. [less ▲]

Detailed reference viewed: 429 (68 UL)
Full Text
Peer Reviewed
See detailEnabling Model Testing of Cyber-Physical Systems
Gonzalez Perez, Carlos Alberto UL; Varmazyar, Mojtaba UL; Nejati, Shiva UL et al

in Proceedings of ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS’18) (2018, October)

Applying traditional testing techniques to Cyber-Physical Systems (CPS) is challenging due to the deep intertwining of software and hardware, and the complex, continuous interactions between the system ... [more ▼]

Applying traditional testing techniques to Cyber-Physical Systems (CPS) is challenging due to the deep intertwining of software and hardware, and the complex, continuous interactions between the system and its environment. To alleviate these challenges we propose to conduct testing at early stages and over executable models of the system and its environment. Model testing of CPSs is however not without difficulties. The complexity and heterogeneity of CPSs renders necessary the combination of different modeling formalisms to build faithful models of their different components. The execution of CPS models thus requires an execution framework supporting the co-simulation of different types of models, including models of the software (e.g., SysML), hardware (e.g., SysML or Simulink), and physical environment (e.g., Simulink). Furthermore, to enable testing in realistic conditions, the co-simulation process must be (1) fast, so that thousands of simulations can be conducted in practical time, (2) controllable, to precisely emulate the expected runtime behavior of the system and, (3) observable, by producing simulation data enabling the detection of failures. To tackle these challenges, we propose a SysML-based modeling methodology for model testing of CPSs, and an efficient SysML-Simulink co-simulation framework. Our approach was validated on a case study from the satellite domain. [less ▲]

Detailed reference viewed: 442 (53 UL)
Full Text
Peer Reviewed
See detailHITECS: A UML Profile and Analysis Framework for Hardware-in-the-Loop Testing of Cyber Physical Systems
Shin, Seung Yeob UL; Chaouch, Karim UL; Nejati, Shiva UL et al

in Proceedings of ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS’18) (2018, October)

Hardware-in-the-loop (HiL) testing is an important step in the development of cyber physical systems (CPS). CPS HiL test cases manipulate hardware components, are time-consuming and their behaviors are ... [more ▼]

Hardware-in-the-loop (HiL) testing is an important step in the development of cyber physical systems (CPS). CPS HiL test cases manipulate hardware components, are time-consuming and their behaviors are impacted by the uncertainties in the CPS environment. To mitigate the risks associated with HiL testing, engineers have to ensure that (1) HiL test cases are well-behaved, i.e., they implement valid test scenarios and do not accidentally damage hardware, and (2) HiL test cases can execute within the time budget allotted to HiL testing. This paper proposes an approach to help engineers systematically specify and analyze CPS HiL test cases. Leveraging the UML profile mechanism, we develop an executable domain-specific language, HITECS, for HiL test case specification. HITECS builds on the UML Testing Profile (UTP) and the UML action language (Alf). Using HITECS, we provide analysis methods to check whether HiL test cases are well-behaved, and to estimate the execution times of these test cases before the actual HiL testing stage. We apply HITECS to an industrial case study from the satellite domain. Our results show that: (1) HITECS is feasible to use in practice; (2) HITECS helps engineers define more complete and effective well-behavedness assertions for HiL test cases, compared to when these assertions are defined without systematic guidance; (3) HITECS verifies in practical time that HiL test cases are well-behaved; and (4) HITECS accurately estimates HiL test case execution times. [less ▲]

Detailed reference viewed: 390 (90 UL)
Full Text
Peer Reviewed
See detailModel-Driven Trace Diagnostics for Pattern-based Temporal Specifications
Dou, Wei UL; Bianculli, Domenico UL; Briand, Lionel UL

in Proceedings of the 2018 ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018) (2018, October)

Offline trace checking tools check whether a specification holds on a log of events recorded at run time; they yield a verification verdict (typically a boolean value) when the checking process ends. When ... [more ▼]

Offline trace checking tools check whether a specification holds on a log of events recorded at run time; they yield a verification verdict (typically a boolean value) when the checking process ends. When the verdict is false, a software engineer needs to diagnose the property violations found in the trace in order to understand their cause and, if needed, decide for corrective actions to be performed on the system. However, a boolean verdict may not be informative enough to perform trace diagnostics, since it does not provide any useful information about the cause of the violation and because a property can be violated for multiple reasons. The goal of this paper is to provide a practical and scalable so- lution to solve the trace diagnostics problem, in the settings of model-driven trace checking of temporal properties expressed in TemPsy, a pattern-based specification language. The main contributions of the paper are: a model-driven approach for trace diagnostics of pattern-based temporal properties expressed in TemPsy, which relies on the evaluation of OCL queries on an instance of a trace meta-model; the implementation of this trace diagnostics procedure in the TemPsy-Report tool; the evaluation of the scalability of TemPsy-Report, when used for the diagnostics of violations of real properties derived from a case study of our industrial partner. The results show that TemPsy-Report is able to collect diagnostic information from large traces (with one million events) in less than ten seconds; TemPsy-Report scales linearly with respect to the length of the trace and keeps approximately constant performance as the number of violations increases. [less ▲]

Detailed reference viewed: 257 (37 UL)
Full Text
Peer Reviewed
See detailModel-driven Run-time Enforcement of Complex Role-based Access Control Policies
Ben Fadhel, Ameni UL; Bianculli, Domenico UL; Briand, Lionel UL

in Proceeding of the 2018 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE ’18) (2018, September)

A Role-based Access Control (RBAC) mechanism prevents unauthorized users to perform an operation, according to authorization policies which are defined on the user’s role within an enterprise. Several ... [more ▼]

A Role-based Access Control (RBAC) mechanism prevents unauthorized users to perform an operation, according to authorization policies which are defined on the user’s role within an enterprise. Several models have been proposed to specify complex RBAC policies. However, existing approaches for policy enforcement do not fully support all the types of policies that can be expressed in these models, which hinders their adoption among practitioners. In this paper we propose a model-driven enforcement framework for complex policies captured by GemRBAC+CTX, a comprehensive RBAC model proposed in the literature. We reduce the problem of making an access decision to checking whether a system state (from an RBAC point of view), expressed as an instance of the GemRBAC+CTX model, satisfies the constraints corresponding to the RBAC policies to be enforced at run time. We provide enforcement algorithms for various types of access requests and events, and a prototype tool (MORRO) implementing them. We also show how to integrate MORRO into an industrial Web application. The evaluation results show the applicability of our approach on a industrial system and its scalability with respect to the various parameters characterizing an AC configuration. [less ▲]

Detailed reference viewed: 343 (41 UL)
Full Text
Peer Reviewed
See detailA Machine Learning-Driven Evolutionary Approach for Testing Web Application Firewalls
Appelt, Dennis UL; Nguyen, Duy Cu UL; Panichella, Annibale UL et al

in IEEE Transactions on Reliability (2018), 67(3), 733-757

Web application firewalls (WAF) are an essential protection mechanism for online software systems. Because of the relentless flow of new kinds of attacks as well as their increased sophistication, WAFs ... [more ▼]

Web application firewalls (WAF) are an essential protection mechanism for online software systems. Because of the relentless flow of new kinds of attacks as well as their increased sophistication, WAFs have to be updated and tested regularly to prevent attackers from easily circumventing them. In this paper, we focus on testing WAFs for SQL injection attacks, but the general principles and strategy we propose can be adapted to other contexts. We present ML-Driven, an approach based on machine learning and an evolutionary algorithm to automatically detect holes in WAFs that let SQL injection attacks bypass them. Initially, ML-Driven automatically generates a diverse set of attacks and submit them to the system being protected by the target WAF. Then, ML-Driven selects attacks that exhibit patterns (substrings) associated with bypassing the WAF and evolve them to generate new successful bypassing attacks. Machine learning is used to incrementally learn attack patterns from previously generated attacks according to their testing results, i.e., if they are blocked or bypass the WAF. We implemented ML-Driven in a tool and evaluated it on ModSecurity, a widely used open-source WAF, and a proprietary WAF protecting a financial institution. Our empirical results indicate that ML-Driven is effective and efficient at generating SQL injection attacks bypassing WAFs and identifying attack patterns. [less ▲]

Detailed reference viewed: 799 (106 UL)
Full Text
Peer Reviewed
See detailSoftware Engineering Research and Industry: A Symbiotic Relationship to Foster Impact
Basili, Victor; Briand, Lionel UL; Bianculli, Domenico UL et al

in IEEE Software (2018), 35(5), 44-49

Software engineering is not only an increasingly challenging endeavor that goes beyond the intellectual capabilities of any single individual engineer, but is also an intensely human one. Tools and ... [more ▼]

Software engineering is not only an increasingly challenging endeavor that goes beyond the intellectual capabilities of any single individual engineer, but is also an intensely human one. Tools and methods to develop software are employed by engineers of varied backgrounds within a large variety of organizations and application domains. As a result, the variation in challenges and practices in system requirements, architecture, and quality assurance is staggering. Human, domain and organizational factors define the context within which software engineering methodologies and technologies are to be applied and therefore the context that research needs to account for, if it is to be impactful. This paper provides an assessment of the current challenges faced by software engineering research in achieving its potential, a description of the root causes of such challenges, and a proposal for the field to move forward and become more impactful through collaborative research and innovation between public research and industry. [less ▲]

Detailed reference viewed: 336 (42 UL)
Full Text
Peer Reviewed
See detailAutomated Extraction of Semantic Legal Metadata Using Natural Language Processing
Sleimi, Amin UL; Sannier, Nicolas UL; Sabetzadeh, Mehrdad UL et al

in the 26th IEEE International Requirements Engineering Conference, Banff, Alberta, 20-24 August 2018 (2018, August)

[Context] Semantic legal metadata provides information that helps with understanding and interpreting the meaning of legal provisions. Such metadata is important for the systematic analysis of legal ... [more ▼]

[Context] Semantic legal metadata provides information that helps with understanding and interpreting the meaning of legal provisions. Such metadata is important for the systematic analysis of legal requirements. [Objectives] Our work is motivated by two observations: (1) The existing requirements engineering (RE) literature does not provide a harmonized view on the semantic metadata types that are useful for legal requirements analysis. (2) Automated support for the extraction of semantic legal metadata is scarce, and further does not exploit the full potential of natural language processing (NLP). Our objective is to take steps toward addressing these limitations. [Methods] We review and reconcile the semantic legal metadata types proposed in RE. Subsequently, we conduct a qualitative study aimed at investigating how the identified metadata types can be extracted automatically. [Results and Conclusions] We propose (1) a harmonized conceptual model for the semantic metadata types pertinent to legal requirements analysis, and (2) automated extraction rules for these metadata types based on NLP. We evaluate the extraction rules through a case study. Our results indicate that the rules generate metadata annotations with high accuracy. [less ▲]

Detailed reference viewed: 497 (62 UL)
Full Text
Peer Reviewed
See detailTest Case Prioritization for Acceptance Testing of Cyber Physical Systems: A Multi-Objective Search-Based Approach
Shin, Seung Yeob UL; Nejati, Shiva UL; Sabetzadeh, Mehrdad UL et al

in Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'18) (2018, July)

Acceptance testing validates that a system meets its requirements and determines whether it can be sufficiently trusted and put into operation. For cyber physical systems (CPS), acceptance testing is a ... [more ▼]

Acceptance testing validates that a system meets its requirements and determines whether it can be sufficiently trusted and put into operation. For cyber physical systems (CPS), acceptance testing is a hardware-in-the-loop process conducted in a (near-)operational environment. Acceptance testing of a CPS often necessitates that the test cases be prioritized, as there are usually too many scenarios to consider given time constraints. CPS acceptance testing is further complicated by the uncertainty in the environment and the impact of testing on hardware. We propose an automated test case prioritization approach for CPS acceptance testing, accounting for time budget constraints, uncertainty, and hardware damage risks. Our approach is based on multi-objective search, combined with a test case minimization algorithm that eliminates redundant operations from an ordered sequence of test cases. We evaluate our approach on a representative case study from the satellite domain. The results indicate that, compared to test cases that are prioritized manually by satellite engineers, our automated approach more than doubles the number of test cases that fit into a given time frame, while reducing to less than one third the number of operations that entail the risk of damage to key hardware components. [less ▲]

Detailed reference viewed: 479 (76 UL)
Full Text
Peer Reviewed
See detailConfiguring use case models in product families
Hajri, Ines UL; Göknil, Arda UL; Briand, Lionel UL et al

in Software & Systems Modeling (2018), 17(3), 939-971

In many domains such as automotive and avionics, the size and complexity of software systems is quickly increasing. At the same time, many stakeholders tend to be involved in the development of such ... [more ▼]

In many domains such as automotive and avionics, the size and complexity of software systems is quickly increasing. At the same time, many stakeholders tend to be involved in the development of such systems, which typically must also be configured for multiple customers with varying needs. Product Line Engineering (PLE) is therefore an inevitable practice for such systems. Furthermore, because in many areas requirements must be explicit and traceability to them is required by standards, use cases and domain models are common practice for requirements elicitation and analysis. In this paper, based on the above observations, we aim at supporting PLE in the context of use case-centric development. Therefore, we propose, apply, and assess a use case-driven configuration approach which interactively receives configuration decisions from the analysts to generate Product Specific (PS) use case and domain models. Our approach provides the following: (1) a use case-centric product line modeling method (PUM), (2) automated, interactive configuration support based on PUM, and (3) an automatic generation of PS use case and domain models from Product Line (PL) models and configuration decisions. The approach is supported by a tool relying on Natural Language Processing (NLP), and integrated with an industrial requirements management tool, i.e., IBM Doors. We successfully applied and evaluated our approach to an industrial case study in the automotive domain, thus showing evidence that the approach is practical and beneficial to capture variability at the appropriate level of granularity and to configure PS use case and domain models in industrial settings. [less ▲]

Detailed reference viewed: 638 (221 UL)
Full Text
Peer Reviewed
See detailChange Impact Analysis for Evolving Configuration Decisions in Product Line Use Case Models
Hajri, Ines UL; Göknil, Arda UL; Briand, Lionel UL et al

in Journal of Systems and Software (2018), 139

Product Line Engineering is becoming a key practice in many software development environments where complex systems are developed for multiple customers with varying needs. In many business contexts, use ... [more ▼]

Product Line Engineering is becoming a key practice in many software development environments where complex systems are developed for multiple customers with varying needs. In many business contexts, use cases are the main artifacts for communicating requirements among stakeholders. In such contexts, Product Line (PL) use cases capture variable and common requirements while use case-driven configuration generates Product Specific (PS) use cases for each new customer in a product family. In this paper, we propose, apply, and assess a change impact analysis approach for evolving configuration decisions in PL use case models. Our approach includes: (1) automated support to identify the impact of decision changes on prior and subsequent decisions in PL use case diagrams and (2) automated incremental regeneration of PS use case models from PL use case models and evolving configuration decisions. Our tool support is integrated with IBM Doors. Our approach has been evaluated in an industrial case study, which provides evidence that it is practical and beneficial to analyze the impact of decision changes and to incrementally regenerate PS use case models in industrial settings. [less ▲]

Detailed reference viewed: 338 (113 UL)
Full Text
Peer Reviewed
See detailEffective Fault Localization of Automotive Simulink Models: Achieving the Trade-Off between Test Oracle Effort and Fault Localization Accuracy
Liu, Bing; Nejati, Shiva UL; Lucia, Lucia et al

in Empirical Software Engineering (2018), 24(1), 444-490

One promising way to improve the accuracy of fault localization based on statistical debugging is to increase diversity among test cases in the underlying test suite. In many practical situations, adding ... [more ▼]

One promising way to improve the accuracy of fault localization based on statistical debugging is to increase diversity among test cases in the underlying test suite. In many practical situations, adding test cases is not a cost-free option because test oracles are developed manually or running test cases is expensive. Hence, we require to have test suites that are both diverse and small to improve debugging. In this paper, we focus on improving fault localization of Simulink models by generating test cases. We identify four test objectives that aim to increase test suite diversity. We use four objectives in a search-based algorithm to generate diversified but small test suites. To further minimize test suite sizes, we develop a prediction model to stop test generation when adding test cases is unlikely to improve fault localization. We evaluate our approach using three industrial subjects. Our results show (1) expanding test suites used for fault localization using any of our four test objectives, even when the expansion is small, can significantly improve the accuracy of fault localization, (2) varying test objectives used to generate the initial test suites for fault localization does not have a significant impact on the fault localization results obtained based on those test suites, and (3) we identify an optimal configuration for prediction models to help stop test generation when it is unlikely to be beneficial. We further show that our optimal prediction model is able to maintain almost the same fault localization accuracy while reducing the average number of newly generated test cases by more than half. [less ▲]

Detailed reference viewed: 434 (64 UL)
Full Text
Peer Reviewed
See detailModel-Based Simulation of Legal Policies: Framework, Tool Support, and Validation
Soltana, Ghanem UL; Sannier, Nicolas UL; Sabetzadeh, Mehrdad UL et al

in Software & Systems Modeling (2018), 17(3), 851-883

Simulation of legal policies is an important decision-support tool in domains such as taxation. The primary goal of legal policy simulation is predicting how changes in the law affect measures of interest ... [more ▼]

Simulation of legal policies is an important decision-support tool in domains such as taxation. The primary goal of legal policy simulation is predicting how changes in the law affect measures of interest, e.g., revenue. Legal policy simulation is currently implemented using a combination of spreadsheets and software code. Such a direct implementation poses a validation challenge. In particular, legal experts often lack the necessary software background to review complex spreadsheets and code. Consequently, these experts currently have no reliable means to check the correctness of simulations against the requirements envisaged by the law. A further challenge is that representative data for simulation may be unavailable, thus necessitating a data generator. A hard-coded generator is difficult to build and validate. We develop a framework for legal policy simulation that is aimed at addressing the challenges above. The framework uses models for specifying both legal policies and the probabilistic characteristics of the underlying population. We devise an automated algorithm for simulation data generation. We evaluate our framework through a case study on Luxembourg’s Tax Law. [less ▲]

Detailed reference viewed: 475 (91 UL)
Full Text
Peer Reviewed
See detailTesting Vision-Based Control Systems Using Learnable Evolutionary Algorithms
Ben Abdessalem (helali), Raja UL; Nejati, Shiva UL; Briand, Lionel UL et al

in Proceedings of the 40th International Conference on Software Engineering (ICSE 2018) (2018)

Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We ... [more ▼]

Vision-based control systems are key enablers of many autonomous vehicular systems, including self-driving cars. Testing such systems is complicated by complex and multidimensional input spaces. We propose an automated testing algorithm that builds on learnable evolutionary algorithms. These algorithms rely on machine learning or a combination of machine learning and Darwinian genetic operators to guide the generation of new solutions (test scenarios in our context). Our approach combines multiobjective population-based search algorithms and decision tree classification models to achieve the following goals: First, classification models guide the search-based generation of tests faster towards critical test scenarios (i.e., test scenarios leading to failures). Second, search algorithms refine classification models so that the models can accurately characterize critical regions (i.e., the regions of a test input space that are likely to contain most critical test scenarios). Our evaluation performed on an industrial automotive vision-based control system shows that: (1) Our algorithm outperforms a baseline evolutionary search algorithm and generates 78% more distinct, critical test scenarios compared to the baseline algorithm. (2) Our algorithm accurately characterizes critical regions of the system under test, thus identifying the conditions that likely to lead to system failures. [less ▲]

Detailed reference viewed: 894 (183 UL)
Full Text
Peer Reviewed
See detailTesting Autonomous Cars for Feature Interaction Failures using Many-Objective Search
Ben Abdessalem (helali), Raja UL; Panichella, Annibale; Nejati, Shiva UL et al

in Proceedings of the 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2018) (2018)

Complex systems such as autonomous cars are typically built as a composition of features that are independent units of functionality. Features tend to interact and impact one another’s behavior in unknown ... [more ▼]

Complex systems such as autonomous cars are typically built as a composition of features that are independent units of functionality. Features tend to interact and impact one another’s behavior in unknown ways. A challenge is to detect and manage feature interactions, in particular, those that violate system requirements, hence leading to failures. In this paper, we propose a technique to detect feature interaction failures by casting our approach into a search-based test generation problem. We define a set of hybrid test objectives (distance functions) that combine traditional coverage-based heuristics with new heuristics specifically aimed at revealing feature interaction failures. We develop a new search-based test generation algorithm, called FITEST, that is guided by our hybrid test objectives. FITEST extends recently proposed many-objective evolutionary algorithms to reduce the time required to compute fitness values. We evaluate our approach using two versions of an industrial self-driving system. Our results show that our hybrid test objectives are able to identify more than twice as many feature interaction failures as two baseline test objectives used in the software testing literature (i.e., coverage-based and failure-based test objectives). Further, the feedback from domain experts indicates that the detected feature interaction failures represent real faults in their systems that were not previously identified based on analysis of the system features and their requirements. [less ▲]

Detailed reference viewed: 651 (75 UL)
Full Text
Peer Reviewed
See detailA Search-based Approach for Accurate Identification of Log Message Formats
Messaoudi, Salma UL; Panichella, Annibale UL; Bianculli, Domenico UL et al

in Proceedings of the 26th IEEE/ACM International Conference on Program Comprehension (ICPC ’18) (2018)

Many software engineering activities process the events contained in log files. However, before performing any processing activity, it is necessary to parse the entries in a log file, to retrieve the ... [more ▼]

Many software engineering activities process the events contained in log files. However, before performing any processing activity, it is necessary to parse the entries in a log file, to retrieve the actual events recorded in the log. Each event is denoted by a log message, which is composed of a fixed part-called (event) template-that is the same for all occurrences of the same event type, and a variable part, which may vary with each event occurrence. The formats of log messages, in complex and evolving systems, have numerous variations, are typically not entirely known, and change on a frequent basis; therefore, they need to be identified automatically. The log message format identification problem deals with the identification of the different templates used in the messages of a log. Any solution to this problem has to generate templates that meet two main goals: generating templates that are not too general, so as to distinguish different events, but also not too specific, so as not to consider different occurrences of the same event as following different templates; however, these goals are conflicting. In this paper, we present the MoLFI approach, which recasts the log message identification problem as a multi-objective problem. MoLFI uses an evolutionary approach to solve this problem, by tailoring the NSGA-II algorithm to search the space of solutions for a Pareto optimal set of message templates. We have implemented MoLFI in a tool, which we have evaluated on six real-world datasets, containing log files with a number of entries ranging from 2K to 300K. The experiments results show that MoLFI extracts by far the highest number of correct log message templates, significantly outperforming two state-of-the-art approaches on all datasets. [less ▲]

Detailed reference viewed: 1241 (107 UL)