References of "ACM Transactions on Software Engineering and Methodology"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailEstimating Probabilistic Safe WCET Ranges of Real-Time Systems at Design Stages
Lee, Jaekwon UL; Shin, Seung Yeob UL; Nejati, Shiva et al

in ACM Transactions on Software Engineering and Methodology (in press)

Estimating worst-case execution times (WCET) is an important activity at early design stages of real-time systems. Based on WCET estimates, engineers make design and implementation decisions to ensure ... [more ▼]

Estimating worst-case execution times (WCET) is an important activity at early design stages of real-time systems. Based on WCET estimates, engineers make design and implementation decisions to ensure that task execution always complete before their specified deadlines. However, in practice, engineers often cannot provide precise point WCET estimates and prefer to provide plausible WCET ranges. Given a set of real-time tasks with such ranges, we provide an automated technique to determine for what WCET values the system is likely to meet its deadlines, and hence operate safely with a probabilistic guarantee. Our approach combines a search algorithm for generating worst-case scheduling scenarios with polynomial logistic regression for inferring probabilistic safe WCET ranges. We evaluated our approach by applying it to three industrial systems from different domains and several synthetic systems. Our approach efficiently and accurately estimates probabilistic safe WCET ranges within which deadlines are likely to be satisfied with a high degree of confidence. [less ▲]

Detailed reference viewed: 46 (5 UL)
Full Text
Peer Reviewed
See detailAutomated, Cost-effective, and Update-driven App Testing
Ngo, Chanh Duc UL; Pastore, Fabrizio UL; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (in press)

Apps’ pervasive role in our society led to the definition of test automation approaches to ensure their dependability. However, state-of-the-art approaches tend to generate large numbers of test inputs ... [more ▼]

Apps’ pervasive role in our society led to the definition of test automation approaches to ensure their dependability. However, state-of-the-art approaches tend to generate large numbers of test inputs and are unlikely to achieve more than 50% method coverage. In this paper, we propose a strategy to achieve significantly higher coverage of the code affected by updates with a much smaller number of test inputs, thus alleviating the test oracle problem. More specifically, we present ATUA, a model-based approach that synthesizes App models with static analysis, integrates a dynamically-refined state abstraction function, and combines complementary testing strategies, including (1) coverage of the model structure, (2) coverage of the App code, (3) random exploration, and (4) coverage of dependencies identified through information retrieval. Its model-based strategy enables ATUA to generate a small set of inputs that exercise only the code affected by the updates. In turn, this makes common test oracle solutions more cost-effective as they tend to involve human effort. A large empirical evaluation, conducted with 72 App versions belonging to nine popular Android Apps, has shown that ATUA is more effective and less effort-intensive than state-of-the-art approaches when testingApp updates. [less ▲]

Detailed reference viewed: 50 (16 UL)
Full Text
Peer Reviewed
See detailA Machine Learning Approach for Automated Filling of Categorical Fields in Data Entry Forms
Belgacem, Hichem UL; Li, Xiaochen; Bianculli, Domenico UL et al

in ACM Transactions on Software Engineering and Methodology (in press)

Users frequently interact with software systems through data entry forms. However, form filling is time-consuming and error-prone. Although several techniques have been proposed to auto-complete or pre ... [more ▼]

Users frequently interact with software systems through data entry forms. However, form filling is time-consuming and error-prone. Although several techniques have been proposed to auto-complete or pre-fill fields in the forms, they provide limited support to help users fill categorical fields, i.e., fields that require users to choose the right value among a large set of options. In this paper, we propose LAFF, a learning-based automated approach for filling categorical fields in data entry forms. LAFF first builds Bayesian Network models by learning field dependencies from a set of historical input instances, representing the values of the fields that have been filled in the past. To improve its learning ability, LAFF uses local modeling to effectively mine the local dependencies of fields in a cluster of input instances. During the form filling phase, LAFF uses such models to predict possible values of a target field, based on the values in the already-filled fields of the form and their dependencies; the predicted values (endorsed based on field dependencies and prediction confidence) are then provided to the end-user as a list of suggestions. We evaluated LAFF by assessing its effectiveness and efficiency in form filling on two datasets, one of them proprietary from the banking domain. Experimental results show that LAFF is able to provide accurate suggestions with a Mean Reciprocal Rank value above 0.73. Furthermore, LAFF is efficient, requiring at most 317 ms per suggestion. [less ▲]

Detailed reference viewed: 65 (12 UL)
Full Text
Peer Reviewed
See detailAn Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement
Hu, Qiang UL; Guo, Yuejun UL; Cordy, Maxime UL et al

in ACM Transactions on Software Engineering and Methodology (2022)

Similar to traditional software that is constantly under evolution, deep neural networks (DNNs) need to evolve upon the rapid growth of test data for continuous enhancement, e.g., adapting to distribution ... [more ▼]

Similar to traditional software that is constantly under evolution, deep neural networks (DNNs) need to evolve upon the rapid growth of test data for continuous enhancement, e.g., adapting to distribution shift in a new environment for deployment. However, it is labor-intensive to manually label all the collected test data. Test selection solves this problem by strategically choosing a small set to label. Via retraining with the selected set, DNNs will achieve competitive accuracy. Unfortunately, existing selection metrics involve three main limitations: 1) using different retraining processes; 2) ignoring data distribution shifts; 3) being insufficiently evaluated. To fill this gap, we first conduct a systemically empirical study to reveal the impact of the retraining process and data distribution on model enhancement. Then based on our findings, we propose a novel distribution-aware test (DAT) selection metric. Experimental results reveal that retraining using both the training and selected data outperforms using only the selected data. None of the selection metrics perform the best under various data distributions. By contrast, DAT effectively alleviates the impact of distribution shifts and outperforms the compared metrics by up to 5 times and 30.09% accuracy improvement for model enhancement on simulated and in-the-wild distribution shift scenarios, respectively. [less ▲]

Detailed reference viewed: 198 (51 UL)
Full Text
Peer Reviewed
See detailOn the Impact of Sample Duplication in Machine Learning based Android Malware Detection
Zhao, Yanjie; Li, Li; Wang, Haoyu et al

in ACM Transactions on Software Engineering and Methodology (2021), 30(3), 1-38

Detailed reference viewed: 31 (0 UL)
Full Text
Peer Reviewed
See detailTaming Reflection: An Essential Step Toward Whole-program Analysis of Android Apps
Sun, Xiaoyu; Li, Li; Bissyande, Tegawendé François D Assise UL et al

in ACM Transactions on Software Engineering and Methodology (2021), 30(3), 1-36

Android developers heavily use reflection in their apps for legitimate reasons. However, reflection is also significantly used for hiding malicious actions. Unfortunately, current state-of-the-art static ... [more ▼]

Android developers heavily use reflection in their apps for legitimate reasons. However, reflection is also significantly used for hiding malicious actions. Unfortunately, current state-of-the-art static analysis tools for Android are challenged by the presence of reflective calls which they usually ignore. Thus, the results of their security analysis, e.g., for private data leaks, are incomplete, given the measures taken by malware writers to elude static detection. We propose a new instrumentation-based approach to address this issue in a non-invasive way. Specifically, we introduce to the community a prototype tool called DroidRA, which reduces the resolution of reflective calls to a composite constant propagation problem and then leverages the COAL solver to infer the values of reflection targets. After that, it automatically instruments the app to replace reflective calls with their corresponding Java calls in a traditional paradigm. Our approach augments an app so that it can be more effectively statically analyzable, including by such static analyzers that are not reflection-aware. We evaluate DroidRA on benchmark apps as well as on real-world apps, and demonstrate that it can indeed infer the target values of reflective calls and subsequently allow state-of-the-art tools to provide more sound and complete analysis results. [less ▲]

Detailed reference viewed: 34 (2 UL)
Full Text
Peer Reviewed
See detailA Formal Framework of Software Product Line Analyses
Castro, Thiago; Teixeira, Leopoldo; Alves, Vander et al

in ACM Transactions on Software Engineering and Methodology (2021), 30(3), 1-37

Detailed reference viewed: 80 (11 UL)
Full Text
Peer Reviewed
See detailKilling Stubborn Mutants with Symbolic Execution
Titcheu Chekam, Thierry UL; Papadakis, Mike UL; Cordy, Maxime UL et al

in ACM Transactions on Software Engineering and Methodology (2021), 30(2), 191--1923

Detailed reference viewed: 239 (16 UL)
Full Text
Peer Reviewed
See detailWhat You See is What it Means! Semantic Representation Learning of Code based on Visualization
Keller, Patrick UL; Kabore, Abdoul Kader UL; Plein, Laura et al

in ACM Transactions on Software Engineering and Methodology (2021)

Recent successes in training word embeddings for NLP tasks have encouraged a wave of research on representation learning for sourcecode, which builds on similar NLP methods. The overall objective is then ... [more ▼]

Recent successes in training word embeddings for NLP tasks have encouraged a wave of research on representation learning for sourcecode, which builds on similar NLP methods. The overall objective is then to produce code embeddings that capture the maximumof program semantics. State-of-the-art approaches invariably rely on a syntactic representation (i.e., raw lexical tokens, abstractsyntax trees, or intermediate representation tokens) to generate embeddings, which are criticized in the literature as non-robustor non-generalizable. In this work, we investigate a novel embedding approach based on the intuition that source code has visualpatterns of semantics. We further use these patterns to address the outstanding challenge of identifying semantic code clones. Wepropose theWySiWiM(“What You See Is What It Means”) approach where visual representations of source code are fed into powerfulpre-trained image classification neural networks from the field of computer vision to benefit from the practical advantages of transferlearning. We evaluate the proposed embedding approach on the task of vulnerable code prediction in source code and on two variationsof the task of semantic code clone identification: code clone detection (a binary classification problem), and code classification (amulti-classification problem). We show with experiments on the BigCloneBench (Java), Open Judge (C) that although simple, ourWySiWiMapproach performs as effectively as state of the art approaches such as ASTNN or TBCNN. We also showed with datafrom NVD and SARD thatWySiWiMrepresentation can be used to learn a vulnerable code detector with reasonable performance(accuracy∼90%). We further explore the influence of different steps in our approach, such as the choice of visual representations or theclassification algorithm, to eventually discuss the promises and limitations of this research direction. [less ▲]

Detailed reference viewed: 90 (6 UL)
Full Text
Peer Reviewed
See detailTest Selection for Deep Learning Systems
Ma, Wei UL; Papadakis, Mike UL; Tsakmalis, Anestis et al

in ACM Transactions on Software Engineering and Methodology (2021), 30(2), 131--1322

Detailed reference viewed: 354 (39 UL)
Full Text
Peer Reviewed
See detailFine-grained Code Coverage Measurement in Automated Black-box Android Testing
Pilgun, Aleksandr UL; Gadyatskaya, Olga UL; Zhauniarovich, Yury et al

in ACM Transactions on Software Engineering and Methodology (2020), 29(4), 1-35

Today, there are millions of third-party Android applications. Some of them are buggy or even malicious. To identify such applications, novel frameworks for automated black-box testing and dynamic ... [more ▼]

Today, there are millions of third-party Android applications. Some of them are buggy or even malicious. To identify such applications, novel frameworks for automated black-box testing and dynamic analysis are being developed by the Android community. Code coverage is one of the most common metrics for evaluating effectiveness of these frameworks. Furthermore, code coverage is used as a fitness function for guiding evolutionary and fuzzy testing techniques. However, there are no reliable tools for measuring fine-grained code coverage in black-box Android app testing. We present the Android Code coVerage Tool, ACVTool for short, that instruments Android apps and measures code coverage in the black-box setting at class, method and instruction granularity. ACVTool has successfully instrumented 96.9% of apps in our experiments. It introduces a negligible instrumentation time overhead, and its runtime overhead is acceptable for automated testing tools. We demonstrate practical value of ACVTool in a large-scale experiment with Sapienz, a state-of-art automated testing tool. Using ACVTool on the same cohort of apps, we have compared different coverage granularities applied by Sapienz in terms of the found amount of crashes. Our results show that none of the applied coverage granularities clearly outperforms others in this aspect. [less ▲]

Detailed reference viewed: 86 (10 UL)
Full Text
Peer Reviewed
See detailPractical Constraint Solving for Generating System Test Data
Soltana, Ghanem; Sabetzadeh, Mehrdad UL; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (2020), 29(2), 111-1148

The ability to generate test data is often a necessary prerequisite for automated software testing. For the generated data to be fit for its intended purpose, the data usually has to satisfy various ... [more ▼]

The ability to generate test data is often a necessary prerequisite for automated software testing. For the generated data to be fit for its intended purpose, the data usually has to satisfy various logical constraints. When testing is performed at a system level, these constraints tend to be complex and are typically captured in expressive formalisms based on first-order logic. Motivated by improving the feasibility and scalability of data generation for system testing, we present a novel approach, whereby we employ a combination of metaheuristic search and Satisfiability Modulo Theories (SMT) for constraint solving. Our approach delegates constraint solving tasks to metaheuristic search and SMT in such a way as to take advantage of the complementary strengths of the two techniques. We ground our work on test data models specified in UML, with OCL used as the constraint language. We present tool support and an evaluation of our approach over three industrial case studies. The results indicate that, for complex system test data generation problems, our approach presents substantial benefits over the state of the art in terms of applicability and scalability. [less ▲]

Detailed reference viewed: 217 (45 UL)
Full Text
Peer Reviewed
See detailAn Active Learning Approach for Improving the Accuracy of Automated Domain Model Extraction
Arora, Chetan UL; Sabetzadeh, Mehrdad UL; Nejati, Shiva UL et al

in ACM Transactions on Software Engineering and Methodology (2019), 28(1),

Domain models are a useful vehicle for making the interpretation and elaboration of natural-language requirements more precise. Advances in natural language processing (NLP) have made it possible to ... [more ▼]

Domain models are a useful vehicle for making the interpretation and elaboration of natural-language requirements more precise. Advances in natural language processing (NLP) have made it possible to automatically extract from requirements most of the information that is relevant to domain model construction. However, alongside the relevant information, NLP extracts from requirements a significant amount of information that is superfluous, i.e., not relevant to the domain model. Our objective in this article is to develop automated assistance for filtering the superfluous information extracted by NLP during domain model extraction. To this end, we devise an active-learning-based approach that iteratively learns from analysts’ feedback over the relevance and superfluousness of the extracted domain model elements, and uses this feedback to provide recommendations for filtering superfluous elements. We empirically evaluate our approach over three industrial case studies. Our results indicate that, once trained, our approach automatically detects an average of ≈ 45% of the superfluous elements with a precision of ≈ 96%. Since precision is very high, the automatic recommendations made by our approach are trustworthy. Consequently, analysts can dispose of a considerable fraction – nearly half – of the superfluous elements with minimal manual work. The results are particularly promising, as they should be considered in light of the non-negligible subjectivity that is inherently tied to the notion of relevance. [less ▲]

Detailed reference viewed: 667 (132 UL)
Full Text
Peer Reviewed
See detailOracles for Testing Software Timeliness with Uncertainty
Wang, Chunhui UL; Pastore, Fabrizio UL; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (2019), 28(1),

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time ... [more ▼]

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time uncertainty leads to non-determinism. For example, time-triggered operations may either generate different valid outputs across different executions, or experience failures (e.g., results not being generated in the expected time window) that occur only occasionally over many executions. For these reasons, time uncertainty makes the generation of effective test oracles for timing requirements a challenging task. To address the above challenge, we propose STUIOS (Stochastic Testing with Unique Input Output Sequences), an approach for the automated generation of stochastic oracles that verify the capability of a software system to fulfill timing constraints in the presence of time uncertainty. Such stochastic oracles entail the statistical analysis of repeated test case executions based on test output probabilities predicted by means of statistical model checking. Results from two industrial case studies in the automotive domain demonstrate that this approach improves the fault detection effectiveness of tests suites derived from timed automata, compared to traditional approaches. [less ▲]

Detailed reference viewed: 427 (68 UL)
Full Text
Peer Reviewed
See detailAugmenting Field Data for Testing Systems Subject to Incremental Requirements Changes
Di Nardo, Daniel; Pastore, Fabrizio; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (2017), 26(1), 1-40

Detailed reference viewed: 343 (78 UL)
Full Text
Peer Reviewed
See detailCombining Genetic Algorithms and Constraint Programming to Support Stress Testing of Task Deadlines
DI ALESIO, Stefano; Briand, Lionel UL; Nejati, Shiva UL et al

in ACM Transactions on Software Engineering and Methodology (2015), 25(1),

Tasks in Real Time Embedded Systems (RTES) are often subject to hard deadlines, that constrain how quickly the system must react to external inputs. These inputs and their timing vary in a large domain ... [more ▼]

Tasks in Real Time Embedded Systems (RTES) are often subject to hard deadlines, that constrain how quickly the system must react to external inputs. These inputs and their timing vary in a large domain depending on the environment state, and can never be fully predicted prior to system execution. Therefore, approaches for stress testing must be developed to uncover possible deadline misses of tasks for different input arrival times. In this paper, we describe stress test case generation as a search problem over the space of task arrival times. Specifically, we search for worst case scenarios maximizing deadline misses where each scenario characterizes a test case. In order to scale our search to large industrial-size problems, we combine two state-of-the-art search strategies, namely Genetic Algorithms (GA) and Constraint Programming (CP). Our experimental results show that, in comparison with GA and CP in isolation, GA+CP achieves nearly the same effectiveness as CP and the same efficiency and solution diversity as GA, thus combining the advantages of the two strategies. In light of these results, we conclude that a combined GA+CP approach to stress testing is more likely to scale to large and complex systems. [less ▲]

Detailed reference viewed: 541 (54 UL)
Full Text
Peer Reviewed
See detailaToucan: An Automated Framework to Derive UML Analysis Models from Use Case Models
Yue, Tao; Briand, Lionel UL; Labiche, Yvan

in ACM Transactions on Software Engineering and Methodology (2015), 24(3),

Detailed reference viewed: 372 (34 UL)
Full Text
Peer Reviewed
See detailArchitecture-Level Configuration of Large-Scale Embedded Software Systems
Behjati, Razieh; Nejati, Shiva UL; Briand, Lionel UL

in ACM Transactions on Software Engineering and Methodology (2014), 23(3),

Configuration in the domain of integrated control systems (ICS) is largely manual, laborious, and error-prone. In this paper, we propose a model-based configuration approach that provides automation ... [more ▼]

Configuration in the domain of integrated control systems (ICS) is largely manual, laborious, and error-prone. In this paper, we propose a model-based configuration approach that provides automation support for reducing configuration effort and the likelihood of configuration errors in the ICS domain. We ground our approach on componentbased specifications of ICS families. We then develop a configuration algorithm using constraint satisfaction techniques over finite domains to generate products that are consistent with respect to their ICS family specifications. We reason about the termination and consistency of our configuration algorithm analytically. We evaluate the effectiveness of our configuration approach by applying it to a real subsea oil production system. Specifically, we have rebuilt a number of existing verified product configurations of our industry partner. Our experience shows that our approach can automatically infer up to 50% of the configuration decisions, and reduces the complexity of making configuration decisions. [less ▲]

Detailed reference viewed: 281 (36 UL)
Full Text
Peer Reviewed
See detailTraceability and SysML Design Slices to Support Safety Inspections: A Controlled Experiment
Briand, Lionel UL; Falessi, Davide; Nejati, Shiva UL et al

in ACM Transactions on Software Engineering and Methodology (2014), 23(1),

Detailed reference viewed: 257 (51 UL)