Fine-grained Code Coverage Measurement in Automated Black-box Android Testing

in ACM Transactions on Software Engineering and Methodology (2020), 29(4), 1-35

Today, there are millions of third-party Android applications. Some of them are buggy or even malicious. To identify such applications, novel frameworks for automated black-box testing and dynamic ... [more ▼]

Today, there are millions of third-party Android applications. Some of them are buggy or even malicious. To identify such applications, novel frameworks for automated black-box testing and dynamic analysis are being developed by the Android community. Code coverage is one of the most common metrics for evaluating effectiveness of these frameworks. Furthermore, code coverage is used as a fitness function for guiding evolutionary and fuzzy testing techniques. However, there are no reliable tools for measuring fine-grained code coverage in black-box Android app testing. We present the Android Code coVerage Tool, ACVTool for short, that instruments Android apps and measures code coverage in the black-box setting at class, method and instruction granularity. ACVTool has successfully instrumented 96.9% of apps in our experiments. It introduces a negligible instrumentation time overhead, and its runtime overhead is acceptable for automated testing tools. We demonstrate practical value of ACVTool in a large-scale experiment with Sapienz, a state-of-art automated testing tool. Using ACVTool on the same cohort of apps, we have compared different coverage granularities applied by Sapienz in terms of the found amount of crashes. Our results show that none of the applied coverage granularities clearly outperforms others in this aspect. [less ▲]

DEMO: An Effective Android Code Coverage Tool

Poster (2018, October 15)

The deluge of Android apps from third-party developers calls for sophisticated security testing and analysis techniques to inspect suspicious apps without accessing their source code. Code coverage is an ... [more ▼]

The deluge of Android apps from third-party developers calls for sophisticated security testing and analysis techniques to inspect suspicious apps without accessing their source code. Code coverage is an important metric used in these techniques to evaluate their effectiveness, and even as a fitness function to help achieving better results in evolutionary and fuzzy approaches. Yet, so far there are no reliable tools for measuring fine-grained bytecode coverage of Android apps. In this work we present ACVTool that instruments Android apps and measures the smali code coverage at the level of classes, methods, and instructions. Tool repository: https://github.com/pilgun/acvtool [less ▲]

Small changes, big changes: an updated view on the Android permission system

in Research in Attacks, Intrusions, and Defenses - 19th International Symposium, RAID 2016, Paris, France, September 19-21, 2016, Proceedings (2016, September)

Since the appearance of Android, its permission system was central to many studies of Android security. For a long time, the description of the architecture provided by Enck et al. was immutably used in ... [more ▼]

Since the appearance of Android, its permission system was central to many studies of Android security. For a long time, the description of the architecture provided by Enck et al. was immutably used in various research papers. The introduction of highly anticipated runtime permissions in Android 6.0 forced us to reconsider this model. To our surprise, the permission system evolved with almost every release. After analysis of 16 Android versions, we can con firm that the modi fications, especially introduced in Android 6.0, considerably impact the aptness of old conclusions and tools for newer releases. For instance, since Android 6.0 some signature permissions, previously granted only to apps signed with a platform certi cate, can be granted to third-party apps even if they are signed with a non-platform certi cate; many permissions considered before as threatening are now granted by default. In this paper, we review in detail the updated system, introduced changes, and their security implications. We highlight some bizarre behaviors, which may be of interest for developers and security researchers. We also found a number of bugs during our analysis, and provided patches to AOSP where possible. [less ▲]

StaDynA: Addressing the Problem of Dynamic Code Updates in the Security Analysis of Android Applications

in Proceedings of CODASPY '15 (2015, March)

Static analysis of Android applications can be hindered by the presence of the popular dynamic code update techniques: dynamic class loading and reflection. Recent Android malware samples do actually use ... [more ▼]

Static analysis of Android applications can be hindered by the presence of the popular dynamic code update techniques: dynamic class loading and reflection. Recent Android malware samples do actually use these mechanisms to conceal their malicious behavior from static analyzers. These techniques defuse even the most recent static analyzers that usually operate under the "closed world" assumption (the targets of reflective calls can be resolved at analysis time; only classes reachable from the class path at analysis time are used at runtime). Our proposed solution allows existing static analyzers to remove this assumption. This is achieved by combining static and dynamic analysis of applications in order to reveal the hidden/updated behavior and extend static analysis results with this information. This paper presents design, implementation and preliminary evaluation results of our solution called StaDynA. [less ▲]