Paper published in a book (Scientific congresses, symposiums and conference proceedings)
Negative Results of Fusing Code and Documentation for Learning to Accurately Identify Sensitive Source and Sink Methods An Application to the Android Framework for Data Leak Detection
[en] Apps on mobile phones manipulate all sorts of data, including sensitive data, leading to privacy related concerns. Recent regulations like the European GDPR provide rules for the processing of personal and sensitive data, like that no such data may be leaked without the consent of the user.
Researchers have proposed sophisticated approaches to track sensitive data within mobile apps, all of which rely on specific lists of sensitive source and sink methods. The data flow analysis results greatly depend on these lists' quality. Previous approaches either used incomplete hand-written lists and quickly became outdated or relied on machine learning. The latter, however, leads to numerous false positives, as we show.
This paper introduces CoDoC that aims to revive the machine-learning approach to precisely identify the privacy-related source and sink API methods. In contrast to previous approaches, CoDoC uses deep learning techniques and combines the source code with the documentation of API methods.
Firstly, we propose novel definitions that clarify the concepts of taint analysis, source, and sink methods.
Secondly, based on these definitions, we build a new ground truth of Android methods representing sensitive source, sink, and neither methods that will be used to train our classifier.
We evaluate CoDoC and show that, on our validation dataset, it achieves a precision, recall, and F1 score of 91%, outperforming the state-of-the-art SuSi.
However, similarly to existing tools, we show that in the wild, i.e., with unseen data, CoDoC performs poorly and generates many false-positive results. Our findings suggest that machine-learning models for abstract concepts such as privacy fail in practice despite good lab results.
To encourage future research, we release all our artifacts to the community.
Research center :
- Interdisciplinary Centre for Security, Reliability and Trust (SnT) > TruX - Trustworthy Software Engineering
Disciplines :
Computer science
Author, co-author :
Samhi, Jordan ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > TruX
Kober, Kober
Kabore, Abdoul Kader ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > TruX
Arzt, Steven; Fraunhofer Institute for Secure Information Technology, Darmstadt, Hessen, Germany
Klein, Jacques ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > TruX
External co-authors :
yes
Language :
English
Title :
Negative Results of Fusing Code and Documentation for Learning to Accurately Identify Sensitive Source and Sink Methods An Application to the Android Framework for Data Leak Detection
Publication date :
March 2023
Event name :
30th edition of the IEEE International Conference on Software Analysis, Evolution and Reengineering
Event place :
Macao, China
Event date :
from 21/03/2023 to 24/03/2023
Audience :
International
Main work title :
30th IEEE International Conference on Software Analysis, Evolution and Reengineering
Peer reviewed :
Peer reviewed
Focus Area :
Security, Reliability and Trust
FnR Project :
FNR14596679 - Dissecting Android Applications Using Static Analysis, 2020 (01/03/2020-31/10/2023) - Jordan Samhi
Abien Fred Agarap. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375, 2018.
Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. Code2vec: Learning distributed representations of code. Proc. ACM Program. Lang., 3(POPL), January 2019.
Steven Arzt, Siegfried Rasthofer, and Eric Bodden. Susi: A tool for the fully automated classification and categorization of android sources and sinks. University of Darmstadt, Tech. Rep. TUDCS-2013-0114, 2013.
Steven Arzt, Siegfried Rasthofer, Christian Fritz, Eric Bodden, Alexandre Bartel, Jacques Klein, Yves Le Traon, Damien Octeau, and Patrick McDaniel. Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. SIGPLAN Not., 49(6):259-269, June 2014.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.
Chaity Banerjee, Tathagata Mukherjee, and Eduardo Pasiliao Jr. An empirical study on generalizations of the relu activation function. In Proceedings of the 2019 ACM Southeast Conference, pages 164-167, 2019.
Luciano Bello and Marco Pistoia. Ares: triggering payload of evasive android malware. In 2018 IEEE/ACM 5th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pages 2-12. IEEE, 2018.
David Brumley, Cody Hartwig, Zhenkai Liang, James Newsome, Dawn Song, and Heng Yin. Automatically identifying trigger-based behavior in malware. In Botnet Detection, pages 65-88. Springer, 2008.
Jun Cai, Peng Zou, Jinxin Ma, and Jun He. Sworddta: A dynamic taint analysis tool for software vulnerability detection. Wuhan University Journal of Natural Sciences, 21(1):10-20, Feb 2016.
Erika Chin, Adrienne Porter Felt, Kate Greenwood, and David Wagner. Analyzing inter-application communication in android. In Proceedings of the 9th International Conference on Mobile Systems, Applications, and Services, MobiSys '11, page 239-252, New York, NY, USA, 2011. Association for Computing Machinery.
Jacob Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37-46, 1960.
Janez Demŝar. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res., 7:1-30, December 2006.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
William Enck, Peter Gilbert, Seungyeop Han, Vasant Tendulkar, Byung-Gon Chun, Landon P Cox, Jaeyeon Jung, Patrick McDaniel, and Anmol N Sheth. Taintdroid: an information-flow tracking system for realtime privacy monitoring on smartphones. ACM Transactions on Computer Systems (TOCS), 32(2):1-29, 2014.
William Enck, Damien Octeau, Patrick D McDaniel, and Swarat Chaudhuri. A study of android application security. In USENIX security symposium, volume 2, 2011.
Ming Fan, Le Yu, Sen Chen, Hao Zhou, Xiapu Luo, Shuyue Li, Yang Liu, Jun Liu, and Ting Liu. An empirical evaluation of gdpr compliance violations in android mhealth apps. In 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), pages 253-264, 2020.
Pietro Ferrara and Fausto Spoto. Static analysis for gdpr compliance. In ITASEC, 2018.
Y. Fratantonio, A. Bianchi, W. Robertson, E. Kirda, C. Kruegel, and G. Vigna. Triggerscope: Towards detecting logic bombs in android applications. In 2016 IEEE Symposium on Security and Privacy (SP), pages 377-396, 2016.
Clint Gibler, Jonathan Crussell, Jeremy Erickson, and Hao Chen. Androidleaks: Automatically detecting potential privacy leaks in android applications on a large scale. In Stefan Katzenbeisser, Edgar Weippl, L. Jean Camp, Melanie Volkamer, Mike Reiter, and Xinwen Zhang, editors, Trust and Trustworthy Computing, pages 291-307, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg.
Michael I Gordon, Deokhwan Kim, Jeff H Perkins, Limei Gilham, Nguyen Nguyen, and Martin C Rinard. Information flow analysis of android applications in droidsafe. In NDSS, volume 15, page 110, 2015.
Sigmund Albert Gorski, Benjamin Andow, Adwait Nadkarni, Sunil Manandhar, William Enck, Eric Bodden, and Alexandre Bartel. Acminer: Extraction and analysis of authorization checks in android's middleware. In Proceedings of the Ninth ACM Conference on Data and Application Security and Privacy, CODASPY '19, page 25-36, New York, NY, USA, 2019. Association for Computing Machinery.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700-4708, 2017.
Hao Jiang, Hongli Yang, Shengchao Qin, Zhendong Su, Jian Zhang, and Jun Yan. Detecting energy bugs in android apps using static analysis. In Zhenhua Duan and Luke Ong, editors, Formal Methods and Software Engineering, pages 192-208, Cham, 2017. Springer International Publishing.
Mohsin Junaid, Donggang Liu, and David Kung. Dexteroid: Detecting malicious behaviors in android apps using reverse-engineered life cycle models. Computers & Security, 59:92-117, 2016.
Jinyung Kim, Yongho Yoon, Kwangkeun Yi, Junbum Shin, and SWRD Center. Scandal: Static analyzer for detecting privacy leaks in android applications. MoST, 12:1, 2012.
Ron Kohavi et al. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, volume 14, pages 1137-1145. Montreal, Canada, 1995.
Li Li, Kevin Allix, Daoyuan Li, Alexandre Bartel, Tegawendé F. Bissyandé, and Jacques Klein. Potential component leaks in android apps: An investigation into a new feature set for malware detection. In 2015 IEEE International Conference on Software Quality, Reliability and Security, pages 195-200, 2015.
Li Li, Alexandre Bartel, Tegawendé F Bissyandé, Jacques Klein, Yves Le Traon, Steven Arzt, Siegfried Rasthofer, Eric Bodden, Damien Octeau, and Patrick McDaniel. Iccta: Detecting inter-component privacy leaks in android apps. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, volume 1, pages 280-291. IEEE, 2015.
Li Li, Alexandre Bartel, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon. Apkcombiner: Combining multiple android apps to support inter-app analysis. In Hannes Federrath and Dieter Gollmann, editors, ICT Systems Security and Privacy Protection, pages 513-527, Cham, 2015. Springer International Publishing.
Li Li, Tegawendé F. Bissyandé, Yves Le Traon, and Jacques Klein. Accessing inaccessible android apis: An empirical study. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 411-422, 2016.
Li Li, Tegawendé F. Bissyandé, Mike Papadakis, Siegfried Rasthofer, Alexandre Bartel, Damien Octeau, Jacques Klein, and Le Traon. Static analysis of android apps: A systematic literature review. Information and Software Technology, 88:67-95, 2017.
Zhuang Liu, Wayne Lin, Ya Shi, and Jun Zhao. A robustly optimized bert pre-training approach with post-training. In Sheng Li, Maosong Sun, Yang Liu, Hua Wu, Liu Kang, Wanxiang Che, Shizhu He, and Gaoqi Rao, editors, Chinese Computational Linguistics, pages 471-484, Cham, 2021. Springer International Publishing.
V. Benjamin Livshits and Monica S. Lam. Finding security vulnerabilities in java applications with static analysis. In Proceedings of the 14th Conference on USENIX Security Symposium-Volume 14, SSYM'05, page 18, USA, 2005. USENIX Association.
Long Lu, Zhichun Li, Zhenyu Wu, Wenke Lee, and Guofei Jiang. Chex: Statically vetting android apps for component hijacking vulnerabilities. In Proceedings of the 2012 ACM Conference on Computer and Communications Security, CCS '12, page 229-240, New York, NY, USA, 2012. Association for Computing Machinery.
Linghui Luo, Eric Bodden, and Johannes Späth. A qualitative analysis of android taint-analysis results. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 102-114, 2019.
Yuhong Nan, Zhemin Yang, Xiaofeng Wang, Yuan Zhang, Donglai Zhu, and Min Yang. Finding clues for your secrets: Semantics-driven, learning-based privacy discovery in mobile apps. In NDSS, 2018.
James Newsome and Dawn Xiaodong Song. Dynamic taint analysis for automatic detection, analysis, and signaturegeneration of exploits on commodity software. In NDSS, volume 5, pages 3-4. Citeseer, 2005.
Damien Octeau, Daniel Luchaup, Matthew Dering, Somesh Jha, and Patrick McDaniel. Composite constant propagation: Application to android inter-component communication analysis. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, volume 1, pages 77-88, 2015.
Xiaorui Pan, Xueqiang Wang, Yue Duan, XiaoFeng Wang, and Heng Yin. Dark hazard: Learning-based, large-scale discovery of hidden sensitive operations in android apps. In NDSS, 2017.
Thanasis Petsas, Giannis Voyatzis, Elias Athanasopoulos, Michalis Polychronakis, and Sotiris Ioannidis. Rage against the virtual machine: Hindering dynamic analysis of android malware. In Proceedings of the Seventh European Workshop on System Security, EuroSec '14, New York, NY, USA, 2014. Association for Computing Machinery.
Siegfried Rasthofer, Steven Arzt, Enrico Lovat, and Eric Bodden. Droidforce: Enforcing complex, data-centric, system-wide policies in android. In 2014 Ninth International Conference on Availability, Reliability and Security, pages 40-49. IEEE, 2014.
Siegfried Rasthofer, Steven Arzt, Marc Miltenberger, and Eric Bodden. Harvesting runtime values in android applications that feature antianalysis techniques. In NDSS, 2016.
Dhruv Rathi and Rajni Jindal. Droidmark: A tool for android malware detection using taint analysis and bayesian network. arXiv preprint arXiv:1805.06620, 2018.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
J. Samhi, A. Bartel, T. F. Bissyande, and J. Klein. Raicc: Revealing atypical inter-component communication in android apps. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 1398-1409, Los Alamitos, CA, USA, may 2021. IEEE Computer Society.
J. Samhi, L. Li, T. F. Bissyande, and J. Klein. Difuzer: Uncovering suspicious hidden sensitive operations in android apps. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), pages 723-735, Los Alamitos, CA, USA, May 2022. IEEE Computer Society.
Jordan Samhi, Kevin Allix, Tegawendé F. Bissyandé, and Jacques Klein. A first look at android applications in google play related to covid-19. Empirical Software Engineering, 26(4):57, April 2021.
Jordan Samhi and Alexandre Bartel. On the (in)effectiveness of static logic bomb detector for android apps. IEEE Transactions on Dependable and Secure Computing, pages 1-1, August 2021.
Jordan Samhi, Jun Gao, Nadia Daoudi, Pierre Graux, Henri Hoyez, Xiaoyu Sun, Kevin Allix, Tegawendé F Bissyandé, and Jacques Klein. Jucify: A step towards android code unification for enhanced static analysis. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), pages 1232-1244, Los Alamitos, CA, USA, May 2022. IEEE Computer Society.
Golam Sarwar, Olivier Mehani, Roksana Boreli, and Mohamed Ali Kaafar. On the effectiveness of dynamic taint analysis for protecting against private information leaks on android-based devices. In SECRYPT, volume 96435, 2013.
Venkatesh Gauri Shankar, Gaurav Somani, Manoj Singh Gaur, Vijay Laxmi, and Mauro Conti. Androtaint: An efficient android malware detection framework using dynamic taint analysis. In 2017 ISEA Asia Security and Privacy (ISEASP), pages 1-13, 2017.
Sagar Sharma, Simone Sharma, and Anidhya Athaiya. Activation functions in neural networks. towards data science, 6(12):310-316, 2017.
Dawei Shi, Xiucun Tang, and Zhibin Ye. Detecting environmentsensitive malware based on taint analysis. In 2017 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), pages 322-327, 2017.
M. Stone. Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society. Series B (Methodological), 36(2):111-147, 1974.
F. Tomassetti. Javaparser, https://github.com/javaparser/javaparser. Accessed August 2021.
Victor Van Der Veen, Herbert Bos, and Christian Rossow. Dynamic analysis of android malware. Internet & Web Technology Master thesis, VU University Amsterdam, 2013.
Weiping Wang, Jianjian Wei, Shigeng Zhang, and Xi Luo. Lscdroid: Malware detection based on local sensitive api invocation sequences. IEEE Transactions on Reliability, 69(1):174-187, 2020.
Fengguo Wei, Sankardas Roy, Xinming Ou, and Robby. Amandroid: A precise and general inter-component data flow analysis framework for security vetting of android apps. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS '14, page 1329-1341, New York, NY, USA, 2014. Association for Computing Machinery.
Nattanon Wongwiwatchai, Phannawhat Pongkham, and Kunwadee Sripanidkulchai. Comprehensive detection of vulnerable personal information leaks in android applications. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pages 121-126, 2020.
Songyang Wu, Pan Wang, Xun Li, and Yong Zhang. Effective detection of android malware based on the usage of data flow apis and machine learning. Information and software technology, 75:17-25, 2016.
Z. Yang and M. Yang. Leakminer: Detect information leakage on android with static taint analysis. In 2012 Third World Congress on Software Engineering, pages 101-104, 2012.
Zhemin Yang, Min Yang, Yuan Zhang, Guofei Gu, Peng Ning, and X Sean Wang. Appintent: Analyzing sensitive data transmission in android for privacy leakage detection. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security, pages 1043-1054, 2013.
Sergio Yovine and Gonzalo Winniczuk. Checkdroid: A tool for automated detection of bad practices in android applications using taint analysis. In 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pages 175-176, 2017.
Mu Zhang, Yue Duan, Heng Yin, and Zhiruo Zhao. Semantics-aware android malware classification using weighted contextual api dependency graphs. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS '14, page 1105-1116, New York, NY, USA, 2014. Association for Computing Machinery.
Mu Zhang and Heng Yin. Appsealer: Automatic generation of vulnerability-specific patches for preventing component hijacking attacks in android applications. In NDSS. Citeseer, 2014.
Qingchuan Zhao, Chaoshun Zuo, Brendan Dolan-Gavitt, Giancarlo Pellegrino, and Zhiqiang Lin. Automatic uncovering of hidden behaviors from input validation in mobile apps. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1106-1120. IEEE, 2020.
Hao Zhou, Wei Zhang, Fengqiong Wei, and Yunfang Chen. Analysis of android malware family characteristic based on isomorphism of sensitive api call graph. In 2017 IEEE Second International Conference on Data Science in Cyberspace (DSC), pages 319-327, 2017.