L. Ma, F. Zhang, J. Sun, M. Xue, B. Li, F. Juefei-Xu, C. Xie, L. Li, Y. Liu, J. Zhao, and Y. Wang, "Deepmutation: mutation testing of deep learning systems, " in 29th International Symposium on Software Reliability Engineering (ISSRE). Los Alamitos, CA, USA: IEEE Computer Society, October 2018, pp. 100-111. [Online]. Available: https: //doi. ieeecomputersociety. org/10. 1109/ISSRE. 2018. 00021
Q. Hu, L. Ma, X. Xie, B. Yu, Y. Liu, and J. Zhao, "Deepmutation++: A mutation testing framework for deep learning systems, " in 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), San Diego, CA, USA, December 2019, pp. 1158-1161.
A. Panichella and C. C. Liem, "What are we really testing in mutation testing for machine learning a critical reflection, " ser. ICSE-NIER '21. IEEE Press, 2021, pp. 66-70. [Online]. Available: https: //doi-org. proxy. bnl. lu/10. 1109/ICSE-NIER52604. 2021. 00022
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks, " in 6th International Conference on Learning Representations (ICLR), Vancouver Convention Center, Vancouver, BC, Canada, April 2018. [Online]. Available: https: //openreview. net/forum?id=rJzIBfZAb
N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, "Practical black-box attacks against machine learning, " in Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, April 2017, pp. 506-519.
Y. Liu, X. Chen, C. Liu, and D. Song, "Delving into transferable adversarial examples and black-box attacks, " in 5th International Conference on Learning Representations (ICLR), Toulon, France, April 2017. [Online]. Available: https: //openreview. net/forum?id=Sys6GJqxl
S. Kornblith, M. Norouzi, H. Lee, and G. Hinton, "Similarity of neural network representations revisited, " in 36 th International Conference on Machine Learning (ICML). Long Beach, California: PMLR, 2019. [Online]. Available: https: //proceedings. mlr. press/v97/ kornblith19a/kornblith19a. pdf
C. B. Moler, Experiments with MATLAB. Society for Industrial and Applied Mathematics, 2011.
N. Carlini and D. Wagner, "Towards evaluating the robustness of neural networks, " in IEEE Symposium on Security and Privacy (SP). Los Alamitos, CA, USA: IEEE Computer Society, May 2017. [Online]. Available: https: //doi. ieeecomputersociety. org/10. 1109/SP. 2017. 49
J. Wang, G. Dong, J. Sun, X. Wang, and P. Zhang, "Adversarial sample detection for deep neural network through model mutation testing, " in ICSE, ser. ICSE '19. IEEE Press, May, p. 1245-1256. [Online]. Available: https: //doi-org. proxy. bnl. lu/10. 1109/ICSE. 2019. 00126
J. Chen, Z. Wu, Z. Wang, H. You, L. Zhang, and M. Yan, "Practical accuracy estimation for efficient deep neural network testing, " ACM Transactions on Software Engineering and Methodology, vol. 29, no. 4, October 2020. [Online]. Available: https: //doi-org. proxy. bnl. lu/10. 1145/ 3394112
N. Demir, "Ensemble methods: elegant techniques to produce improved machine learning results, " https: //www. kdnuggets. com/2016/02/ ensemble-methods-techniques-produce-improved-machine-learning. html/2, 2016.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, November 1998.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, "Reading digits in natural images with unsupervised feature learning, " in NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011. [Online]. Available: http: //ufldl. stanford. edu/housenumbers/nips2011 housenumbers. pdf
A. Krizhevsky, "Learning multiple layers of features from tiny images, " University of Toronto, Toronto, Tech. Rep., 2009.
A. Athalye, N. Carlini, and D. Wagner, "Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples, " in ICML, vol. 80. Stockholmsmässan, Stockholm Sweden: PMLR, July 2018, pp. 274-283.
M.-I. Nicolae, M. Sinn, M. N. Tran, B. Buesser, A. Rawat, M. Wistuba, V. Zantedeschi, N. Baracaldo, B. Chen, H. Ludwig, I. Molloy, and B. Edwards, "Adversarial robustness toolbox v1. 2. 0, " CoRR, vol. 1807. 01069, 2018.
C. Guo, M. Rana, M. Cisse, and L. van der Maaten, "Countering adversarial images using input transformations, " in 6th International Conference on Learning Representations (ICLR), Vancouver Convention Center, Vancouver, BC, Canada, April 2018. [Online]. Available: https: //openreview. net/pdf?id=SyJ7ClWCb
N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, "Distillation as a defense to adversarial perturbations against deep neural networks, " in IEEE Symposium on Security and Privacy (SP). Los Alamitos, CA, USA: IEEE Computer Society, May 2016, pp. 582-597. [Online]. Available: https: //doi. ieeecomputersociety. org/10. 1109/SP. 2016. 41
D. Warde-Farley, "1 adversarial perturbations of deep neural networks, " 2016. [Online]. Available: https: //api. semanticscholar. org/CorpusID: 28912221
