[en] Recently, convolutional neural networks (CNNs) have become the main drivers in many image recognition applications. However, they are vulnerable to adversarial attacks, which can lead to disastrous consequences. This paper introduces ShuffleDetect as a new and efficient unsupervised method for the detection of adversarial images against trained convolutional neural networks. Its main feature is to split an input image into non-overlapping patches, then swap the patches according to permutations, and count the number of permutations for which the CNN classifies the unshuffled input image and the shuffled image into different categories. The image is declared adversarial if and only if the proportion of such permutations exceeds a certain threshold value. A series of 8 targeted or untargeted attacks was applied on 10 diverse and state-of-the-art ImageNet-trained CNNs, leading to 9500 relevant clean and adversarial images. We assessed the performance of ShuffleDetect intrinsically and compared it with another detector. Experiments show that ShuffleDetect is an easy-to-implement, very fast, and near memory-free detector that achieves high detection rates and low false positive rates.
Research center :
ULHPC - University of Luxembourg: High Performance Computing
Disciplines :
Computer science
Author, co-author :
Chitic, Ioana Raluca ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
Topal, Ali Osman ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
Leprevost, Franck ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
External co-authors :
yes
Language :
English
Title :
ShuffleDetect: Detecting Adversarial Images against Convolutional Neural Networks
Touvron H. Cord M. Douze M. Massa F. Sablayrolles A. Jégou H. Training data-efficient image transformers & distillation through attention Proceedings of the International Conference on Machine Learning, PMLR Virtual 18–24 July 2021 10347 10357
Chakraborty A. Alam M. Dey V. Chattopadhyay A. Mukhopadhyay D. Adversarial Attacks and Defences: A Survey arXiv 2018 1810.00069 10.1049/cit2.12028
Kurakin A. Goodfellow I.J. Bengio S. Adversarial examples in the physical world Artificial Intelligence Safety and Security Chapman and Hall/CRC Boca Raton, FL, USA 2018 99 112
Goodfellow I.J. Shlens J. Szegedy C. Explaining and Harnessing Adversarial Examples arXiv 2015 1810.00069
Carlini N. Wagner D. Towards evaluating the robustness of neural networks Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), IEEE San Jose, CA, USA 22–26 May 2017 39 57
Wiyatno R. Xu A. Maximal Jacobian-based Saliency Map Attack arXiv 2018 1808.07945
Tramèr F. Papernot N. Goodfellow I. Boneh D. McDaniel P. The Space of Transferable Adversarial Examples arXiv 2017 1704.03453
Liu Y. Chen X. Liu C. Song D. Delving into Transferable Adversarial Examples and Black-box Attacks arXiv 2016 1611.02770
Ilyas A. Engstrom L. Athalye A. Lin J. Black-box adversarial attacks with limited queries and information Proceedings of the International Conference on Machine Learning, PMLR Stockholm, Sweden 10–15 July 2018 2137 2146
Narodytska N. Kasiviswanathan S.P. Simple Black-Box Adversarial Perturbations for Deep Networks arXiv 2016 1612.06299
Feinman R. Curtin R.R. Shintre S. Gardner A.B. Detecting adversarial samples from artifacts arXiv 2017 1703.00410
Grosse K. Manoharan P. Papernot N. Backes M. McDaniel P. On the (statistical) detection of adversarial examples arXiv 2017 1702.06280
Xu W. Evans D. Qi Y. Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks arXiv 2017 1704.01155
Liang B. Li H. Su M. Li X. Shi W. Wang X. Detecting adversarial image examples in deep neural networks with adaptive noise reduction IEEE Trans. Dependable Secure Comput. 2018 18 72 85 10.1109/TDSC.2018.2874243
Van Rossum G. Drake F.L. Python 3 Reference Manual CreateSpace Scotts Valley, CA, USA 2009
Oliphant T.E. A Guide to NumPy Trelgol Publishing Austin, TX, USA 2006
Paszke A. Gross S. Massa F. Lerer A. Bradbury J. Chanan G. Killeen T. Lin Z. Gimelshein N. Antiga L. et al. Pytorch: An imperative style, high-performance deep learning library Adv. Neural Inf. Process. Syst. 2019 32 8026 8037
Varrette S. Bouvry P. Cartiaux H. Georgatos F. Management of an Academic HPC Cluster: The UL Experience Proceedings of the 2014 International Conference on High Performance Computing & Simulation (HPCS 2014), IEEE Bologna, Italy 21–25 July 2014 959 967
Topal A.O. Chitic R. Leprévost F. One evolutionary algorithm deceives humans and ten convolutional neural networks trained on ImageNet at image recognition ASC 2022 under review
Aldahdooh A. Hamidouche W. Fezza S.A. Déforges O. Adversarial example detection for DNN models: A review and experimental comparison Artif. Intell. Rev. 2022 55 4403 4462 10.1007/s10462-021-10125-w
Ma X. Li B. Wang Y. Erfani S.M. Wijewickrema S.N.R. Houle M.E. Schoenebeck G. Song D. Bailey J. Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality arXiv 2018 1801.02613
Ma S. Liu Y. Nic: Detecting adversarial samples with neural network invariant checking Proceedings of the 26th Network and Distributed System Security Symposium (NDSS 2019) San Diego, CA, USA 24–27 February 2019
Chitic R. Topal A.O. Leprévost F. Empirical Perturbation Analysis of Two Adversarial Attacks: Black Box versus White Box Appl. Sci. 2022 12 7339 10.3390/app12147339
Leprévost F. How Big is Big? How Fast is Fast? A Hands—On Tutorial on Mathematics of Computation Amazon New York, NY, USA 2020
Deng J. Dong W. Socher R. Li L.J. Li K. Li F.-F. The ImageNet Image Database 2009 Available online: http://image-net.org (accessed on 20 September 2022)
Simonyan K. Zisserman A. Very deep convolutional networks for large-scale image recognition arXiv 2014 1409.1556
He K. Zhang X. Ren S. Sun J. Deep residual learning for image recognition Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 26 June–1 July 2016 770 778
Huang G. Liu Z. Van Der Maaten L. Weinberger K.Q. Densely connected convolutional networks Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Honolulu, HI, USA 21–26 July 2017 4700 4708
Howard A.G. Zhu M. Chen B. Kalenichenko D. Wang W. Weyand T. Andreetto M. Adam H. Mobilenets: Efficient convolutional neural networks for mobile vision applications arXiv 2017 1704.04861
Tan M. Chen B. Pang R. Vasudevan V. Le Q.V. MnasNet: Platform-Aware Neural Architecture Search for Mobile arXiv 2018 1807.11626
Agrafiotis D. Chapter 9—Video Error Concealment Academic Press Library in Signal Processing Theodoridis S. Chellappa R. Elsevier Amsterdam, The Netherlands 2014 Volume 5 295 321 10.1016/B978-0-12-420149-1.00009-0
Nicolae M. Sinn M. Minh T.N. Rawat A. Wistuba M. Zantedeschi V. Molloy I.M. Edwards B. Adversarial Robustness Toolbox v1.0.0 arXiv 2018 1807.01069
Madry A. Makelov A. Schmidt L. Tsipras D. Vladu A. Towards Deep Learning Models Resistant to Adversarial Attacks arXiv 2019 1706.06083
Moosavi-Dezfooli S.M. Fawzi A. Frossard P. Deepfool: A simple and accurate method to fool deep neural networks Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 26 June–1 July 2016 2574 2582
Meng D. Chen H. Magnet: A two-pronged defense against adversarial examples Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security Abu Dhabi, United Arab Emirates 2–6 April 2017 135 147
