[en] In the past years, RGB-based deepfake detection has shown notable progress thanks to the development of effective deep neural networks. However, the performance of deepfake detectors remains primarily dependent on the quality of the forged content and the level of artifacts introduced by the forgery method. To detect these artifacts, it is often necessary to separate and analyze the frequency components of an image. In this context, we propose to utilize the high-frequency components of color images by introducing an end-to-end trainable module that (a) extracts features from high-frequency components and (b) fuses them with the features of the RGB input. The module not only exploits the high-frequency anomalies present in manipulated images but also can be used with most RGB-based deepfake detectors. Experimental results show that the proposed approach boosts the performance of state-of-the-art networks,
such as XceptionNet and EfficientNet, on a challenging deepfake dataset.
Centre de recherche :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Computer Vision Imaging & Machine Intelligence (CVI²)
Disciplines :
Sciences informatiques
Auteur, co-auteur :
MEJRI, Nesryne ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > CVI2
PAPADOPOULOS, Konstantinos ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
AOUADA, Djamila ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
Co-auteurs externes :
no
Langue du document :
Anglais
Titre :
Leveraging High-Frequency Components for Deepfake Detection
Paris, Britt, and Donovan, Joan. "Deepfakes and Cheap Fakes. " Data & Society, September 18, 2019.
C. Szegedy et al., "Going deeper with convolutions, " in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
M. Tan and Q. V. Le, "EfficientNet: Rethinking model scaling for convolutional Neural Networks, " arXiv [cs. LG], 2019.
F. Chollet, "Xception: Deep learning with depthwise separable convolutions, " in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
F. Juefei-Xu, R. Wang, Y. Huang, Q. Guo, L. Ma, and Y. Liu, "Countering malicious DeepFakes: Survey, battleground, and horizon, " arXiv [cs. CV], 2021.
I. Perov et al., "DeepFaceLab: A simple, flexible and extensible face swapping framework, " arXiv [cs. CV], 2020.
Y. Mirsky and W. Lee, "The creation and detection of deepfakes: A survey, " ACM Comput. Surv., vol. 54, no. 1, pp. 1-41, 2021.
D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, "MesoNet: A compact facial video forgery detection network, " in 2018 IEEE International Workshop on Information Forensics and Security (WIFS), 2018.
P. Zhou, X. Han, V. I. Morariu, and L. S. Davis, "Two-stream neural networks for tampered face detection, " in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017.
X. Zhang, S. Karaman, and S.-F. Chang, "Detecting and simulating artifacts in GAN fake images, " in 2019 IEEE International Workshop on Information Forensics and Security (WIFS), 2019.
R. Tolosana, S. Romero-Tapiador, J. Fierrez, and R. Vera-Rodriguez, "DeepFakes evolution: Analysis of facial regions and fake detection performance, " in Pattern Recognition. ICPR International Workshops and Challenges, Cham: Springer International Publishing, 2021, pp. 442-456.
M. Du, S. Pentyala, Y. Li, and X. Hu, "Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder, " in Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020.
A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Niessner, "FaceForensics++: Learning to detect manipulated facial images, " in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, "Celeb-DF: A largescale challenging dataset for DeepFake forensics, " in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
B. Dolhansky et al., "The DeepFake Detection Challenge (DFDC) Dataset, " arXiv [cs. CV], 2020.
Y. Zhang, L. Zheng, and V. L. L. Thing, "Automated face swapping and its detection, " in 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), 2017.
H. Bay, T. Tuytelaars, and L. Van Gool, "SURF: Speeded Up Robust Features, " in Computer Vision-ECCV 2006, Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 404-417.
Y. Li and S. Lyu, "Exposing DeepFake videos by detecting face warping artifacts, " arXiv [cs. CV], 2018.
L. Li et al., "Face X-ray for more general face forgery detection, " in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
F. Marra, D. Gragnaniello, L. Verdoliva, and G. Poggi, "Do GANs Leave Artificial Fingerprints?, " in 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), 2019.
N. Yu, L. Davis, and M. Fritz, "Attributing fake images to GANs: Learning and analyzing GAN fingerprints, " in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
H. Wang, X. Wu, Z. Huang, and E. P. Xing, "High-frequency component helps explain the generalization of convolutional neural networks, " in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
R. Durall, M. Keuper, F.-J. Pfreundt, and J. Keuper, "Unmasking DeepFakes with simple Features, " arXiv [cs. LG], 2019.
Z. Guo, G. Yang, J. Chen, and X. Sun, "Fake face detection via adaptive manipulation traces extraction network, " Comput. Vis. Image Underst., vol. 204, no. 103170, p. 103170, 2021.
P. Zhou, X. Han, V. I. Morariu, and L. S. Davis, "Learning rich features for image manipulation detection, " in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
V. Conotter, E. Bodnari, G. Boato, and H. Farid, "Physiologicallybased detection of computer generated faces in video, " in 2014 IEEE International Conference on Image Processing (ICIP), 2014.
U. A. Ciftci, I. Demir, and L. Yin, "FakeCatcher: Detection of synthetic portrait videos using biological signals, " IEEE Trans. Pattern Anal. Mach. Intell., vol. PP, pp. 1-1, 2020.
U. A. Ciftci, I. Demir, and L. Yin, "How do the hearts of deep fakes beat? Deep fake source detection via interpreting residuals with biological signals, " in 2020 IEEE International Joint Conference on Biometrics (IJCB), 2020.
Selim Seferbekov. https://github. com/selimsef/dfdc deepfake challenge.
N. Bonettini, E. D. Cannas, S. Mandelli, L. Bondi, P. Bestagini, and S. Tubaro, "Video face manipulation detection through ensemble of CNNs, " arXiv [cs. CV], 2020.
K. Xu, M. Qin, F. Sun, Y. Wang, Y.-K. Chen, and F. Ren, "Learning in the frequency domain, " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Y. Qian, G. Yin, L. Sheng, Z. Chen, and J. Shao, "Thinking in frequency: Face forgery detection by mining frequency-aware clues, " in Computer Vision-ECCV 2020, Cham: Springer International Publishing, 2020, pp. 86-103.
J. Frank, T. Eisenhofer, L. Schönherr, A. Fischer, D. Kolossa, and T. Holz, "Leveraging frequency analysis for deep fake image recognition, " arXiv [cs. CV], 2020.
H. Liu et al., "Spatial-Phase Shallow Learning: Rethinking face forgery detection in frequency domain, " arXiv [cs. CV], 2021.
S.-Y. Wang, O. Wang, R. Zhang, A. Owens, and A. A. Efros, "CNNgenerated images are surprisingly easy to spot... for now, " arXiv [cs. CV], 2019.
Y. Luo, Y. Zhang, J. Yan, and W. Liu, "Generalizing face forgery detection with high-frequency features, " arXiv [cs. CV], 2021.
I. Masi, A. Killekar, R. M. Mascarenhas, S. P. Gurudatt, and W. AbdAlmageed, "Two-branch recurrent network for isolating deepfakes in videos, " in Computer Vision-ECCV 2020, Cham: Springer International Publishing, 2020, pp. 667-684.
V. Bazarevsky, Y. Kartynnik, A. Vakunov, K. Raveendran, and M. Grundmann, "BlazeFace: Sub-millisecond neural face detection on mobile GPUs, " arXiv [cs. CV], 2019.
A. Buslaev, V. I. Iglovikov, E. Khvedchenya, A. Parinov, M. Druzhinin, and A. A. Kalinin, "Albumentations: Fast and flexible image augmentations, " Information (Basel), vol. 11, no. 2, p. 125, 2020.
A. Paszke et al., "PyTorch: An imperative style, high-performance deep learning library, " arXiv [cs. LG], 2019.
D. Kingma and J. Ba, "Adam: a method for stochastic optimization. arxiv: 14126980, " 2014.
J. Deng, W. Dong, R. Socher, L. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database, " 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248-255, doi: 10. 1109/CVPR. 2009. 5206848.
P. Korshunov and S. Marcel, "Deepfake detection: humans vs. machines, " arXiv [cs. CV], 2020.
Y. Nirkin, L. Wolf, Y. Keller, and T. Hassner, "DeepFake detection based on the discrepancy between the face and its context, " arXiv [cs. CV], 2020.
H. Qi et al., "DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms, " in Proceedings of the 28th ACM International Conference on Multimedia, 2020.
A. Kumar, A. Bhavsar, and R. Verma, "Detecting deepfakes with metric learning, " in 2020 8th International Workshop on Biometrics and Forensics (IWBF), 2020.
H. Zhao, W. Zhou, D. Chen, T. Wei, W. Zhang, and N. Yu, "Multiattentional Deepfake Detection, " arXiv [cs. CV], 2021.
