UNTAG: Learning Generic Features for Unsupervised Type-Agnostic Deepfake Detection

in IEEE International Conference on Acoustics, Speech and Signal Processing. Proceedings (2023)

This paper introduces a novel framework for unsupervised type-agnostic deepfake detection called UNTAG. Existing methods are generally trained in a supervised manner at the classification level, focusing ... [more ▼]

This paper introduces a novel framework for unsupervised type-agnostic deepfake detection called UNTAG. Existing methods are generally trained in a supervised manner at the classification level, focusing on detecting at most two types of forgeries; thus, limiting their generalization capability across different deepfake types. To handle that, we reformulate the deepfake detection problem as a one-class classification supported by a self-supervision mechanism. Our intuition is that by estimating the distribution of real data in a discriminative feature space, deepfakes can be detected as outliers regardless of their type. UNTAG involves two sequential steps. First, deep representations are learned based on a self-supervised pretext task focusing on manipulated regions. Second, a one-class classifier fitted on authentic image embeddings is used to detect deepfakes. The results reported on several datasets show the effectiveness of UNTAG and the relevance of the proposed new paradigm. The code is publicly available. [less ▲]

Face-swap Deepfake Detection Using High-frequency Components

Bachelor/master dissertation (2021)

Over the last years, very deceitful deepfakes applied to human visuals appeared on theInternet. Given their impressive visual quality, they are nowadays considered a potentialthreat for both individuals ... [more ▼]

Over the last years, very deceitful deepfakes applied to human visuals appeared on theInternet. Given their impressive visual quality, they are nowadays considered a potentialthreat for both individuals and organizations. Hence, researchers started investigatingthe flaws of deepfakes to develop automated tools capable of detecting forged content. Asa result, a wide range of deepfake detection methods has been introduced. In particular,deep learning based-approaches have shown impressive performance. Nevertheless, thesemethods are still not sufficiently robust as they usually consider only one type of artifact,either in the spatial or the frequency domain. In this context, this thesis proposes toleverage the high-frequency components extracted from color images jointly with theoriginal color information to detect unusual traces. It introduces an end-to-end trainablemodule that (a) extracts features from precomputed high-frequency components and (b)fuses them with RGB features. The deepfake detection framework not only exploitsthe high-frequency anomalies present in manipulated images but can also be integratedwith the majority of RGB-based deepfake detectors. Experimental results show thatthe proposed approach improves the performance of state-of-the-art networks, such asXceptionNet and EfficientNet, on a challenging deepfake dataset called Celeb-DF. [less ▲]