[en] Most existing anomaly detection (AD) methods require a dedicated model for each category. Such a paradigm, despite its promising results, is computationally expensive and inefficient, thereby failing to meet the requirements for real-world applications. Inspired by how humans detect anomalies, by comparing a query image to known normal ones, this article proposes a novel few-shot AD (FSAD) framework. Using a training set of normal images from various categories, registration, aiming to align normal images of the same categories, is leveraged as the proxy task for self-supervised category-agnostic representation learning. At test time, an image and its corresponding support set, consisting of a few normal images from the same category, are supplied, and anomalies are identified by comparing the registered features of the test image to its corresponding support image features. Such a setup enables the model to generalize to novel test categories. It is, to our best knowledge, the first FSAD method that requires no model fine-tuning for novel categories: enabling a single model to be applied to all categories. Extensive experiments demonstrate the effectiveness of the proposed method. Particularly, it improves the current state-of-the-art (SOTA) for FSAD by 11.3% and 8.3% on the MVTec and MPDD benchmarks, respectively. The source code is available at https://github.com/Haoyan-Guan/CAReg.
Disciplines :
Computer science
Author, co-author :
Huang, Chaoqin ; Shanghai Jiao Tong University, Cooperative Medianet Innovation Center, Shanghai, China ; National University of Singapore, Department of Electrical and Computer Engineering, Singapore ; National University of Defense Technology, College of Information Communications, Wuhan, China
Guan, Haoyan ; King’s College London, Department of Informatics, London, United Kingdom
Jiang, Aofan; Shanghai Jiao Tong University, Cooperative Medianet Innovation Center, Shanghai, China ; Shanghai Artificial Intelligence Laboratory, Shanghai, China
Zhang, Ya ; Shanghai Jiao Tong University, Cooperative Medianet Innovation Center, Shanghai, China ; Shanghai Artificial Intelligence Laboratory, Shanghai, China
SPRATLING, Michael ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS) > Cognitive Science and Assessment ; King’s College London, Department of Informatics, London, United Kingdom
Wang, Xinchao; National University of Singapore, Department of Electrical and Computer Engineering, Singapore
Wang, Yanfeng ; Shanghai Jiao Tong University, Cooperative Medianet Innovation Center, Shanghai, China ; Shanghai Artificial Intelligence Laboratory, Shanghai, China
External co-authors :
yes
Language :
English
Title :
Few-Shot Anomaly Detection via Category-Agnostic Registration Learning.
Publication date :
July 2025
Journal title :
IEEE Transactions on Neural Networks and Learning Systems
ISSN :
2162-237X
eISSN :
2162-2388
Publisher :
Institute of Electrical and Electronics Engineers Inc., United States
National Key Research and Development Program of China Science and Technology Commission of Shanghai Municipality State Key Laboratory of UHD Video and Audio Production and Presentation Ministry of Education Singapore, under its Academic Research Fund Tier 2, under Award National Research Foundation, Singapore, under its AI Singapore Program (AISG), under Award
Funding text :
This work was supported in part by the National Key Research and Development Program of China under Grant 2022ZD0160702; in part by the STCSM under Grant 22DZ2229005; in part by the 111 Plan under Grant BP0719010; in part by the State Key Laboratory of UHD Video and Audio Production and Presentation; in part by the Ministry of Education Singapore, under its Academic Research Fund Tier 2, under Award MOE-T2EP20122-0006; and in part by the National Research Foundation, Singapore, under its AI Singapore Program (AISG), under Award AISG2-RP-2021023.Received 22 January 2024; revised 13 June 2024; accepted 10 September 2024. This work was supported in part by the National Key Research and Development Program of China under Grant 2022ZD0160702; in part by the STCSM under Grant 22DZ2229005; in part by the 111 Plan under Grant BP0719010; in part by the State Key Laboratory of UHD Video and Audio Production and Presentation; in part by the Ministry of Education Singapore, under its Academic Research Fund Tier 2, under Award MOE-T2EP20122-0006; and in part by the National Research Foundation, Singapore, under its AI Singapore Program (AISG), under Award AISG2-RP-2021023. (Chaoqin Huang and Haoyan Guan contributed equally to this work.) (Corresponding authors: Yanfeng Wang; Ya Zhang.) Chaoqin Huang was with the Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China, and also with the Department of Electrical and Computer Engineering, National University of Singapore, Singapore 119077. He is now with the College of Information Communications, National University of Defense Technology, Wuhan 430030, China (e-mail: huangchaoqin@nudt.edu.cn).
T. Matsubara, R. Tachibana, and K. Uehara, “Anomaly machine component detection by deep generative model with unregularized score,” in Proc. Int. Joint Conf. Neural Netw. (IJCNN), Jul. 2018, pp. 1–8.
K. Eykholt et al., “Robust physical-world attacks on deep learning visual classification,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 1625–1634.
Y. Tian, Y. Liu, G. Pang, F. Liu, Y. Chen, and G. Carneiro, “Pixel-wise energy-biased abstention learning for anomaly segmentation on complex urban driving scenes,” in Proc. ECCV. Cham, Switzerland: Springer, 2022, pp. 246–263.
J. Zhang et al., “Viral pneumonia screening on chest X-rays using confidence-aware anomaly detection,” IEEE Trans. Med. Imag., vol. 40, no. 3, pp. 879–890, Mar. 2021.
J. Su, H. Shen, L. Peng, and D. Hu, “Few-shot domain-adaptive anomaly detection for cross-site brain images,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 3, pp. 1819–1835, Mar. 2024.
B. Zong et al., “Deep autoencoding Gaussian mixture model for unsupervised anomaly detection,” in Proc. ICLR, 2018.
D. Gong et al., “Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2019, pp. 1705–1714.
J.-C. Wu, D.-J. Chen, C.-S. Fuh, and T.-L. Liu, “Learning unsupervised metaformer for anomaly detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Oct. 2021, pp. 4369–4378.
I. Golan and R. El-Yaniv, “Deep anomaly detection using geometric transformations,” in Proc. NeurIPS, vol. 31, 2018, pp. 9758–9769.
F. Ye, C. Huang, J. Cao, M. Li, Y. Zhang, and C. Lu, “Attribute restoration framework for anomaly detection,” IEEE Trans. Multimedia, vol. 24, pp. 116–127, 2022.
M. Salehi, N. Sadjadi, S. Baselizadeh, M. H. Rohban, and H. R. Rabiee, “Multiresolution knowledge distillation for anomaly detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 14902–14912.
Y. Zheng, X. Wang, R. Deng, T. Bao, R. Zhao, and L. Wu, “Focus your distribution: Coarse-to-fine non-contrastive learning for anomaly detection and localization,” in Proc. IEEE Int. Conf. Multimedia Expo. (ICME), Jul. 2022, pp. 1–6.
B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, “Estimating the support of a high-dimensional distribution,” Neural Comput., vol. 13, no. 7, pp. 1443–1471, Jul. 2001.
J. Yi and S. Yoon, “Patch SVDD: Patch-level SVDD for anomaly detection and segmentation,” in Proc. Asian Conf. Comput. Vis. (ACCV), Nov. 2020, pp. 375–390.
S. Sheynin, S. Benaim, and L. Wolf, “A hierarchical transformation-discriminating generative model for few shot anomaly detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 8495–8504.
M. Rudolph, B. Wandt, and B. Rosenhahn, “Same same but DifferNet: Semi-supervised defect detection with normalizing flows,” in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2021, pp. 1907–1916.
L. G. Brown, “A survey of image registration techniques,” ACM Comput. Surv., vol. 24, no. 4, pp. 325–376, Dec. 1992.
B. Zitová and J. Flusser, “Image registration methods: A survey,” Image Vis. Comput., vol. 21, no. 11, pp. 977–1000, Oct. 2003.
H. Peng et al., “BrainAligner: 3D registration atlases of Drosophila brains,” Nature Methods, vol. 8, no. 6, pp. 493–498, Jun. 2011.
X. Chen and K. He, “Exploring simple Siamese representation learning,” in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., Jun. 2021, pp. 15750–15758.
M. Jaderberg et al., “Spatial transformer networks,” in Proc. NeurIPS, vol. 28, 2015, pp. 2017–2025.
T. Defard, A. Setkov, A. Loesch, and R. Audigier, “PaDiM: A patch distribution modeling framework for anomaly detection and localization,” in Proc. Int. Conf. Pattern Recognit. Cham, Switzerland: Springer, Jan. 2021, pp. 475–489.
P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, “MVTec AD—A comprehensive real-world dataset for unsupervised anomaly detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 9584–9592.
S. Jezek, M. Jonak, R. Burget, P. Dvorak, and M. Skotak, “Deep learning-based defect detection of metal parts: Evaluating current methods in complex conditions,” in Proc. 13th Int. Congr. Ultra Modern Telecommun. Control Syst. Workshops (ICUMT), 2021, pp. 66–71.
J.-H. Kim, D.-H. Kim, S. Yi, and T. Lee, “Semi-orthogonal embedding for efficient unsupervised anomaly segmentation,” 2021, arXiv:2105.14737.
K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox, and P. Gehler, “Towards total recall in industrial anomaly detection,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2022, pp. 14318–14328.
C. Huang, H. Guan, A. Jiang, Y. Zhang, M. Spratling, and Y.-F. Wang, “Registration based few-shot anomaly detection,” in Proc. Eur. Conf. Comput. Vis., 2022, pp. 303–319.
G. Pang, C. Aggarwal, C. Shen, and N. Sebe, “Editorial deep learning for anomaly detection,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 6, pp. 2282–2286, Jun. 2022.
Z. Fang, J. Liang, J. T. Zhou, Y. Xiao, and F. Yang, “Anomaly detection with bidirectional consistency in videos,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 3, pp. 1079–1092, Mar. 2022.
B. Ramachandra, M. J. Jones, and R. R. Vatsavai, “A survey of single-scene video anomaly detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 5, pp. 2293–2312, May 2022.
X. Wang et al., “Robust unsupervised video anomaly detection by multipath frame prediction,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 6, pp. 2301–2312, Jun. 2022.
C. Huang et al., “Self-supervised attentive generative adversarial networks for video anomaly detection,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 11, pp. 9389–9403, Nov. 2023.
K. Xu, T. Sun, and X. Jiang, “Video anomaly detection and localization based on an adaptive intra-frame classification network,” IEEE Trans. Multimedia, vol. 22, no. 2, pp. 394–406, Feb. 2020.
M. Sabokrou, M. Fathy, G. Zhao, and E. Adeli, “Deep end-to-end one-class classifier,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 2, pp. 675–684, Feb. 2021.
K. Yamanishi, J.-I. Takeuchi, G. Williams, and P. Milne, “On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms,” Data Mining Knowl. Discovery, vol. 8, no. 3, pp. 275–300, May 2004.
H. Xu, C. Caramanis, and S. Sanghavi, “Robust PCA via outlier pursuit,” IEEE Trans. Inf. Theory, vol. 58, no. 5, pp. 3047–3064, May 2012.
L. Maziarka, M. Smieja, M. Sendera, L. Struski, J. Tabor, and P. Spurek, “OneFlow: One-class flow for anomaly detection based on a minimal volume region,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11, pp. 8508–8519, Nov. 2022.
E. Eskin, “Anomaly detection over noisy data using learned probability distributions,” in Proc. 17th Int. Conf. Mach. Learn. San Francisco, CA, USA: Morgan Kaufmann, 2000, pp. 255–262.
M. Rahmani and G. K. Atia, “Coherence pursuit: Fast, simple, and robust principal component analysis,” IEEE Trans. Signal Process., vol. 65, no. 23, pp. 6260–6275, Dec. 2017.
L. Ruff et al., “Deep one-class classification,” in Proc. Int. Conf. Mach. Learn., 2018, pp. 4393–4402.
M. N. Kurt, Y. Yilmaz, and X. Wang, “Real-time nonparametric anomaly detection in high-dimensional settings,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 7, pp. 2463–2479, Jul. 2021.
D. Gudovskiy, S. Ishizaka, and K. Kozuka, “CFLOW-AD: Real-time unsupervised anomaly detection with localization via conditional normalizing flows,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2022, pp. 98–107.
O. Russakovsky et al., “ImageNet large scale visual recognition challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, Dec. 2015.
C. Doersch, A. Gupta, and A. A. Efros, “Unsupervised visual representation learning by context prediction,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 1422–1430.
M. Noroozi and P. Favaro, “Unsupervised learning of visual representations by solving jigsaw puzzles,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2016, pp. 69–84.
S. Gidaris, P. Singh, and N. Komodakis, “Unsupervised representation learning by predicting image rotations,” in Proc. ICLR, 2019.
P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, “Uninformed students: Student–teacher anomaly detection with discriminative latent embeddings,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 4183–4192.
S. Wang, L. Wu, L. Cui, and Y. Shen, “Glancing at the patch: Anomaly localization with global and local feature comparison,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 254–263.
W. Luo, W. Liu, D. Lian, and S. Gao, “Future frame prediction network for video anomaly detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 11, pp. 7505–7520, Nov. 2022.
S. Wang et al., “E3 outlier: A self-supervised framework for unsupervised deep outlier detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 3, pp. 2952–2969, Mar. 2023.
Y. Xia, X. Cao, F. Wen, G. Hua, and J. Sun, “Learning discriminative reconstructions for unsupervised outlier removal,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 1511–1519.
T. Schlegl, P. Seebóck, S. M. Waldstein, U. Schmidt-Erfurth, and G. Langs, “Unsupervised anomaly detection with generative adversarial networks to guide marker discovery,” in Proc. Int. Conf. Inf. Process. Med. Imag., May 2017, pp. 146–157.
M. Sabokrou, M. Khalooei, M. Fathy, and E. Adeli, “Adversarially learned one-class classifier for novelty detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 3379–3388.
S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon, “GANomaly: Semi-supervised anomaly detection via adversarial training,” in Proc. Asian Conf. Comput. Vis. Perth, WA, Australia: Springer, Dec. 2018, pp. 622–637.
C. Huang, F. Ye, P. Zhao, Y. Zhang, Y. Wang, and Q. Tian, “ESAD: End-to-end semi-supervised anomaly detection,” in Proc. BMVC, 2022.
K. Zhou et al., “Memorizing structure-texture correspondence for image anomaly detection,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 6, pp. 2335–2349, Jun. 2022.
C. Huang, Q. Xu, Y. Wang, Y. Wang, and Y. Zhang, “Self-supervised masking for unsupervised anomaly detection and localization,” IEEE Trans. Multimedia, vol. 25, pp. 4426–4438, 2023.
S.-Y. Lo, P. Oza, and V. M. Patel, “Adversarially robust one-class novelty detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4167–4179, Apr. 2023.
W. Luo et al., “Video anomaly detection with sparse coding inspired deep neural networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 3, pp. 1070–1084, Mar. 2021.
M. I. Georgescu, R. Ionescu, F. S. Khan, M. Popescu, and M. Shah, “A background-agnostic framework with adversarial training for abnormal event detection in video,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 9, pp. 4505–4523, Apr. 2021.
I. Ahmed, T. Galoppo, X. Hu, and Y. Ding, “Graph regularized autoencoder and its application in unsupervised anomaly detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 8, pp. 4110–4124, Aug. 2022.
L. Li, J. Yan, H. Wang, and Y. Jin, “Anomaly detection of time series with smoothness-inducing sequential variational auto-encoder,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 3, pp. 1177–1191, Mar. 2021.
C.-L. Li, K. Sohn, J. Yoon, and T. Pfister, “CutPaste: Self-supervised learning for anomaly detection and localization,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 9664–9674.
J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in Proc. NeurIPS, vol. 30, 2017, pp. 4080–4090.
H. Guan and M. Spratling, “Query semantic reconstruction for background in few-shot segmentation,” Vis. Comput., vol. 40, no. 2, pp. 799–810, Feb. 2024.
J. Willes, J. Harrison, A. Harakeh, C. Finn, M. Pavone, and S. Waslander, “Bayesian embeddings for few-shot open world recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 3, pp. 1–16, Mar. 2024.
S. Yang, S. Wu, T. Liu, and M. Xu, “Bridging the gap between few-shot and many-shot learning via distribution calibration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 12, pp. 9830–9843, Dec. 2022.
P. Rewatbowornwong, N. Tritrong, and S. Suwajanakorn, “Repurposing GANs for one-shot semantic part segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 5114–5125, Apr. 2023.
S. Yang, L. Liu, and M. Xu, “Free lunch for few-shot learning: Distribution calibration,” in Proc. ICLR, 2021.
Z. Li, H. Tang, Z. Peng, G.-J. Qi, and J. Tang, “Knowledge-guided semantic transfer network for few-shot image recognition,” IEEE Trans. Neural Netw. Learn. Syst., early access, Feb. 6, 2023, doi: 10.1109/TNNLS.2023.3240195.
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proc. 34th Int. Conf. Mach. Learn., vol. 70, Aug. 2017, pp. 1126–1135.
H.-G. Jung and S.-W. Lee, “Few-shot learning with geometric constraints,” IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 11, pp. 4660–4672, Nov. 2020.
J. Y. Lee, W. Lee, J. Choi, Y. Lee, and Y. S. Yoon, “Few-shot anomaly detection with adversarial loss for robust feature representations,” in Proc. BMVC, 2023.
Z. Gu, T. Yang, and L. Ma, “Prototype-aware contrastive knowledge distillation for few-shot anomaly detection,” in Proc. BMVC, 2023.
Y. Li, A. G. David, F. Liu, and C.-S. Foo, “PromptAD: Zero-shot anomaly detection using text prompts,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), vol. 2, Jan. 2024, pp. 1082–1091.
A. Frikha, D. Krompaß, H.-G. Köpken, and V. Tresp, “Few-shot one-class classification via meta-learning,” in Proc. AAAI, 2021, vol. 35, no. 8, pp. 7448–7456.
Y. Sun, C. Guo, and Y. Li, “ReAct: Out-of-distribution detection with rectified activations,” in Proc. NeurIPS, vol. 34, 2021, pp. 144–157.
Y. Sun, Y. Ming, X. Zhu, and Y. Li, “Out-of-distribution detection with deep nearest neighbors,” in Proc. Int. Conf. Mach. Learn., 2022, pp. 20827–20840.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp. 770–778.
J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a Siamese time delay neural network,” in Proc. NeurIPS, vol. 6, 1993, pp. 737–744.
F. Mezzadri, “How to generate random matrices from the classical compact groups,” Notices AMS, vol. 54, pp. 592–604, Sep. 2007.
O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,” in Proc. ICLR, 2019.
C. R. Givens and R. M. Shortt, “A class of Wasserstein metrics for probability distributions,” Michigan Math. J., vol. 31, no. 2, pp. 231–240, 1984.
M.-M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, and S.-M. Hu, “Global contrast based salient region detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569–582, Mar. 2015.
L. v. d. Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach. Learn. Res., vol. 9, no. 11, pp. 1–27, 2008.
