[en] In this paper, we propose a new deep learning based approach for disentangling face identity representations from expressive 3D faces. Given a 3D face, our approach not only extracts a disentangled identity representation, but also generates a realistic 3D face with a neutral expression while predicting its identity. The proposed network consists of three components; (1) a Graph Convolutional Autoencoder (GCA) to encode the 3D faces into latent representations, (2) a Generative Adversarial Network (GAN) that translates the latent representations of expressive faces into those of neutral faces, (3) and an identity recognition sub-network taking advantage of the neutralized latent representations for 3D face recognition. The whole network is trained in an end-to-end manner. Experiments are conducted on three publicly available datasets showing the effectiveness of the proposed approach.
Centre de recherche :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Computer Vision Imaging & Machine Intelligence (CVI²)
Disciplines :
Sciences informatiques
Auteur, co-auteur :
KACEM, Anis ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
cherenkova, kseniya; Artec3D
AOUADA, Djamila ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > CVI2
Co-auteurs externes :
no
Langue du document :
Anglais
Titre :
Disentangled Face Identity Representationsfor Joint 3D Face Recognition and Neutralisation
Date de publication/diffusion :
28 mai 2022
Nom de la manifestation :
International Conference on Virtual Reality
Date de la manifestation :
26-28 May 2022
Sur invitation :
Oui
Manifestation à portée :
International
Peer reviewed :
Peer reviewed
Focus Area :
Computational Sciences
Projet FnR :
FNR11643091 - Face Identification Under Deformations, 2017 (01/05/2018-31/10/2021) - Djamila Aouada
Face ID apple inc. https://support.apple.com/en-us/HT208108. Accessed: 2022-03-29.
Brian Amberg, Reinhard Knothe, and Thomas Vetter. Expression invariant 3d face recognition with a morphable model. In 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition, pages 1-6. IEEE, 2008.
Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pages 187-194, 1999.
James Booth, Anastasios Roussos, Stefanos Zafeiriou, Allan Ponniah, and David Dunaway. A 3d morphable model learnt from 10, 000 faces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5543-5552, 2016.
Giorgos Bouritsas, Sergiy Bokhnyak, Stylianos Ploumpis, Michael Bronstein, and Stefanos Zafeiriou. Neural 3d morphable models: Spiral convolutional networks for 3d shape representation learning and generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7213-7222, 2019.
Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.
Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, 20(3):413-425, 2013.
Shiyang Cheng, Michael Bronstein, Yuxiang Zhou, Irene Kotsia, Maja Pantic, and Stefanos Zafeiriou. Meshgan: Non-linear 3d morphable models of faces. arXiv preprint arXiv: 1903.10384, 2019.
Michael Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems, pages 3844-3852, 2016.
Hassen Drira, Boulbaba Ben Amor, Anuj Srivastava, Mohamed Daoudi, and Rim Slama. 3d face recognition under expressions, occlusions, and pose variations. [EEE Transactions on Pattern Analysis and Machine Intelligence, 35(9):2270-2283, 2013.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 315-323, 2011.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-Toimage translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125-1134, 2017.
Zi-Hang Jiang, Qianyi Wu, Keyu Chen, and Juyong Zhang. Disentangled representation learning for 3d face shape. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 11957-11966, 2019.
Yinjie Lei, Mohammed Bennamoun, Munawar Hayat, and Yulan Guo. An efficient 3d face recognition approach using local geometrical signatures. Pattern Recognition, 47(2):509-524, 2014.
Huibin Li, Di Huang, Jean-Marie Morvan, Yunhong Wang, and Liming Chen. Towards 3d face recognition in the real: A registrationfree approach using fine-grained matching of 3d keypoint descriptors. International Journal of Computer Vision, 113(2):128-142, 2015.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212-220, 2017.
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv: 1411.1784, 2014.
Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. 2015.
Dijana Petrovska-Delacretaz, Gerard Chollet, and Bernadette Dorizzi. Guide to biometric reference systems and performance evaluation. Springer, 2009.
Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J Black. Generating 3d faces using convolutional mesh autoencoders. In Proceedings of the European Conference on Computer Vision (ECCV), pages 704-720, 2018.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.
Haotian Yang, Hao Zhu, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. Facescape: A large-scale high quality 3d face dataset and detailed riggable 3d face prediction. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Sun, H., Pears, N. & Gu, Y. Information Bottlenecked Variational Autoencoder for Disentangled 3D Facial Expression Modelling. Proceedings Of The IEEE/CVF Winter Conference On Applications Of Computer Vision. pp. 157-166 (2022)
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-To-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223-2232, 2017.
