[en] Multi-relational graph learning aims to embed entities and relations in knowledge graphs into low-dimensional representations, which has been successfully applied to various multi-relationship prediction tasks, such as information retrieval, question answering, and etc. Recently, contrastive learning has shown remarkable performance in multi-relational graph learning by data augmentation mechanisms to deal with highly sparse data. In this paper, we present a Multi-Relational Graph Contrastive Learning architecture (MRGCL) for multi-relational graph learning. More specifically, our MRGCL first proposes a Multi-relational Graph Hierarchical Attention Networks (MGHAN) to identify the importance between entities, which can learn the importance at different levels between entities for extracting the local graph dependency. Then, two graph augmented views with adaptive topology are automatically learned by the variant MGHAN, which can automatically adapt for different multi-relational graph datasets from diverse domains. Moreover, a subgraph contrastive loss is designed, which generates positives per anchor by calculating strongly connected subgraph embeddings of the anchor as the supervised signals. Comprehensive experiments on multi-relational datasets from three application domains indicate the superiority of our MRGCL over various state-of-the-art methods. Our datasets and source code are published at https://github.com/Legendary-L/MRGCL.
Disciplines :
Computer science
Author, co-author :
Mo, Xian ; School of Information Engineering, Ningxia University, Yinchuan 750021, China, Ningxia Key Laboratory of Artificial Intelligence and Information Security for Channeling Computing Resources from the East to the West, Ningxia University, Yinchuan 750021, China. Electronic address: mxian168@nxu.edu.cn
PANG, Jun ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)
Wan, Binyuan; School of Information Engineering, Ningxia University, Yinchuan 750021, China. Electronic address: binyuanw@outlook.com
Tang, Rui ; School of Cyber Science and Engineering, Sichuan University, Chengdu 610065, Sichuan, China. Electronic address: tangrscu@scu.edu.cn
Liu, Hao; School of Information Engineering, Ningxia University, Yinchuan 750021, China, Ningxia Key Laboratory of Artificial Intelligence and Information Security for Channeling Computing Resources from the East to the West, Ningxia University, Yinchuan 750021, China. Electronic address: liuhao@nxu.edu.cn
Jiang, Shuyu ; School of Cyber Science and Engineering, Sichuan University, Chengdu 610065, Sichuan, China. Electronic address: jiang.shuyu07@gmail.com
External co-authors :
yes
Language :
English
Title :
Multi-relational graph contrastive learning with learnable graph augmentation.
Balazevic, I., Allen, C., & Hospedales, T. M. (2019). Multi-relational Poincaré Graph Embeddings. In Proceedings of the 32th annual conference on neural information processing systems (pp. 4465–4475).
Bansal, T., Juan, D., Ravi, S., & McCallum, A. (2019). A2N: Attending to Neighbors for Knowledge Graph Inference. In Proceedings of the 57th conference of the association for computational linguistics (pp. 4387–4392).
Binkowski, J., Sawczyn, A., Janiak, D., Bielak, P., & Kajdanowicz, T. (2023). Graph-Level Representations Using Ensemble-Based Readout Functions. In Proceedings of the 23th international conference on computational science (pp. 393–405).
Bordes, A., Usunier, N., García-Durán, A., Weston, J., & Yakhnenko, O. (2013). Translating Embeddings for Modeling Multi-relational Data. In Proceedings of the 27th annual conference on neural information processing systems 2013 (pp. 2787–2795).
Cao, X., Shi, Y., Wang, J., Yu, H., Wang, X., & Yan, Z. (2022). Cross-modal Knowledge Graph Contrastive Learning for Machine Learning Method Recommendation. In Proceedings of the 30th ACM international conference on multimedia (pp. 3694–3702).
Chen, G., Fang, J., Meng, Z., Zhang, Q., & Liang, S. (2022). Multi-Relational Graph Representation Learning with Bayesian Gaussian Process Network. In Proceedings of the 36th AAAI conference on artificial intelligence (pp. 5530–5538).
Chen, Y., Yang, Y., Wang, Y., Bai, J., Song, X., & King, I. (2022). Attentive Knowledge-aware Graph Convolutional Networks with Collaborative Guidance for Personalized Recommendation. In Proceedings of the 38th IEEE international conference on data engineering (pp. 299–311).
Chen, C., Zhang, M., Zhang, Y., Ma, W., Liu, Y., & Ma, S. (2020). Efficient Heterogeneous Collaborative Filtering without Negative Sampling for Recommendation. In Proceedings of the 34th AAAI conference on artificial intelligence (pp. 19–26).
Dettmers, T., Minervini, P., Stenetorp, P., & Riedel, S. (2018). Convolutional 2D Knowledge Graph Embeddings. In Proceedings of the 32th AAAI conference on artificial intelligence (pp. 1811–1818).
Ding, X., Zhang, H., Ma, C., Zhang, X., Zhong, K., User identification across multiple social networks based on naive Bayes model. IEEE Transactions on Neural Networks and Learning Systems 35:3 (2024), 4274–4285.
Fan, Z., Yang, Y., Xu, M., Chen, H., Node-based knowledge graph contrastive learning for medical relationship prediction. 2023 CoRR abs/2310.10138.
Fang, Y., Li, X., Ye, R., Tan, X., Zhao, P., Wang, M., Relation-aware graph convolutional networks for multi-relational network alignment. ACM Transactions on Intelligent Systems and Technology 14:2 (2023), 37:1–37:23.
Fang, Q., Zhang, X., Hu, J., Wu, X., Xu, C., Contrastive multi-modal knowledge graph representation learning. IEEE Transactions on Knowledge and Data Engineering 35:9 (2023), 8983–8996.
Hu, Z., Gutiérrez-Basulto, V., Xiang, Z., Li, R., & Pan, J. Z. (2023). HyperFormer: Enhancing Entity and Relation Interaction for Hyper-Relational Knowledge Graph Completion. In Proceedings of the 32nd ACM international conference on information and knowledge management (pp. 803–812).
Jiang, Y., Huang, C., & Huang, L. (2023). Adaptive Graph Contrastive Learning for Recommendation. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining (pp. 4252–4261).
Kacupaj, E., Singh, K., Maleshkova, M., & Lehmann, J. (2022). Contrastive Representation Learning for Conversational Question Answering over Knowledge Graphs. In M. A. Hasan, & L. Xiong (Eds.), Proceedings of the 31st ACM international conference on information & knowledge management (pp. 925–934).
Khatir, M., Choudhary, N., Choudhury, S., Agarwal, K., & Reddy, C. K. (2023). A Unification Framework for Euclidean and Hyperbolic Graph Neural Networks. In Proceedings of the 32th international joint conference on artificial intelligence (pp. 3875–3883).
Kipf, T. N., & Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th international conference on learning representations (pp. 1–14).
Li, Q., Joty, S., Wang, D., Feng, S., Zhang, Y., & Qin, C. (2023). Contrastive Learning with Generated Representations for Inductive Knowledge Graph Embedding. In Proceedings of the 61tt annual meeting of the association for computational linguistics (pp. 14273–14287).
Li, P., Li, Y., Hsieh, C., Zhang, S., Liu, X., Liu, H., et al. TrimNet: learning molecular representation from triplet messages for biomedicine. Briefings in Bioinformatics, 22(4), 2021.
Lin, Z., Tian, C., Hou, Y., & Zhao, W. X. (2022). Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning. In Proceedings of the 7th ACM web conference 2022 (pp. 2320–2329).
Liu, L., Chen, Y., Das, M., Yang, H., & Tong, H. (2023). Knowledge Graph Question Answering with Ambiguous Query. In Proceedings of the ACM web conference (pp. 2477–2486).
Luo, H., E, H., Yang, Y., Guo, Y., Sun, M., Yao, T., et al. (2023). HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level. In Proceedings of the 61tt annual meeting of the association for computational linguistics (pp. 8095–8107).
McPherson, M., Smith-Lovin, L., Cook, J.M., Birds of a feather: Homophily in social networks. Annual Review of Sociology 27:1 (2001), 415–444.
Nathani, D., Chauhan, J., Sharma, C., & Kaul, M. (2019). Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs. In Proceedings of the 57th conference of the association for computational linguistics (pp. 4710–4723).
Qu, M., Gao, T., Xhonneux, L. A. C., & Tang, J. (2020). Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs. In Proceedings of the 37th international conference on machine learning (pp. 7867–7876).
Schlichtkrull, M. S., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018). Modeling Relational Data with Graph Convolutional Networks. In Proceedings of the 15th international conference on semantic web (pp. 593–607).
Shang, C., Tang, Y., Huang, J., Bi, J., He, X., & Zhou, B. (2019). End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion. In Proceedings of the 33th AAAI conference on artificial intelligence (pp. 3060–3067).
Shen, X., Sun, D., Pan, S., Zhou, X., & Yang, L. T. (2023). Neighbor Contrastive Learning on Learnable Graph Augmentation. In Proceedings of the 37th AAAI conference on artificial intelligence (pp. 9782–9791).
Tan, Z., Chen, Z., Feng, S., Zhang, Q., Zheng, Q., Li, J., et al. (2023). KRACL: Contrastive Learning with Graph Context Modeling for Sparse Knowledge Graph Completion. In Proceedings of the ACM web conference 2023 (pp. 2548–2559).
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., & Isola, P. (2020). What Makes for Good Views for Contrastive Learning?. In Proceedings of the 33th advances in neural information processing systems (pp. 6827–6839).
van den Oord, A., Li, Y., Vinyals, O., Representation learning with contrastive predictive coding. 2018 CoRR abs/1807.03748.
Vashishth, S., Sanyal, S., Nitin, V., & Talukdar, P. P. (2020). Composition-based Multi-Relational Graph Convolutional Networks. In Proceedings of the 8th international conference on learning representations (pp. 1–16).
Velickovic, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., & Hjelm, R. D. (2019). Deep Graph Infomax. In Proceedings of the 7th international conference on learning representations (pp. 1–17).
Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., et al. (2019). Heterogeneous Graph Attention Network. In Proceedings of the world wide web conference (pp. 2022–2032).
Wei, W., Huang, C., Xia, L., Xu, Y., Zhao, J., & Yin, D. (2022). Contrastive Meta Learning with Behavior Multiplicity for Recommendation. In Proceedings of the 15th ACM international conference on web search and data mining (pp. 1120–1128).
Wei, W., Xia, L., & Huang, C. (2023). Multi-Relational Contrastive Learning for Recommendation. In Proceedings of the 17th ACM conference on recommender systems (pp. 338–349).
Wu, J., Wang, X., Feng, F., He, X., Chen, L., Lian, J., et al. (2021). Self-supervised Graph Learning for Recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (pp. 726–735).
Xia, L., Huang, C., Xu, Y., Dai, P., Zhang, X., Yang, H., et al. (2021). Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation. In Proceedings of the 35th AAAI conference on artificial intelligence (pp. 4486–4493).
Xia, L., Xu, Y., Huang, C., Dai, P., & Bo, L. (2021). Graph Meta Network for Multi-Behavior Recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (pp. 757–766).
Xiong, Z., Liu, S., Huang, F., Wang, Z., Liu, X., Zhang, Z., et al. (2023). Multi-Relational Contrastive Learning Graph Neural Network for Drug-Drug Interaction Event Prediction. In Proceedings of the 37th AAAI conference on artificial intelligence (pp. 5339–5347).
Yang, Y., Guan, Z., Li, J., Zhao, W., Cui, J., Wang, Q., Interpretable and efficient heterogeneous graph convolutional network. IEEE Transactions on Knowledge and Data Engineering 35:2 (2023), 1637–1650.
Yang, Y., Huang, C., Xia, L., & Li, C. (2022). Knowledge Graph Contrastive Learning for Recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval (pp. 1434–1443).
Ye, R., Li, X., Fang, Y., Zang, H., & Wang, M. (2019). A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment. In Proceedings of the 28th international joint conference on artificial intelligence (pp. 4135–4141).
Yu, W., Yang, J., & Yang, D. (2024). Robust Link Prediction over Noisy Hyper-Relational Knowledge Graphs via Active Learning. In Proceedings of the ACM on web conference (pp. 2282–2293).
Yu, J., Yin, H., Xia, X., Chen, T., Cui, L., & Nguyen, Q. V. H. (2022). Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval (pp. 1294–1303).
Yun, S., Jeong, M., Kim, R., Kang, J., & Kim, H. J. (2019). Graph Transformer Networks. In Proceedings of the advances in neural information processing systems (pp. 11960–11970).
Zhao, Y., Du, H., Liu, Y., Wei, S., Chen, X., Zhuang, F., et al. Stock movement prediction based on bi-typed hybrid-relational market knowledge graph via dual attention networks. IEEE Transactions on Knowledge and Data Engineering 35:8 (2023), 8559–8571.
Zhao, Y., Wei, S., Du, H., Chen, X., Li, Q., Zhuang, F., et al. Learning bi-typed multi-relational heterogeneous graph via dual hierarchical attention networks. IEEE Transactions on Knowledge and Data Engineering 35:9 (2023), 9054–9066.
Zhou, Y., Chen, X., He, B., Ye, Z., & Sun, L. (2022). Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval (pp. 916–926).
Zhu, P., Wang, B., Tang, K., Zhang, H., Cui, X., Wang, Z., A knowledge-guided graph attention network for emotion-cause pair extraction. Knowledge-Based Systems, 286, 2024, 111342.
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., & Wang, L. (2021). Graph Contrastive Learning with Adaptive Augmentation. In Proceedings of the web conference (pp. 2069–2080).
Zhuang, L., Wang, H., Zhao, J., Sun, Y., Adaptive dual graph contrastive learning based on heterogeneous signed network for predicting adverse drug reaction. Information Sciences, 642, 2023, 119139.
