Deep autoencoder architecture with outliers for temporal attributed network embedding

Autoencoders; Network embedding; Outlier nodes; Temporal attributed networks; 'current; Auto encoders; Learn+; Link prediction; Link structure; Low dimensional; Outlier node; Temporal attributed network; Temporal networks; Engineering (all); Computer Science Applications; Artificial Intelligence

Abstract :

[en] Temporal attributed network embedding aspires to learn a low-dimensional vector representation for each node in each snapshot of a temporal network, which can be capable of various network analysis tasks such as link prediction and node classification. In temporal attributed networks, attribute similarities or link structures of certain nodes may deviate from the regular nodes of the community they belong to, which are called community outlier nodes. However, many existing embedding methods consider only the link structures and their attributes of the nodes adhere to the community structure of the network while ignoring outlier nodes, this can affect the embedding performance of the regular nodes. In this paper, we propose a temporal attributed network embedding framework with outliers, based on autoencoders, to solve the problem. In particular, we propose an outlier-aware autoencoder to model the node information, which combines the current snapshot and previous snapshots to jointly learn embedded vectors of nodes in the current snapshot of a temporal network. In feature preprocessing, we propose a simplified higher graph convolutional mechanism to incorporate attribute information into link structure information, which can leverage attribute features into link structure. Experimental results on node classification and link prediction reveal that our model is competitive against various baseline models.

Disciplines :

Computer science

Author, co-author :

Mo, Xian ; School of Information Engineering, Ningxia University, Yinchuan, China ; College of Computer & Information Science, Southwest University, Chongqing, China

PANG, Jun ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS)

Liu, Zhiming ; College of Computer & Information Science, Southwest University, Chongqing, China

External co-authors :

yes

Language :

English

Title :

Deep autoencoder architecture with outliers for temporal attributed network embedding

Publication date :

15 April 2024

Journal title :

Expert Systems with Applications

ISSN :

0957-4174

eISSN :

1873-6793

Publisher :

Elsevier Ltd

Volume :

240

Pages :

122596

Peer reviewed :

Peer Reviewed verified by ORBi

Funding text :

The work is financed by the National Natural Science Foundation of China ( 62306157 , 62202320 ), 62032019 , Capacity Development Grant of Southwest University ( SWU116007 ).

Available on ORBilu :

since 31 May 2024

Statistics

Number of views

69 (1 by Unilu)

Number of downloads

102 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenAlex citations

WoS citations^™

Bibliography

Amara, A., Taieb, M.A.H., Aouicha, M.B., Network representation learning systematic review: ancestors and current development state. 2021 CoRR, abs/2109.07583.
Bai, S., Kolter, J.Z., Koltun, V., An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv: Learning, 2018.
Bandyopadhyay, S., Lokesh, N., & Murty, M. N. (2019). Outlier Aware Network Embedding for Attributed Networks. In Proceedings of the 33th AAAI conference on artificial intelligence (pp. 12–19).
Bandyopadhyay, S., N, L., Vivek, S. V., & Murty, M. N. (2020). Outlier Resistant Unsupervised Deep Architectures for Attributed Network Embedding. In Proceedings of 13th international conference on web search and data mining (pp. 25–33).
Cai, X., Huang, C., Xia, L., & Ren, X. (2023). LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In Proceedings of the 7th international conference on learning representations (pp. 1–15).
Ding, K., Li, J., Bhanushali, R., & Liu, H. (2019). Deep Anomaly Detection on Attributed Networks. In Proceedings of 2019 international conference on data mining (pp. 594–602).
Du, X., Yu, J., Chu, Z., Jin, L., Chen, J., Graph autoencoder-based unsupervised outlier detection. Information Sciences 608 (2022), 532–550.
Fathy, A., & Li, K. (2020). TemporalGAT: Attention-Based Dynamic Graph Representation Learning. In Proceedings of the 24th Pacific-Asia conference on knowledge discovery and data mining (pp. 413–423).
Grover, A., & Leskovec, J. (2016). node2vec: Scalable feature learning for networks. In Proceedings of the 22th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 855–864).
Hsieh, I., Li, C., CoANE: Modeling context co-occurrence for attributed network embedding. IEEE Transactions on Knowledge and Data Engineering 35:1 (2023), 167–180.
Huang, L., Zhu, Y., Gao, Y., Liu, T., Chang, C., Liu, C., et al. Hybrid-order anomaly detection on attributed networks. IEEE Transactions on Knowledge and Data Engineering, 2021, 1–14.
Issaid, C.B., Elgabli, A., Bennis, M., DR-DSGD: A distributionally robust decentralized learning algorithm over graphs. 2022 CoRR, abs/2208.13810.
J., H., X., L.C., Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on Knowledge and Data Engineering 17 (2005), 299–310.
Ji, Y., Huang, L., He, H., Wang, C., Xie, G., Shi, W., et al. (2019). Multi-view Outlier Detection in Deep Intact Space. In Proceedings of the 2019 IEEE international conference on data mining (pp. 1132–1137).
Jiao, P., Guo, X., Jing, X., He, D., Wu, H., Pan, S., et al. Temporal network embedding for link prediction via VAE joint attention mechanism. IEEE Transactions on Neural Networks and Learning Systems 33:12 (2022), 7400–7413.
Keikha, M.M., Rahgozar, M., Asadpour, M., Community aware random walk for network embedding. Knowledge Based Systems 148 (2018), 47–54.
Kipf, T. N., & Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the 5th international conference on learning representations (pp. 1–14).
Li, J., Cheng, K., Liang, W., & Liu, H. (2018). Streaming Link Prediction on Dynamic Attributed Networks. In Proceedings of the 11th ACM international conference on web search and data mining (pp. 369–377).
Li, J., Dani, H., Hu, X., Tang, J., Chang, Y., & Liu, H. (2017). Attributed Network Embedding for Learning in a Dynamic Environment. In Proceedings of the 26th conference on information and knowledge management (pp. 387–396).
Li, Z., & Lai, D. (2022). Dynamic Network Embedding via Temporal Path Adjacency Matrix Factorization. In Proceedings of the 31th ACM international conference on information & knowledge management (pp. 1219–1228).
Li, Y., Wang, Y., Zhang, T., Zhang, J., & Chang, Y. (2019). Learning Network Embedding with Community Structural Information. In Proceedings of the 28th international joint conference on artificial intelligence (pp. 2937–2943).
Liang, J., Jacobs, P., Sun, J., & Parthasarathy, S. (2018). Semi-supervised Embedding in Attributed Networks with Outliers. In Proceedings of the 2018 SIAM international conference on data mining (pp. 153–161).
Liang, S., Zhang, X., Ren, Z., & Kanoulas, E. (2018). Dynamic Embeddings for User Profiling in Twitter. In Proceedings of the 24th conference on knowledge discovery and data mining (pp. 1764–1773).
Lin, Z., Tian, C., Hou, Y., & Zhao, W. X. (2022). Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning. In Proceedings of the 7th ACM web conference 2022 (pp. 2320–2329).
Liu, Z., Huang, C., Yu, Y., & Dong, J. (2021). Motif-Preserving Dynamic Attributed Network Embedding. In Proceedings of the 21th web conference (pp. 1629–1638).
Ma, J., Zhang, Q., Lou, J., Xiong, L., & Ho, J. C. (2021). Temporal Network Embedding via Tensor Factorization. In Proceedings of the 30th ACM international conference on information and knowledge management (pp. 3313–3317).
Meng, Z., Liang, S., Fang, J., & Xiao, T. (2019). Semi-supervisedly Co-embedding Attributed Networks. In Proceedings of the 32th annual conference on neural information processing systems 2019 (pp. 6504–6513).
Meng, Z., Liang, S., Zhang, X., Mccreadie, R., Ounis, I., Jointly learning representations of nodes and attributes for attributed networks. ACM Transactions on Information Systems 38:2 (2020), 1–32.
Miller, M., Lynn, S.-L., James, M.C., Birds of a feather: Homophily in social networks. Annual Review of Sociology 27 (2001), 415–444.
Sadeghi, A., Ma, M., Li, B., & Giannakis, G. B. (2021). Distributionally Robust Semi-Supervised Learning Over Graphs. CoRR,.
Sankar, A., Wu, Y., Gou, L., Zhang, W., & Yang, H. (2020). DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks. In Proceedings of the 13th ACM international conference on web search and data mining (pp. 519–527).
Sun, G., Shen, Y., Zhou, S., Chen, X., Liu, H., Wu, C., et al. (2023). Self-Supervised Interest Transfer Network via Prototypical Contrastive Learning for Recommendation. In Proceedings of the 37th AAAI conference on artificial intelligence (pp. 4614–4622).
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2018). Graph Attention Networks. In Proceedings of the 6th international conference on learning representations (pp. 1–12).
Wang, X., Cui, P., Wang, J., Pei, J., Zhu, W., & Yang, S. (2017). Community Preserving Network Embedding. In Proceedings of the 41th AAAI conference on artificial intelligence (pp. 203–209).
Wang, X., Lu, Y., Shi, C., Wang, R., Cui, P., Mou, S., Dynamic heterogeneous information network embedding with meta-path based proximity. IEEE Transactions on Knowledge and Data Engineering 34:3 (2022), 1117–1132.
Wang, X., Pun, Y., So, A.M., Distributionally robust graph learning from smooth signals under moment uncertainty. IEEE Transactions on Signal Processing 70 (2022), 6216–6231.
Wang, Y., Wang, X., Huang, X., Yu, Y., Li, H., Zhang, M., et al. (2023). Intent-aware Recommendation via Disentangled Graph Contrastive Learning. In Proceedings of the 32th international joint conference on artificial intelligence (pp. 2343–2351).
Wei, H., Hu, G., Bai, W., Xia, S., Pan, Z., Lifelong representation learning in dynamic attributed networks. Neurocomputing 358 (2019), 1–9.
Wu, F., Jr., A. H. S., Zhang, T., Fifty, C., Yu, T., & Weinberger, K. Q. (2019). Simplifying Graph Convolutional Networks. In Proceedings of the 36th international conference on machine learning (pp. 6861–6871).
Xie, S., Li, Y., Tam, D. S. H., Liu, X., Ying, Q., Lau, W. C., et al. (2023). GTEA: Inductive Representation Learning on Temporal Interaction Graphs via Temporal Edge Aggregation. In Proceedings of the 27th Pacific-Asia conference on knowledge discovery and data mining (pp. 28–39).
Xu, Y., Shi, B., Ma, T., Dong, B., Zhou, H., & Zheng, Q. (2023). CLDG: Contrastive Learning on Dynamic Graphs. In Proceedings of the 39th IEEE international conference on data engineering (pp. 696–707).
Yang, Y., Huang, C., Xia, L., Huang, C., Luo, D., & Lin, K. (2023). Debiased Contrastive Learning for Sequential Recommendation. In Proceedings of the ACM web conference 2023 (pp. 1063–1073).
Yao, H., Zhang, B., Zhang, P., Wu, S., Jiang, C., Guo, S., RDAM: A reinforcement learning based dynamic attribute matrix representation for virtual network embedding. IEEE Transactions on Emerging Topics in Computing 9:2 (2021), 901–914.
Yu, W., Cheng, W., Aggarwal, C. C., Zhang, K., Chen, H., & Wang, W. (2018). NetWalk: A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks. In Proceedings of the 24th ACM international conference on knowledge discovery and data mining (pp. 2672–2681).
Zhang, Z., Yang, H., Bu, J., Zhou, S., Yu, P., Zhang, J., et al. (2018). ANRL: Attributed Network Representation Learning via Deep Neural Networks. In Proceedings of the 27th international joint conference on artificial intelligence (pp. 3155–3161).
Zhang, C., Yao, Z., Yao, H., Huang, F., Chen, C.L.P., Dynamic representation learning via recurrent graph neural networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53:2 (2023), 1284–1297.
Zhao, S., Chen, J., Chen, J., Zhang, Y., Tang, J., Hierarchical label with imbalance and attributed network structure fusion for network embedding. AI Open 3 (2022), 91–100.
Zhao, L., Song, Y., Zhang, C., Liu, Y., Wang, P., Lin, T., et al. T-GCN: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems 21:9 (2020), 3848–3858.
Zou, H., Duan, Z., Guo, X., Zhao, S., Chen, J., Zhang, Y., et al. On embedding sequence correlations in attributed network for semi-supervised node classification. Information Sciences 562 (2021), 385–397.