Communication efficiency; data compression; deep learning; goal-oriented semantic communication; redundancy reduction; 'current; Communications systems; Deep learning; Down-stream; Goal-oriented; Goal-oriented semantic communication; Invariant representation; Redundancy reductions; Semantic communication; Software; Computer Networks and Communications; Electrical and Electronic Engineering
Abstract :
[en] Goal-oriented semantic communication (SC) aims to revolutionize communication systems by transmitting only task-essential information. However, current approaches face challenges such as joint training at transceivers, leading to redundant data exchange and reliance on labeled datasets, which limits their task-agnostic utility. To address these challenges, we propose a novel framework called Goal-oriented Invariant Representation-based SC (SC-GIR) for image transmission. Our framework leverages self-supervised learning to extract an invariant representation that encapsulates crucial information from the source data, independent of the specific downstream task. This compressed representation facilitates efficient communication while retaining key features for successful downstream task execution. Focusing on machine-to-machine tasks, we utilize covariance-based contrastive learning techniques to obtain a latent representation that is both meaningful and semantically dense. To evaluate the effectiveness of the proposed scheme on downstream tasks, we apply it to various image datasets for lossy compression. The compressed representations are then used in a goal-oriented AI task. Extensive experiments on several datasets demonstrate that SC-GIR outperforms baseline schemes by nearly 10%, and achieves over 85% classification accuracy for compressed data under different SNR conditions. These results underscore the effectiveness of the proposed framework in learning compact and informative latent representations.
Disciplines :
Computer science
Author, co-author :
Wanasekara, Senura Hansaja; University of Sydney, Sydney, Australia ; VinUniversity, College of Engineering and Computer Science, Hanoi, Viet Nam
Nguyen, Van-Dinh; College of Engineering and Computer Science, Viet Nam ; VinUniversity, Center for Environmental Intelligence (CEI), Hanoi, Viet Nam
Wong, Kok-Seng; College of Engineering and Computer Science, Viet Nam ; VinUniversity, Center for Environmental Intelligence (CEI), Hanoi, Viet Nam
Nguyen, M.-Duong; VinUniversity, Department of Intelligent Computing and Data Science, Hanoi, Viet Nam
CHATZINOTAS, Symeon ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom
Dobre, Octavia A.; Memorial University, Dept. of Electrical and Computer Engineering, St. John's, Canada
External co-authors :
yes
Language :
English
Title :
SC-GIR: Goal-oriented Semantic Communication via Invariant Representation Learning for Image Transmission
Publication date :
2025
Journal title :
IEEE Transactions on Mobile Computing
ISSN :
1536-1233
Publisher :
Institute of Electrical and Electronics Engineers Inc.
T.-H. Vu, S. K. Jagatheesaperumal, M.-D. Nguyen, N. V. Huynh, S. Kim, and Q.-V. Pham, “Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Survey,” IEEE Internet of Things Journ., Aug. 2024.
J. A. Cabrera, H. Boche, C. Deppe, R. F. Schaefer, C. Scheunert, and F. H. P. Fitzek, 6G and the post-Shannon theory. John Wiley & Sons, Ltd, 2022, pp. 271–294.
Z. Qin, L. Liang, Z. Wang, S. Jin, X. Tao, W. Tong, and G. Y. Li, “AI empowered wireless communications: From bits to semantics,” Proc. IEEE, vol. 112, no. 7, pp. 621–652, 2024.
A. Mostaani, T. X. Vu, S. K. Sharma, V.-D. Nguyen, Q. Liao, and S. Chatzinotas, “Task-oriented communication design in cyber-physical systems: A survey on theory and applications,” IEEE Access, vol. 10, pp. 133 842–133 868, 2022.
Q. Hu, G. Zhang, Z. Qin, Y. Cai, G. Yu, and G. Y. Li, “Robust semantic communications against semantic noise,” in 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall), 2022.
J. Shao, Y. Mao, and J. Zhang, “Learning task-oriented communication for edge inference: An information bottleneck approach,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 197–211, 2021.
Y. Wang, S. Guo, Y. Deng, H. Zhang, and Y. Fang, “Privacy-preserving task-oriented semantic communications against model inversion attacks,” IEEE Transactions on Wireless Communications, vol. 23, no. 8, pp. 10 150–10 165, 2024.
Y. Lin, Z. Gao, H. Du, D. Niyato, J. Kang, Z. Xiong, and Z. Zheng, “Blockchain-based efficient and trustworthy aigc services in metaverse,” IEEE Transactions on Services Computing, Mar. 2024.
H. Xie and Z. Qin, “A lite distributed semantic communication system for internet of things,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 142–153, 2021.
E. Bourtsoulatze, D. Burth Kurka, and D. Gündüz, “Deep joint source-channel coding for wireless image transmission,” IEEE Trans. Cogn. Commun. and Network., vol. 5, no. 3, pp. 567–579, 2019.
X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang, “Self-supervised learning: Generative or contrastive,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 1, pp. 857–876, 2023.
C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, no. 3, pp. 379–423, 1948.
A. Jerri, “The Shannon sampling theorem—Its various extensions and applications: A tutorial review,” Proc. IEEE, vol. 65, no. 11, pp. 1565–1596, 1977.
G. Shi, Y. Xiao, Y. Li, and X. Xie, “From semantic communication to semantic-aware networking: Model, architecture, and open problems,” IEEE Communications Magazine, vol. 59, no. 8, pp. 44–50, 2021.
S. Al-Sarawi, M. Anbar, K. Alieyan, and M. Alzubaidi, “Internet of Things (IoT) communication protocols: Review,” in Proc. 8th Inter. Confe. Infor. Tech. IEEE, 2017, pp. 685–690.
C. Bayılmış, M. A. Ebleme, Ünal Çavuşoglu, K. Küçük, and A. Sevin, “A survey on communication protocols and performance evaluations for internet of things,” Digi. Commun. and Net., vol. 8, no. 6, pp. 1094–1104, 2022.
J. A. Stankovic, “Research directions for the Internet of Things,” IEEE Internet of Things J., vol. 1, no. 1, pp. 3–9, 2014.
H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Trans. Sign. Process., Sep. 2021.
G. Zhang, Q. Hu, Z. Qin, Y. Cai, G. Yu, and X. Tao, “A Unified MultiTask Semantic Communication System for Multimodal Data,” arXiv preprint arXiv:2209.07689, Aug. 2023.
H. Xie, Z. Qin, X. Tao, and K. B. Letaief, “Task-Oriented Multi-User Semantic Communications,” IEEE Jour. of Sel. Areas in Comm., Mar. 2022.
H. Xie, Z. Qin, and G. Y. Li, “Semantic communication with memory,” IEEE Jour. of Sel. Areas in Comm., Jun. 2023.
C. Liu, C. Guo, Y. Yang, and N. Jiang, “Adaptable Semantic Compression and Resource Allocation for Task-Oriented Communications,” IEEE Trans. Cogn. Comm. and Netw., Jun. 2024.
D. B. Kurka and D. Gündüz, “DeepJSCC-f: Deep Joint Source-Channel Coding of Images With Feedback,” IEEE Jour. of Sel. Areas in Comm., May 2020.
E. Bourtsoulatze, D. Burth Kurka, and D. Gündüz, “Deep Joint Source-Channel Coding for Wireless Image Transmission,” IEEE Trans. Cogn. Comm. and Netw., Apr. 2019.
Q. Liao and T.-Y. Tung, “AdaSem: Adaptive Goal-Oriented Semantic Communications for End-to-End Camera Relocalization,” in INFOCOM, May 2024.
S. Tang, Q. Yang, L. Fan, X. Lei, A. Nallanathan, and G. K. Karagiannidis, “Contrastive learning-based semantic communications,” IEEE Trans. on Comm., Oct. 2024.
W. Zhang, K. Bai, S. Zeadally, H. Zhang, H. Shao, H. Ma, and V. C. M. Leung, “DeepMA: End-to-end deep multiple access for wireless image transmission in semantic communication,” IEEE Trans. Cogn. Comm. and Netw., Apr. 2024.
E. Erdemir, T.-Y. Tung, P. L. Dragotti, and D. Gündüz, “Generative joint source-channel coding for semantic image transmission,” IEEE Jour. of Sel. Areas in Comm., Aug. 2023.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in Proceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139. PMLR, 18–24 Jul 2021, pp. 8748–8763. [Online]. Available: https://proceedings.mlr.press/v139/radford21a.html
R. Lopez, J. Regier, M. I. Jordan, and N. Yosef, “Information constraints on auto-encoding variational bayes,” Advances in neural information processing systems, vol. 31, 2018.
H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Trans. Signal Process., vol. 69, pp. 2663–2675, 2021.
M. Federici, A. Dutta, P. Forré, N. Kushman, and Z. Akata, “Learning robust representations via multi-view information bottleneck,” in International Conference on Learning Representations, 2020.
R. Shwartz Ziv and Y. LeCun, “To compress or not to compress—self-supervised learning and information theory: A review,” Entropy, vol. 26, no. 3, p. 252, 2024.
N. Tishby and N. Zaslavsky, “Deep learning and the information bottleneck principle,” in 2015 IEEE Information Theory Workshop (ITW), 2015, pp. 1–5.
H. Li, W. Yu, H. He, J. Shao, S. Song, J. Zhang, and K. B. Letaief, “Task-oriented communication with out-of-distribution detection: An information bottleneck framework,” in GLOBECOM 2023 - 2023 IEEE Global Communications Conference, Feb. 2024.
Y. Wu, X. Wang, A. Zhang, X. He, and T.-S. Chua, “Discovering Invariant Rationales for Graph Neural Networks,” in Int. Conf. Learn. Represent., May 2022.
B. Li, Y. Wang, S. Zhang, D. Li, K. Keutzer, T. Darrell, and H. Zhao, “Learning invariant representations and risks for semi-supervised domain adaptation,” in IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2021, pp. 1104–1113.
C. Wu, F. Wu, and Y. Huang, “Rethinking infonce: How many negative samples do you need?” in Proc. 31st Inter. Joint Conf. Artificial Intelligence, IJCAI-22, 7 2022, pp. 2509–2515.
A. Bardes, J. Ponce, and Y. LeCun, “VICReg: Variance-invariance-covariance regularization for self-supervised learning,” in International Conference on Learning Representations, 2022.
J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny, “Barlow twins: Self-supervised learning via redundancy reduction,” in Proc. 38th Inter. Conf. Machine Learning, (ICML), vol. 139, 2021, pp. 12 310–12 320.
F. Lv, J. Liang, S. Li, B. Zang, C. H. Liu, Z. Wang, and D. Liu, “Causality inspired representation learning for domain generalization,” in IEEE Conf. Comput. Vis. Pattern Recog., June 2022, pp. 8046–8056.
W. Huang et al., “Generalizable heterogeneous federated cross-correlation and instance similarity learning,” IEEE Trans. Patt. Analysis and Machine Intelligence, vol. 46, no. 2, pp. 712–728, 2024.
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in Proc. Inter. Conf. Machine Learning. PMLR, 2020, pp. 1597–1607.
H. Wang, X. Guo, Z. Deng, and Y. Lu, “Rethinking minimal sufficient representation in contrastive learning,” in Proc. IEEE/CVF Conf. Comput. Vision and Patt. Recog. (CVPR), 2022, pp. 16 020–16 029.
N. Tishby and N. Zaslavsky, “Deep learning and the information bottleneck principle,” in IEEE Infor. Theory Workshop (ITW), 2015, pp. 1–5.
A. Krizhevsky, G. Hinton et al., “Learning multiple layers of features from tiny images,” Department of Computer Science, University of Toronto, 2009.
G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: Extending mnist to handwritten letters,” in 2017 International Joint Conference on Neural Networks (IJCNN), 2017, pp. 2921–2926.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.
A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” in Proc. 14th Inter. Conf. Artificial Intelligence and Statistics, vol. 15. Fort Lauderdale, FL, USA: PMLR, 11–13 Apr 2011, pp. 215–223.
M.-E. Nilsback and A. Zisserman, “A visual vocabulary for flower classification,” in IEEE Conf. Comput. Vision and Patt. Recog.(CVPR’06), vol. 2, 2006, pp. 1447–1454.
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3213–3223.
D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Deeper, broader and artier domain generalization,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5542–5550.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELU),” arXiv preprint arXiv:1606.08415, 2016.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2014.
K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in IEEE Inter. Conf. Computer Vision (ICCV), 2015, pp. 1026–1034.
I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” in Inter. Conf. Learning Represent., 2017.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
S. Tang, Q. Yang, L. Fan, X. Lei, A. Nallanathan, and G. K. Karagiannidis, “Contrastive learning-based semantic communications,” IEEE Transactions on Communications, vol. 72, no. 10, pp. 6328–6343, 2024.
H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Transactions on Signal Processing, vol. 69, pp. 2663–2675, 2021.
E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” Advances in neural information processing systems, vol. 34, pp. 12 077–12 090, 2021.
D. Brüggemann, C. Sakaridis, P. Truong, and L. Van Gool, “Refign: Align and refine for adaptation of semantic segmentation to adverse conditions,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 3174–3184.
C. Sakaridis, D. Dai, and L. Van Gool, “Map-guided curriculum domain adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 6, pp. 3139–3153, 2020.
