Computation Rate Maximization for SCMA-Aided Edge Computing in IoT Networks: A Multi-Agent Reinforcement Learning Approach

Liu, Pengtao; An, Kang; Lei, Jing; Sun, Yifu; Liu, Wei; CHATZINOTAS, Symeon

doi:10.1109/TWC.2024.3371791

Article (Scientific journals)

Computation Rate Maximization for SCMA-Aided Edge Computing in IoT Networks: A Multi-Agent Reinforcement Learning Approach

Liu, Pengtao; An, Kang; Lei, Jing et al.

2024 • In IEEE Transactions on Wireless Communications, 23 (8), p. 10414 - 10429

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/10993/68091

DOI
10.1109/TWC.2024.3371791

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Computation_Rate_Maximization_for_SCMA-Aided_Edge_Computing_in_IoT_Networks_A_Multi-Agent_Reinforcement_Learning_Approach.pdf

Author postprint (10.67 MB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

computation offloading; IoT; MEC; multi-agent reinforcement learning; resource allocation; SCMA; Computation offloading; Heuristics algorithm; Multi-agent reinforcement learning; Multiple access; NOMA; Optimisations; Resource management; Resources allocation; Sparse code multiple access; Sparse codes; Task analysis; Computer Science Applications; Electrical and Electronic Engineering; Applied Mathematics; Internet of Things; Heuristic algorithms; Optimization; Long short term memory

Abstract :

[en] Integrating sparse code multiple access (SCMA) and mobile edge computing (MEC) into the Internet of Things (IoT) networks can enable efficient connectivity and timely computation for resource-limited IoT users. This paper studies the computation rate maximization problem under task deadline constraints in dynamic SCMA-MEC networks. Specifically, we propose a predictive deep Q-network for SCMA resource allocation and computation offloading (PQ-RACO) algorithm for single-cell scenarios, where IoT devices use long short-term memory (LSTM) networks to predict the states and actions of other agents. However, the PQ-RACO algorithm is not scalable for increasing numbers of IoT devices. To address this issue, an improved multi-agent deep Q-network for SCMA resource allocation and computation offloading algorithm (MQ-RACO) is proposed for multi-cell scenarios. The algorithm is a centralized training and decentralized execution (CTDE) multi-agent reinforcement learning (MARL) algorithm with explicit rewards, which is tailored to the special structure of joint rewards. Simulation results demonstrate that the proposed algorithm outperforms several state-of-the-art MARL algorithms and other benchmark schemes in terms of convergence speed and computation rate.

Disciplines :

Computer science

Author, co-author :

Liu, Pengtao ; National University of Defense Technology, College of Electronic Science and Technology, Changsha, China

An, Kang ; Sixty-Third Research Institute, National University of Defense Technology, Nanjing, China

Lei, Jing ; National University of Defense Technology, College of Electronic Science and Technology, Changsha, China

Sun, Yifu ; National University of Defense Technology, College of Electronic Science and Technology, Changsha, China

Liu, Wei ; National University of Defense Technology, College of Electronic Science and Technology, Changsha, China

CHATZINOTAS, Symeon ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

External co-authors :

yes

Language :

English

Title :

Computation Rate Maximization for SCMA-Aided Edge Computing in IoT Networks: A Multi-Agent Reinforcement Learning Approach

Publication date :

2024

Journal title :

IEEE Transactions on Wireless Communications

ISSN :

1536-1276

eISSN :

1558-2248

Publisher :

Institute of Electrical and Electronics Engineers Inc.

Volume :

Issue :

Pages :

10414 - 10429

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

http://xplorestaging.ieee.org/ielx7/7693/10634844/10462923.pdf?arnumber=10462923

Funders :

National Natural Science Foundation of China
National Natural Science Foundation of China

Available on ORBilu :

since 26 March 2026

Statistics

Number of views

18 (0 by Unilu)

Number of downloads

10 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

WoS citations^™

Bibliography

P. Liu, J. Lei, and W. Liu, “A deep reinforcement learning scheme for SCMA-based edge computing in IoT networks,” in Proc. IEEE Global Commun. Conf., Dec. 2022, pp. 5044–5049.
Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Commun. Surveys Tuts., vol. 19, no. 4, pp. 2322–2358, 4th Quart., 2017.
Y. Mao, J. Zhang, and K. B. Letaief, “Dynamic computation offloading for mobile-edge computing with energy harvesting devices,” IEEE J. Sel. Areas Commun., vol. 34, no. 12, pp. 3590–3605, Dec. 2016.
X. Wang, Y. Zhang, R. Shen, Y. Xu, and F.-C. Zheng, “DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems,” IEEE Internet Things J., vol. 7, no. 8, pp. 7279–7294, Aug. 2020.
H. Nikopour and H. Baligh, “Sparse code multiple access,” in Proc. IEEE 24th Annu. Int. Symp. Pers., Indoor, Mobile Radio Commun. (PIMRC), Sep. 2013, pp. 332–336.
L. Yu et al., “Sparse code multiple access for 6G wireless communication networks: Recent advances and future directions,” IEEE Commun. Standards Mag., vol. 5, no. 2, pp. 92–99, Jun. 2021.
J. V. C. Evangelista, Z. Sattar, G. Kaddoum, and A. Chaaban, “Fairness and sum-rate maximization via joint subcarrier and power allocation in uplink SCMA transmission,” IEEE Trans. Wireless Commun., vol. 18, no. 12, pp. 5855–5867, Dec. 2019.
M. Moltafet, N. M. Yamchi, M. R. Javan, and P. Azmi, “Comparison study between PD-NOMA and SCMA,” IEEE Trans. Veh. Technol., vol. 67, no. 2, pp. 1830–1834, Feb. 2018.
D.-D. Tran, S. K. Sharma, S. Chatzinotas, and I. Woungang, “Q-learning-based SCMA for efficient random access in mMTC networks with short packets,” in Proc. IEEE 32nd Annu. Int. Symp. Pers., Indoor Mobile Radio Commun. (PIMRC), Sep. 2021, pp. 1334–1338.
Z. Ding, D. Xu, R. Schober, and H. V. Poor, “Hybrid NOMA offloading in multi-user MEC networks,” IEEE Trans. Wireless Commun., vol. 21, no. 7, pp. 5377–5391, Jul. 2022.
C. You, K. Huang, H. Chae, and B.-H. Kim, “Energy-efficient resource allocation for mobile-edge computation offloading,” IEEE Trans. Wireless Commun., vol. 16, no. 3, pp. 1397–1411, Mar. 2017.
S. Bi and Y. J. Zhang, “Computation rate maximization for wireless powered mobile-edge computing with binary computation offloading,” IEEE Trans. Wireless Commun., vol. 17, no. 6, pp. 4177–4190, Jun. 2018.
C. Wang, C. Liang, F. R. Yu, Q. Chen, and L. Tang, “Computation offloading and resource allocation in wireless cellular networks with mobile edge computing,” IEEE Trans. Wireless Commun., vol. 16, no. 8, pp. 4924–4938, Aug. 2017.
X. Dai et al., “A learning-based approach for vehicle-to-vehicle computation offloading,” IEEE Internet Things J., vol. 10, no. 8, pp. 7244–7258, Apr. 2023.
V. Vishnoi, I. Budhiraja, S. Gupta, and N. Kumar, “A deep reinforcement learning scheme for sum rate and fairness maximization among D2D pairs underlaying cellular network with NOMA,” IEEE Trans. Veh. Technol., vol. 72, no. 10, pp. 13506–13522, 2023.
I. Abu Mahady, E. Bedeer, S. Ikki, and H. Yanikomeroglu, “Sum-rate maximization of NOMA systems under imperfect successive interference cancellation,” IEEE Commun. Lett., vol. 23, no. 3, pp. 474–477, Mar. 2019.
L. Yu, H. Zhang, L. Zhang, L. Song, Z. Han, and P. Fan, “Hypergraphbased SCMA codebook allocation in user-centric ultra-dense networks with machine learning,” in Proc. 11th Int. Conf. Wireless Commun. Signal Process. (WCSP), Oct. 2019, pp. 1–6.
M. Moltafet, S. Parsaeefard, M. R. Javan, and N. Mokari, “Robust radio resource allocation in MISO-SCMA assisted C-RAN in 5G networks,” IEEE Trans. Veh. Technol., vol. 68, no. 6, pp. 5758–5768, Jun. 2019.
Y. J. Licea, K. Shen, and D. K. C. So, “Subcarrier and power allocation for sparse code multiple access,” in Proc. IEEE 91st Veh. Technol. Conf. (VTC-Spring), May 2020, pp. 1–5.
M. Cheraghy and W. Chen, “Resource allocation to maximize the average sum rate of the uplink SCMA networks,” in Proc. IEEE/CIC Int. Conf. Commun. China (ICCC), Nov. 2021, pp. 984–989.
Z. Ding, P. Fan, and H. V. Poor, “Impact of non-orthogonal multiple access on the offloading of mobile edge computing,” IEEE Trans. Commun., vol. 67, no. 1, pp. 375–390, Jan. 2019.
M. Zeng, R. Du, V. Fodor, and C. Fischione, “Computation rate maximization for wireless powered mobile edge computing with NOMA,” in Proc. IEEE 20th Int. Symp. World Wireless, Mobile Multimedia Netw. (WoWMoM), Jun. 2019, pp. 1–9.
Z. Ding, D. W. K. Ng, R. Schober, and H. V. Poor, “Delay minimization for NOMA-MEC offloading,” IEEE Signal Process. Lett., vol. 25, no. 12, pp. 1875–1879, Dec. 2018.
J. Shi, Y. Zhou, Z. Li, Z. Zhao, Z. Chu, and P. Xiao, “Delay minimization for NOMA-mmW scheme-based MEC offloading,” IEEE Internet Things J., vol. 10, no. 3, pp. 2285–2296, Feb. 2023.
S. Bi, L. Huang, and Y.-J. A. Zhang, “Joint optimization of service caching placement and computation offloading in mobile edge computing systems,” IEEE Trans. Wireless Commun., vol. 19, no. 7, pp. 4947–4963, Jul. 2020.
A. Kiani and N. Ansari, “Edge computing aware NOMA for 5G networks,” IEEE Internet Things J., vol. 5, no. 2, pp. 1299–1306, Apr. 2018.
C. Li, H. Wang, and R. Song, “Mobility-aware offloading and resource allocation in NOMA-MEC systems via DC,” IEEE Commun. Lett., vol. 26, no. 5, pp. 1091–1095, May 2022.
T. X. Tran and D. Pompili, “Joint task offloading and resource allocation for multi-server mobile-edge computing networks,” IEEE Trans. Veh. Technol., vol. 68, no. 1, pp. 856–868, Jan. 2019.
A. Alnoman, S. Erkucuk, and A. Anpalagan, “Sparse code multiple access-based edge computing for IoT systems,” IEEE Internet Things J., vol. 6, no. 4, pp. 7152–7161, Aug. 2019.
P. Liu, K. An, J. Lei, G. Zheng, Y. Sun, and W. Liu, “SCMA-based multiaccess edge computing in IoT systems: An energy-efficiency and latency tradeoff,” IEEE Internet Things J., vol. 9, no. 7, pp. 4849–4862, Apr. 2022.
P. Liu et al., “SCMA-enabled multi-cell edge computing networks: Design and optimization,” IEEE Trans. Veh. Technol., vol. 72, no. 6, pp. 7987-8003, Jun. 2023, doi: 10.1109/TVT.2023.3242422.
K. Wang, H. Li, Z. Ding, and P. Xiao, “Reinforcement learning based latency minimization in secure NOMA-MEC systems with hybrid SIC,” IEEE Trans. Wireless Commun., vol. 22, no. 1, pp. 408–422, Jan. 2023.
T. Li et al., “Applications of multi-agent reinforcement learning in future Internet: A comprehensive survey,” IEEE Commun. Surveys Tuts., vol. 24, no. 2, pp. 1240–1279, 2nd Quart., 2022.
A. D. Mafuta, B. T. J. Maharaj, and A. S. Alfa, “Decentralized resource allocation-based multiagent deep learning in vehicular network,” IEEE Syst. J., vol. 17, no. 1, pp. 87–98, Mar. 2023.
H. Sharma, N. Kumar, and R. Tekchandani, “Mitigating jamming attack in 5G heterogeneous networks: A federated deep reinforcement learning approach,” IEEE Trans. Veh. Technol., vol. 72, no. 2, pp. 2439–2452, Feb. 2023.
Z. Guo, Z. Chen, P. Liu, J. Luo, X. Yang, and X. Sun, “Multiagent reinforcement learning-based distributed channel access for next generation wireless networks,” IEEE J. Sel. Areas Commun., vol. 40, no. 5, pp. 1587–1599, May 2022.
C. Han, H. Yao, T. Mai, N. Zhang, and M. Guizani, “QMIX aided routing in social-based delay-tolerant networks,” IEEE Trans. Veh. Technol., vol. 71, no. 2, pp. 1952–1963, Feb. 2022.
Z. Gao, L. Yang, and Y. Dai, “Large-scale computation offloading using a multi-agent reinforcement learning in heterogeneous multiaccess edge computing,” IEEE Trans. Mobile Comput., vol. 22, no. 6, pp. 3425–3443, Jun. 2023.
S. Zhu, L. Gui, D. Zhao, N. Cheng, Q. Zhang, and X. Lang, “Learning-based computation offloading approaches in UAVs-assisted edge computing,” IEEE Trans. Veh. Technol., vol. 70, no. 1, pp. 928–944, Jan. 2021.
F. Khoramnejad and M. Erol-Kantarci, “On joint offloading and resource allocation: A double deep Q-network approach,” IEEE Trans. Cognit. Commun. Netw., vol. 7, no. 4, pp. 1126–1141, Dec. 2021.
A. Sacco, F. Esposito, G. Marchetto, and P. Montuschi, “Sustainable task offloading in UAV networks via multi-agent reinforcement learning,” IEEE Trans. Veh. Technol., vol. 70, no. 5, pp. 5003–5015, May 2021.
Z. Yang, Y. Liu, Y. Chen, and N. Al-Dhahir, “Cache-aided NOMA mobile edge computing: A reinforcement learning approach,” IEEE Trans. Wireless Commun., vol. 19, no. 10, pp. 6899–6915, Oct. 2020.
Z. Chen, L. Zhang, Y. Pei, C. Jiang, and L. Yin, “NOMA-based multi-user mobile edge computation offloading via cooperative multi-agent deep reinforcement learning,” IEEE Trans. Cogn. Commun. Netw., vol. 8, no. 1, pp. 350–364, Mar. 2022.
R. Han, H. Li, E. J. Knoblock, and R. D. Apaza, “Dynamic spectrum allocation in urban air transportation system via deep reinforcement learning,” in Proc. IEEE/AIAA 40th Digit. Avionics Syst. Conf. (DASC), Oct. 2021, pp. 1–10.
Y. Jia, C. Zhang, Y. Huang, and W. Zhang, “Lyapunov optimization based mobile edge computing for Internet of Vehicles systems,” IEEE Trans. Commun., vol. 70, no. 11, pp. 7418–7433, Nov. 2022.
C. Liu, F. Tang, Y. Hu, K. Li, Z. Tang, and K. Li, “Distributed task migration optimization in MEC by extending multi-agent deep reinforcement learning approach,” IEEE Trans. Parallel Distrib. Syst., vol. 32, no. 7, pp. 1603–1614, Jul. 2021.
L. Yu, P. Fan, D. Cai, and Z. Ma, “Design and analysis of SCMA codebook based on star-QAM signaling constellations,” IEEE Trans. Veh. Technol., vol. 67, no. 11, pp. 10543–10553, Nov. 2018.
X. Li, Z. Gao, Y. Gui, Z. Liu, P. Xiao, and L. Yu, “Design of power-imbalanced SCMA codebook,” IEEE Trans. Veh. Technol., vol. 71, no. 2, pp. 2140–2145, Feb. 2022.
P. Hernandez-Leal, M. Kaisers, T. Baarslag, and E. M. de Cote, “A survey of learning in multiagent environments: Dealing with nonstationarity,” 2017, arXiv:1707.09183.
J. Li et al., “Shapley counterfactual credits for multi-agent reinforcement learning,” in Proc. 27th ACM SIGKDD Conf. Knowl. Discovery Data Mining, Aug. 2021, pp. 934–942.
S. Hu et al., “MARLlib: A scalable and efficient multi-agent reinforcement learning library,” 2022, arXiv:2210.13708.