asynchronous federated learning; LEO satellite; meta-reinforcement learning; multi-objective optimization; 6g mobile communication; Asynchronoi federated learning; Computational modelling; Low earth orbit satellites; Meta-reinforcement learning; Mobile communications; Multi-objectives optimization; Optimisations; Reinforcement learnings; Computer Networks and Communications; Electrical and Electronic Engineering; Computational modeling; Optimization; Training; Data models; Satellites
Abstract :
[en] Wireless-based federated learning (FL), as an emerging distributed learning approach, has been widely studied for 6G systems. When the paradigm shifts from terrestrial to non-terrestrial networks (NTN), FL may need to address several open challenges, e.g., the limited service time of low earth orbit (LEO) satellites, the straggler issue in synchronous FL, and time-efficient uploading and aggregation for massive devices. In this work, we exploit the synergy of LEO and FL for future integrated 6G-satellite systems by taking advantage of ubiquitous wireless access provided by LEO and appealing characteristics of collaborative training and data privacy preservation in FL. The studied LEO-FL framework may need to improve multi-metric performance in practice. Different from most FL works, we simultaneously improve the communication-training efficiency and local training accuracy from a multi-objective optimization (MOO) perspective. To solve the problem, we propose a decomposition and meta-deep reinforcement learning based MOO algorithm for FL (DMMA-FL), aiming at adapting to the dynamic satellite-terrestrial environments, achieving efficient uploading and aggregation, and approaching Pareto optimal sets. Compared to single-objective optimization, heuristics-based, and learning-based MOO algorithms, the effectiveness and advantages of the proposed LEO-FL framework and DMMA-FL algorithm are assessed on MNIST and CIFAR-10 datasets.
Disciplines :
Computer science
Author, co-author :
ZHOU, Yunxiu ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX ; Xi'an Jiaotong University, School of Information and Communications Engineering, Xi'an, China ; The Hong Kong Polytechnic University, Department of Computing, Hong Kong, Hong Kong
LEI, Lei ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust > SigCom > Team Symeon CHATZINOTAS ; Xi'an Jiaotong University, School of Information and Communications Engineering, Xi'an, China ; Southeast University, National Mobile Communications Research Laboratory, Nanjing, China
Zhao, Xiaohui; Xi'an Jiaotong University, School of Information and Communications Engineering, Xi'an, China
You, Lei ; Technical University of Denmark, Department of Engineering Technology, Ballerup, Denmark
Sun, Yaohua ; Beijing University of Posts and Telecommunications, State Key Laboratory of Networking and Switching Technology (SKL-NST), Beijing, China
CHATZINOTAS, Symeon ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom
External co-authors :
yes
Language :
English
Title :
Decomposition and Meta-DRL Based Multi-Objective Optimization for Asynchronous Federated Learning in 6G-Satellite Systems
Publication date :
May 2024
Journal title :
IEEE Journal on Selected Areas In Communications
ISSN :
0733-8716
eISSN :
1558-0008
Publisher :
Institute of Electrical and Electronics Engineers Inc.
Open Research Fund of National Mobile Communications Research Laboratory of Southeast University National Key Laboratory Foundation of China Qin Chuangyuan Innovation and Talent Project Natural Science Foundation of Sichuan Province National Natural Science Foundation of China Young Elite Scientists Sponsorship Program by the China Association for Science and Technology
Funding text :
This work was supported in part by the Open Research Fund of National Mobile Communications Research Laboratory of Southeast University under Grant 2023D02, in part by the National Key Laboratory Foundation under Grant 2023-JCJQ-LB-007, in part by the Qin Chuangyuan Innovation and Talent Project under Grant QCYRCXM-2023-049, in part by the Natural Science Foundation of Sichuan Province under Grant 2023NSFSC0455, in part by the National Natural Science Foundation of China under Grant 62371071, and in part by the Young Elite Scientists Sponsorship Program by the China Association for Science and Technology (CAST) under Grant 2021QNRC001.
J. A. Guerrero-Ibanez, S. Zeadally, and J. Contreras-Castillo, "Integration challenges of intelligent transportation systems with connected vehicle, cloud computing, and Internet of Things technologies, " IEEE Wireless Commun., vol. 22, no. 6, pp. 122-128, Dec. 2015.
M. Chen, N. Shlezinger, H. V. Poor, Y. C. Eldar, and S. Cui, "Communication-efficient federated learning, " Proc. Nat. Acad. Sci. USA, vol. 118, no. 17, 2021, Art. no. e2024789118.
B. McMahan et al., "Communication-efficient learning of deep networks from decentralized data, " in Proc. 20th Int. Conf. Artif. Intell. Statist., 2017, pp. 1273-1282.
L. U. Khan et al., "Federated learning for edge networks: Resource optimization and incentive mechanism, " IEEE Communs. Mag., vol. 58, no. 10, pp. 88-93, Oct. 2020.
W. Y. B. Lim, J. S. Ng, Z. Xiong, D. Niyato, C. Miao, and D. I. Kim, "Dynamic edge association and resource allocation in self-organizing hierarchical federated learning networks, " IEEE J. Sel. Areas Commun., vol. 39, no. 12, pp. 3640-3653, Dec. 2021.
S. Wang et al., "Adaptive federated learning in resource constrained edge computing systems, " IEEE J. Sel. Areas Commun., vol. 37, no. 6, pp. 1205-1221, Jun. 2019.
H. Al-Hraishawi, H. Chougrani, S. Kisseleff, E. Lagunas, and S. Chatzinotas, "A survey on nongeostationary satellite systems: The communication perspective, " IEEE Commun. Surveys Tuts., vol. 25, no. 1, pp. 101-132, 1st Quart., 2023.
N. Razmi, B. Matthiesen, A. Dekorsy, and P. Popovski, "Ground-assisted federated learning in LEO satellite constellations, " IEEE Wireless Commun. Lett., vol. 11, no. 4, pp. 717-721, Apr. 2022.
H. Chen, M. Xiao, and Z. Pang, "Satellite-based computing networks with federated learning, " IEEE Wireless Commun., vol. 29, no. 1, pp. 78-84, Feb. 2022.
B. Matthiesen, N. Razmi, I. Leyva-Mayorga, A. Dekorsy, and P. Popovski, "Federated learning in satellite constellations, " IEEE Netw., early access, May 8, 2023, doi: 10.1109/MNET.132.2200504.
M. Chen, H. V. Poor, W. Saad, and S. Cui, "Convergence time optimization for federated learning over wireless networks, " IEEE Trans. Wireless Commun., vol. 20, no. 4, pp. 2457-2471, Apr. 2021.
R. Chen, D. Shi, X. Qin, D. Liu, M. Pan, and S. Cui, "Service delay minimization for federated learning over mobile devices, " IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 990-1006, Apr. 2023.
Y. Chen, Y. Ning, M. Slawski, and H. Rangwala, "Asynchronous online federated learning for edge devices with non-IID data, " in Proc. IEEE Int. Conf. Big Data (Big Data), Mar. 2020, pp. 15-24.
C.-H. Hu, Z. Chen, and E. G. Larsson, "Scheduling and aggregation design for asynchronous federated learning over wireless networks, " IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 874-886, Apr. 2023.
J. Nguyen et al., "Federated learning with buffered asynchronous aggregation, " in Proc. 25th Int. Conf. Artif. Intell. Statist., vol. 151, 2022, pp. 3581-3607.
Y. Yuan, L. Lei, T. X. Vu, S. Fowler, and S. Chatzinotas, "Efficient resource scheduling and optimization for over-loaded LEOterrestrial networks, " in Proc. IEEE Int. Conf. Commun., May 2022, pp. 1052-1057.
S. Wang, Y. Li, Q. Wang, M. Su, and W. Zhou, "Dynamic downlink resource allocation based on imperfect estimation in LEO-HAP cognitive system, " in Proc. 11th Int. Conf. Wireless Commun. Signal Process. (WCSP), Oct. 2019, pp. 1-6.
Y. Luo, J. Xu, W. Xu, and K. Wang, "Sliding differential evolution scheduling for federated learning in bandwidth-limited networks, " IEEE Commun. Lett., vol. 25, no. 2, pp. 503-507, Feb. 2021.
B. Shen, Y. Wu, J. An, C. Xing, L. Zhao, and W. Zhang, "Random access with massive MIMO-OTFS in LEO satellite communications, " IEEE J. Sel. Areas Commun., vol. 40, no. 10, pp. 2865-2881, Oct. 2022.
W. U. Khan et al., "Rate splitting multiple access for next generation cognitive radio enabled LEO satellite networks, " IEEE Trans. Wireless Commun., vol. 22, no. 11, pp. 8423-8435, Nov. 2023.
W. Lim et al., "Federated learning in mobile edge networks:1 A comprehensive survey, " IEEE Commun. Surveys Tuts., vol. 22, no. 3, pp. 2031-2063, 3rd Quart., 2020.
V.-D. Nguyen, S. K. Sharma, T. X. Vu, S. Chatzinotas, and B. Ottersten, "Efficient federated learning algorithm for resource allocation in wireless IoT networks, " IEEE Internet Things J., vol. 8, no. 5, pp. 3394-3409, Mar. 2021.
M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, "A joint learning and communications framework for federated learning over wireless networks, " IEEE Trans. Wireless Commun., vol. 20, no. 1, pp. 269-283, Jan. 2021.
B. Li, A. Swami, and S. Segarra, "Power allocation for wireless federated learning using graph neural networks, " in Proc. ICASSP-IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), May 2022, pp. 5243-5247.
J. Liu, Z. Chang, G. Min, and Z. Han, "Incentive mechanism design for federated learning in multi-access edge computing, " in Proc. IEEE Global Commun. Conf., Dec. 2022, pp. 3454-3459.
M. Ma and V. W. S. Wong, "Age of information driven cache content update scheduling for dynamic contents in heterogeneous networks, " IEEE Trans. Wireless Commun., vol. 19, no. 12, pp. 8427-8441, Dec. 2020.
K. Li, T. Zhang, and R. Wang, "Deep reinforcement learning for multiobjective optimization, " IEEE Trans. Cybern., vol. 51, no. 6, pp. 3103-3114, Jun. 2021.
Z. Zhang, Z. Wu, H. Zhang, and J. Wang, "Meta-learning-based deep reinforcement learning for multiobjective optimization problems, " IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 10, pp. 7978-7991, Oct. 2023.
Y. Yuan, L. Lei, T. X. Vu, Z. Chang, S. Chatzinotas, and S. Sun, "Adapting to dynamic LEO-B5G systems: Meta-critic learning based efficient resource scheduling, " IEEE Trans. Wireless Commun., vol. 21, no. 11, pp. 9582-9595, Nov. 2022.
J. Ye and H. Gharavi, "Deep reinforcement learning-assisted energy harvesting wireless networks, " IEEE Trans. Green Commun. Netw., vol. 5, no. 2, pp. 990-1002, Jun. 2021.
Y. Xu, K. Zhu, H. Xu, and J. Ji, "Deep reinforcement learning for multi-objective resource allocation in multi-platoon cooperative vehicular networks, " IEEE Trans. Wireless Commun., vol. 22, no. 9, pp. 6185-6198, Sep. 2023.
S. Luo, L. Zhang, and Y. Fan, "Real-time scheduling for dynamic partialno-wait multiobjective flexible job shop by deep reinforcement learning, " IEEE Trans. Autom. Sci. Eng., vol. 19, no. 4, pp. 3020-3038, Oct. 2022.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
A. Krizhevsky, "Learning multiple layers of features from tiny images, " Univ. Toronto, Toronto, ON, Canada, Tech. Rep. Krizhevsky-2009-17719, 2009.
B. Di, H. Zhang, L. Song, Y. Li, and G. Y. Li, "Ultra-dense LEO: Integrating terrestrial-satellite networks into 5G and beyond for data offloading, " IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 47-62, Jan. 2019.
S. Xia, Q. Jiang, C. Zou, and G. Li, "Beam coverage comparison of LEO satellite systems based on user diversification, " IEEE Access, vol. 7, pp. 181656-181667, 2019.
T. Nishio and R. Yonetani, "Client selection for federated learning with heterogeneous resources in mobile edge, " in Proc. IEEE Int. Conf. Commun., Jun. 2019, pp. 1-7.
H.-G. Han, Z. Liu, W. Lu, Y. Hou, and J.-F. Qiao, "Dynamic MOPSObased optimal control for wastewater treatment process, " IEEE Trans. Cybern., vol. 51, no. 5, pp. 2518-2528, May 2021.
H. Zhu and Y. Jin, "Multi-objective evolutionary federated learning, " IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 4, pp. 1310-1322, Apr. 2020.
Y. Zhou, X. Liu, and L. Lei, "Multi-objective optimization for bandwidth-limited federated learning in wireless edge systems, " IEEE Open J. Commun. Soc., vol. 4, pp. 954-966, 2023.
X. Tu and K. Zhu, "Learning-based multi-objective resource allocation for over-the-air federated learning, " in Proc. IEEE Global Commun. Conf., Dec. 2022, pp. 3065-3070.
T. Xu, Y. Liu, Z. Ma, Y. Huang, and P. Liu, "A DQN-based multiobjective participant selection for efficient federated learning, " Future Internet, vol. 15, no. 6, p. 209, Jun. 2023.
Y. Jing, C. Jiang, N. Ge, and L. Kuang, "Resource optimization for signal recognition in satellite MEC with federated learning, " in Proc. 13th Int. Conf. Wireless Commun. Signal Process. (WCSP), Oct. 2021, pp. 1-5.
W. Zhang et al., "Optimizing federated learning in distributed industrial IoT: A multi-agent approach, " IEEE J. Sel. Areas Commun., vol. 39, no. 12, pp. 3688-3703, Dec. 2021.
C. Feng, Z. Zhao, Y. Wang, T. Q. S. Quek, and M. Peng, "On the design of federated learning in the mobile edge computing systems, " IEEE Trans. Commun., vol. 69, no. 9, pp. 5902-5916, Sep. 2021.
S. Samarakoon, M. Bennis, W. Saad, and M. Debbah, "Distributed federated learning for ultra-reliable low-latency vehicular communications, " IEEE Trans. Commun., vol. 68, no. 2, pp. 1146-1159, Feb. 2020.
Z. Wang et al., "Federated learning via intelligent reflecting surface, " IEEE Trans. Wireless Commun., vol. 21, no. 2, pp. 808-822, Feb. 2022.
J. Bader and E. Zitzler, "HypE: An algorithm for fast hypervolume-based many-objective optimization, " Evol. Comput., vol. 19, no. 1, pp. 45-76, Mar. 2011.
S. Liu, J. Yu, X. Deng, and S. Wan, "FedCPF: An efficientcommunication federated learning approach for vehicular edge computing in 6G communication networks, " IEEE Trans. Intell. Transp. Syst., vol. 23, no. 2, pp. 1616-1629, Feb. 2022.
W. Wu, L. He, W. Lin, R. Mao, C. Maple, and S. Jarvis, "SAFA: A semiasynchronous protocol for fast federated learning with low overhead, " IEEE Trans. Comput., vol. 70, no. 5, pp. 655-668, May 2021.
