Automotove Industry 5.0; Wehicular Edge Computing; Task Offloading
Abstract :
[en] The rapid growth of Automotive-Industry 5.0 and its emergence with beyond fifth-generation (B5G) communications, is making vehicular edge computing networks (VECNs) increasingly complex. The latency constraints of modern automotive applications make it difficult to run complex applications on vehicle on-board units (OBUs). While multi-access edge computing (MEC) can facilitate task offloading to execute these applications, it is still a challenge to access them promptly and optimally. Traditional algorithms struggle to guarantee accuracy in such dynamic environment, but deep reinforcement learning (DRL) methods offer improved accuracy, robustness, and real-time decision-making capabilities. In this paper, we propose a DRL-based mobility, contact, and load aware cooperative task offloading (DCTO) scheme. DCTO is designed for both cellular and mmWave radio access technologies (RATs), and both binary and partial offloading mechanisms. DCTO targets delay minimization by opportunistically switching RATs and offloading mechanisms. We consider relative efficacy and neutrality factors as key performance indicators and use them to derive the DRL agent’s reward function. Extensive evaluations demonstrate that the DCTO scheme exhibits a substantial enhancement in task success rate, with an increase from 2.61% to 21.34%. It also improves the efficacy factor from 1.38 to 3.52 and reduces the neutrality factor from 4.99 to 0.76. Furthermore, the average task processing time is reduced by a range of 3.77% to 24.15%. Additionally, the DCTO scheme outperforms the other evaluated schemes in terms of reward and TFPS ratio.
Disciplines :
Electrical & electronics engineering
Author, co-author :
Mirza, M. Ayzed
Junsheng, Yu
Raza, Salman
Krichen, Moez
Ahmed, Manzoor
Khan, Wali Ullah ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom
Rabie, Khaled
Shongwe, Thokozani
External co-authors :
yes
Language :
English
Title :
DRL-Assisted Delay Optimized Task Offloading in Automotive-Industry 5.0 based VECNs
Alternative titles :
[en] DRL-Assisted Delay Optimized Task Offloading in Automotive-Industry 5.0 based VECNs
Publication date :
04 February 2023
Journal title :
Journal of King Saud University - Computer and Information Sciences
3GPP, 2019. Study on evaluation methodology of new vehicle-to-everything v2x use cases for lte and nr (release 15). 3gpp rel 15, no. TR 37.885.
Ahmed, M., Raza, S., Mirza, M.A., Aziz, A., Khan, M.A., Khan, W.U., Li, J., Han, Z., A survey on vehicular task offloading: Classification, issues, and challenges. J. King Saud Univ.-Compu. Informat. Sci. 34:7 (2022), 4135–4162.
Boukerche, A., Sotoro, V., Computation offloading and retrieval for vehicular edge computing: Algorithms, model and classification. ACM Comput. Surv. (CSUR) 53:4 (August 2020), 1–35.
Chen, C., Zeng, Y., Li, H., Liu, Y., Wan, S., A multi-hop task offloading decision model in mec-enabled internet of vehicles. IEEE Internet Things J., 2022 1–1.
Chen, C., Li, H., Li, H., Fu, R., Liu, Y., Wan, S., Efficiency and fairness oriented dynamic task offloading in internet of vehicles. IEEE Trans. Green Commun. Network. 6:3 (2022), 1481–1493.
Cui, Y., Du, L., Wang, H., Wu, D., Wang, R., Reinforcement learning for joint optimization of communication and computation in vehicular networks. IEEE Trans. Vehicular Technol. 70:12 (2021), 13062–13072.
Degris, T., White, M., Sutton, R.S., 2012. Linear off-policy actor-critic. In: Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, June 26 - July 1, 2012. icml.cc/ Omnipress.
Deng, X., Sun, Z., Li, D., Luo, J., Wan, S., User-centric computation offloading for edge computing. IEEE Internet Things J. 8:16 (2021), 12559–12568.
Du, J., Sun, Y., Zhang, N., Xiong, Z., Sun, A., Ding, Z., Cost-effective task offloading in noma-enabled vehicular mobile edge computing. IEEE Syst. J., 2022.
Gu, L., Xu, X., Qi, L., Zhang, Y., Zhang, X., Dou, W., 2021. Cooperative task offloading for internet of vehicles in cloud-edge computing. In: 2021 IEEE 23rd Int Conf on High Performance Computing & Communications; 7th Int Conf on Data Science & Systems; 19th Int Conf on Smart City; 7th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). IEEE, 2021, pp. 1537–1544.
ITU, 2021. Land mobile (including wireless access) - volume 4: Intelligent transport systems. In: ITU-R WP5A. Radiocommunication Bureau, ITU, March 2021. [Online]. Available: http://handle.itu.int/11.1002/pub/81734039-en.
Jiang, X., Yu, F.R., Song, T., Leung, V.C., Resource allocation of video streaming over vehicular networks: a survey, some research issues and challenges. IEEE Trans. Intell. Transp. Syst. 23:7 (March 2021), 5955–5975.
Jin, H., Gregory, M.A., Li, S., A review of intelligent computation offloading in multi-access edge computing. IEEE Access 10 (2022), 71481–71495.
Khan, W.U., Ihsan, A., Nguyen, T.N., Ali, Z., Javed, M.A., Noma-enabled backscatter communications for green transportation in automotive-industry 5.0. IEEE Trans. Industr. Inf. 18:11 (2022), 7862–7874.
Li, Y., 2017. Deep reinforcement learning: An overview, arXiv preprint arXiv:1701.07274.
Liu, S., Yu, J., Deng, X., Wan, S., Fedcpf: An efficient-communication federated learning approach for vehicular edge computing in 6g communication networks. IEEE Trans. Intell. Transp. Syst. 23:2 (2021), 1616–1629.
Liu, J., Ahmed, M., Mirza, M.A., Khan, W.U., Xu, D., Li, J., Aziz, A., Han, Z., RL/DRL meets vehicular task offloading using edge and vehicular cloudlet: A survey. IEEE Internet Things J. 9:11 (2022), 8315–8338.
Luo, Q., Li, C., Luan, T., Shi, W., Minimizing the delay and cost of computation offloading for vehicular edge computing. IEEE Trans. Services Comput., 2021.
Lv, P., Xu, W., Nie, J., Yuan, Y., Cai, C., Chen, Z., Xu, J., Edge computing task offloading for environmental perception of autonomous vehicles in 6g networks. IEEE Trans. Network Sci. Eng., 2022, 1–18.
Naik, G., Choudhury, B., Park, J.-M., IEEE 802.11 bd & 5G NR V2X: Evolution of radio access technologies for V2X communications. IEEE Access 7 (2019), 70169–70184.
Nguyen, K., Drew, S., Huang, C., Zhou, J., Parked vehicles task offloading in edge computing. IEEE Access 10 (2022), 41592–41606.
Raza, S., Wang, S., Ahmed, M., Anwar, M.R., Mirza, M.A., Khan, W.U., Task offloading and resource allocation for iov using 5g nr-v2x communication. IEEE Internet Things J. 9:13 (2021), 10 97–10410.
Raza, S., Ahmed, M., Ahmad, H., Mirza, M.A., Habib, M.A., Wang, S., Task offloading in mmwave based 5g vehicular cloud computing. J. Ambient Intell. Humanized Comput., 2022, 1–13.
Schulman, J., Moritz, P., Levine, S., Jordan, M.I., Abbeel, P., 2016. High-dimensional continuous control using generalized advantage estimation. In: Bengio, Y., LeCun, Y. (Eds.), 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, Conference Track Proceedings.
Shibata, Y., Sakuraba, A., Sato, G., Uchida, N., Iot based wide area road surface state sensing and communication system for future safety driving. International Conference on Advanced Information Networking and Applications, Matsue, Japan, March 2019, 2019, 1123–1132.
Shuai, R., Wang, L., Guo, S., Zhang, H., Adaptive task offloading in vehicular edge computing networks based on deep reinforcement learning. 2021 IEEE/CIC International Conference on Communications in China (ICCC), 2021, IEEE, Xiamen, China.
Shu, W., Li, Y., Joint offloading strategy based on quantum particle swarm optimization for mec-enabled vehicular networks. Digital Commun. Networks, 2022.
Tang, F., Mao, B., Kato, N., Gui, G., Comprehensive survey on machine learning in vehicular network: technology, applications and challenges. IEEE Commun. Surv. Tutor. 23:3 (2021), 2027–2057.
Tang, H., Wu, H., Qu, G., Li, R., Double deep q-network based dynamic framing offloading in vehicular edge computing. IEEE Trans. Network Sci. Eng., 2022.
Van Hasselt, H., Guez, A., Silver, D., 2016. Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1, Phoenix, Arizona USA.
Wang, Y., Sheng, M., Wang, X., Wang, L., Li, J., Mobile-edge computing: Partial computation offloading using dynamic voltage scaling. IEEE Trans. Commun. 64:10 (2016), 4268–4282.
Wang, H., Li, X., Ji, H., Zhang, H., Federated offloading scheme to minimize latency in mec-enabled vehicular networks. 2018 IEEE Globecom Workshops (GC Wkshps), 2018, IEEE, 1–6.
Wang, D., Song, B., Lin, P., Yu, F.R., Du, X., Guizani, M., Resource management for edge intelligence (ei)-assisted iov using quantum-inspired reinforcement learning. IEEE Internet Things J. 9:14 (2022), 12588–12600.
Yao, L., Xu, X., Bilal, M., Wang, H., Dynamic edge computation offloading for internet of vehicles with deep reinforcement learning. IEEE Trans. Intell. Transport. Syst., 2022, 1–9.
Ye, Q., Shi, W., Qu, K., He, H., Zhuang, W., Shen, X., Joint ran slicing and computation offloading for autonomous vehicular networks: A learning-assisted hierarchical approach. IEEE Open J. Vehicular Technol. 2 (June 2021), 272–288.
Zhang, S., Gu, H., Chi, K., Huang, L., Yu, K., Mumtaz, S., Drl-based partial offloading for maximizing sum computation rate of wireless powered mobile edge computing network. IEEE Trans. Wireless Commun. 21:12 (2022), 10934–10948.
Zhang, Q., Wen, H., Liu, Y., Chang, S., Han, Z., Federated reinforcement learning enabled joint communication, sensing and computing resources allocation in connected automated vehicles networks. IEEE Internet Things J., 12, 2002 1-1.
Zhang, L., Zhou, W., Xia, J., Gao, C., Zhu, F., Fan, C., Ou, J., Dqn-based mobile edge computing for smart internet of vehicle. EURASIP J. Adv. Signal Process. 2022:1 (2022), 1–16.
Zhou, Z., Wang, Z., Yu, H., Liao, H., Mumtaz, S., Oliveira, L., Frascolla, V., Learning-based urllc-aware task offloading for internet of health things. IEEE J. Sel. Areas Commun. 39:2 (2020), 396–410.
