Deep reinforcement learning approach for multi-hop task offloading in vehicular edge computing

The rise of Vehicular Edge Computing (VEC) has gained attention for its ability to alleviate backhaul network load and provide ultra-low latency. In meeting the escalating computational needs of cutting-edge vehicular applications such as augmented reality and autonomous driving, the abundant computational resources of vehicles can prove critical for task computation in a VEC environment. Nevertheless, the high mobility of vehicles has the potential to disrupt ongoing task computation due to varying communication network connectivity. This paper proposes a task offloading scheme that leverages multi-hop vehicle computation resources in Vehicle-to-Vehicle (V2V) communication, relying on mobility analysis. Vehicles capable of fulfilling the requisite communication and computation demands via multi-hop connectivity can assist in performing tasks offloaded by the client vehicle, along with the single-hop vehicles in the vicinity of the client vehicle. We formulate an NP-hard optimization problem for task offloading to minimize all tasks’ weighted sum of computation delay. For this, a proximal policy optimization-based Multi-hop Vehicular Task Offloading (MVTO-PPO) scheme in vehicular edge computing is designed for low complexity that provides the optimal solution. Our approach involved modeling the task offloading process as a Markov decision process. We then developed an offloading decision algorithm that utilizes deep reinforcement learning to choose the appropriate vehicle for task execution. This approach improves the quality of environmental perception by enabling reasonable task offloading, ultimately leading to significant long-term benefits. Furthermore, we explore the integration of fifth-generation new-radio vehicle-to-everything (5G NR V2X) communication, utilizing both cellular links and millimeter wave technology to enhance system performance. Simulation results demonstrate that the proposed algorithm significantly reduces task offloading delays, outperforming benchmark approaches in various scenarios.