RL/DRL Meets Vehicular Task Offloading Using Edge and Vehicular Cloudlet: A Survey

Liu, Jinshi; Ahmed, Manzoor; Mirza, Muhammad Ayzed; KHAN, Wali Ullah; Xu, Dianlei; Li, Jianbo; Aziz, Abdul; Han, Zhu

doi:10.1109/JIOT.2022.3155667

Request a copy

Article (Scientific journals)

RL/DRL Meets Vehicular Task Offloading Using Edge and Vehicular Cloudlet: A Survey

Liu, Jinshi; Ahmed, Manzoor; Mirza, Muhammad Ayzed et al.

2022 • In IEEE Internet of Things Journal

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/10993/54097

DOI
10.1109/JIOT.2022.3155667

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

RL_DRL_Meets_Vehicular_Task_Offloading_Using_Edge_and_Vehicular_Cloudlet_A_Survey.pdf

Publisher postprint (4.49 MB)

Request a copy

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Vehicular task offloading; Reinforcement learning; Survey

Abstract :

[en] The last two decades have seen a clear trend toward crafting intelligent vehicles based on the significant advances in communication and computing paradigms, which provide a safer, stress-free, and more enjoyable driving experience. Moreover, emerging applications and services necessitate massive volumes of data, real-time data processing, and ultrareliable and low-latency communication (URLLC). However, the computing capability of current intelligent vehicles is minimal, making it challenging to meet the delay-sensitive and computation-intensive demand of such applications. In this situation, vehicular task/computation offloading toward the edge cloud (EC) and vehicular cloudlet (VC) seems to be a promising solution to improve the network’s performance and applications’ Quality of Service (QoS). At the same time, artificial intelligence (AI) has dramatically changed people’s lives. Especially for vehicular task offloading applications, AI achieves state-of-the-art performance in various vehicular environments. Motivated by the outstanding performance of integrating reinforcement learning (RL)/deep RL (DRL) to the vehicular task offloading systems, we present a survey on various RL/DRL techniques applied to vehicular task offloading. Precisely, we classify the vehicular task offloading works into two main categories: 1) RL/ DRL solutions leveraging EC and 2) RL/DRL solutions using VC computing. Moreover, the EC section-based RL/DRL solutions are further subcategorized into multiaccess edge computing (MEC) server, nearby vehicles, and hybrid MEC (HMEC). To the best of our knowledge, we are the first to cover RL/DRL-based vehicular task offloading. Also, we provide lessons learned and open research challenges in this field and discuss the possible trend for future research.

Disciplines :

Electrical & electronics engineering

Author, co-author :

Liu, Jinshi

Ahmed, Manzoor

Mirza, Muhammad Ayzed

KHAN, Wali Ullah ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SigCom

Xu, Dianlei

Li, Jianbo

Aziz, Abdul

Han, Zhu

External co-authors :

yes

Language :

English

Title :

RL/DRL Meets Vehicular Task Offloading Using Edge and Vehicular Cloudlet: A Survey

Alternative titles :

[en] RL/DRL Meets Vehicular Task Offloading Using Edge and Vehicular Cloudlet: A Survey

Publication date :

01 March 2022

Journal title :

IEEE Internet of Things Journal

eISSN :

2327-4662

Publisher :

Institute of Electrical and Electronics Engineers

Peer reviewed :

Peer Reviewed verified by ORBi

Focus Area :

Security, Reliability and Trust

Additional URL :

https://ieeexplore-ieee-org.proxy.bnl.lu/document/9723463

Available on ORBilu :

since 25 January 2023

Statistics

Number of views

31 (0 by Unilu)

Number of downloads

0 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenAlex citations

WoS citations^™

Bibliography

J. Guo, B. Song, Y. He, F. R. Yu, and M. Sookhak, "A survey on compressed sensing in vehicular infotainment systems, " IEEE Commun. Surveys Tuts., vol. 19, no. 4, pp. 2662-2680, 4th Quart., 2017.
E. Uhlemann, "Connected-vehicles applications are emerging [connected vehicles], " IEEE Veh. Technol. Mag., vol. 11, no. 1, pp. 25-96, Mar. 2016.
P. K. Singh, S. K. Nandi, and S. Nandi, "A tutorial survey on vehicular communication state of the art, and future research directions, " Veh. Commun., vol. 18, Aug. 2019, Art. no. 100164.
A. Boukerche and V. Soto, "Computation offloading and retrieval for vehicular edge computing: Algorithms, models, and classification, " ACM Comput. Surveys, vol. 53, no. 4, pp. 1-35, 2021.
M. A. Khan, "A survey of computation offloading strategies for performance improvement of applications running on mobile devices, " J. Netw. Comput. Appl., vol. 56, pp. 28-40, Oct. 2015.
Y. Liu et al., "Dependency-aware task scheduling in vehicular edge computing, " IEEE Internet Things J., vol. 7, no. 6, pp. 4961-4971, Jun. 2020.
F. Arena and G. Pau, "An overview of vehicular communications, " Future Internet, vol. 11, no. 2, p. 27, Feb. 2019.
J. Wang, C. Jiang, K. Zhang, T. Q. S. Quek, Y. Ren, and L. Hanzo, "Vehicular sensing networks in a smart city: Principles, technologies and applications, " IEEE Wireless Commun., vol. 25, no. 1, pp. 122-132, Feb. 2018.
H. Vahdat-Nejad, A. Ramazani, T. Mohammadi, and W. Mansoor, "A survey on context-aware vehicular network applications, " Veh. Commun., vol. 3, pp. 43-57, Jan. 2016.
A. Boukerche and E. Robson, "Vehicular cloud computing: Architectures, applications, and mobility, " Comput. Netw., vol. 135, pp. 171-189, Apr. 2018.
K. Zhang, Y. Mao, S. Leng, Y. He, and Y. Zhang, "Mobile-edge computing for vehicular networks: A promising network paradigm with predictive off-loading, " IEEE Veh. Technol. Mag., vol. 12, no. 2, pp. 36-44, Jun. 2017.
H. El-Sayed and M. Chaqfeh, "Exploiting mobile edge computing for enhancing vehicular applications in smart cities, " Sensors, vol. 19, no. 5, p. 1073, Jan. 2019.
X. Hou et al., "Reliable computation offloading for edge-computingenabled software-defined IoV, " IEEE Internet Things J., vol. 7, no. 8, pp. 7097-7111, Aug. 2020.
I. A. Abbasi and A. Shahid Khan, "A review of vehicle to vehicle communication protocols for VANETs in the urban environment, " Future Internet, vol. 10, no. 2, pp. 14-28, Feb. 2018.
C. Silva, L. Silva, L. Santos, J. Sarubbi, and A. Pitsillides, "Broadening understanding on managing the communication infrastructure in vehicular networks: Customizing the coverage using the delta network, " Future Internet, vol. 11, no. 1, pp. 1-19, Jan. 2019.
C. R. Storck and F. Duarte-Figueiredo, "A survey of 5G technology evolution, standards, and infrastructure associated with vehicle-toeverything communications by Internet of Vehicles, " IEEE Access, vol. 8, pp. 117593-117614, 2020.
W. U. Khan, X. Li, A. Ihsan, M. A. Khan, V. G. Menon, and M. Ahmed, "NOMA-enabled optimization framework for nextgeneration small-cell IoV networks under imperfect SIC decoding, " IEEE Trans. Intell. Transp. Syst., early access, Jun. 29, 2021, doi: 10. 1109/TITS. 2021. 3091402.
K. Ganesan, J. Lohr, P. B. Mallick, A. Kunz, and R. Kuchibhotla, "NR sidelink design overview for advanced V2X service, " IEEE Internet Things Mag., vol. 3, no. 1, pp. 26-30, Mar. 2020.
J. Wang, C. Jiang, H. Zhang, Y. Ren, K.-C. Chen, and L. Hanzo, "Thirty years of machine learning: The road to pareto-optimal wireless networks, " IEEE Commun. Surveys Tuts., vol. 22, no. 3, pp. 1472-1514, 3rd Quart., 2020.
Z. Liu, Z. Li, K. Wu, and M. Li, "Urban traffic prediction from mobility data using deep learning, " IEEE Netw., vol. 32, no. 4, pp. 40-46, Jul. /Aug. 2018.
F. Lin, Y. Xu, Y. Yang, and H. Ma, "A spatial-temporal hybrid model for short-term traffic prediction, " Math. Problems Eng., vol. 2019, Jan. 2019, Art. no. 4858546.
E. Walraven, M. T. Spaan, and B. Bakker, "Traffic flow optimization: A reinforcement learning approach, " Eng. Appl. Artif. Intell., vol. 52, pp. 203-212, Jun. 2016.
E. D'Andrea, P. Ducange, B. Lazzerini, and F. Marcelloni, "Real-time detection of traffic from twitter stream analysis, " IEEE Trans. Intell. Transp. Syst., vol. 16, no. 4, pp. 2269-2283, Aug. 2015.
M. Zhu, Y. Wang, Z. Pu, J. Hu, X. Wang, and R. Ke, "Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving, " Transp. Res. C, Emerg. Technol., vol. 117, Aug. 2020, Art. no. 102662.
Y. Xie, F. Li, Y. Wu, S. Yang, and Y. Wang, "D3-Guard: Acousticbased drowsy driving detection using smartphones, " in Proc. IEEE INFOCOM Conf. Comput. Commun., Jun. 2019, pp. 1225-1233.
J. M. Celaya-Padilla et al., "'Texting & driving' detection using deep convolutional neural networks, " Appl. Sci., vol. 9, no. 15, p. 2962, Aug. 2019.
Y. Dai, D. Xu, S. Maharjan, G. Qiao, and Y. Zhang, "Artificial intelligence empowered edge computing and caching for Internet of Vehicles, " IEEE Wireless Commun., vol. 26, no. 3, pp. 12-18, Jul. 2019.
H. Ji, O. Alfarraj, and A. Tolba, "Artificial intelligence-empowered edge of vehicles: Architecture, enabling technologies, and applications, " IEEE Access, vol. 8, pp. 61020-61034, 2020.
A. Haydari and Y. Yilmaz, "Deep reinforcement learning for intelligent transportation systems: A survey, " IEEE Trans. Intell. Transp. Syst., vol. 23, no. 1, pp. 11-32, Jan. 2022.
W. Yu et al., "A survey on the edge computing for the Internet of Things, " IEEE Access, vol. 6, pp. 6900-6919, 2017.
D. Xu et al., "A survey of opportunistic offloading, " IEEE Commun. Surveys Tuts., vol. 20, no. 3, pp. 2198-2236, 3rd Quart., 2018.
S. Raza, S. Wang, M. Ahmed, and M. R. Anwar, "A survey on vehicular edge computing: Architecture, applications, technical issues, and future directions, " Wireless Commun. Mobile Comput., vol. 2019, Feb. 2019, Art. no. 3159762.
L. Liu, C. Chen, Q. Pei, S. Maharjan, and Y. L. Zhang, "Vehicular edge computing and networking: A survey, " Mobile Netw. Appl., vol. 26, no. 3, pp. 1145-1168, 2021.
R. A. Dziyauddin, D. Niyato, N. C. Luong, M. A. M. Izhar, M. Hadhari, and S. Daud, "Computation offloading and content caching delivery in vehicular edge computing: A survey, " Dec. 2019, arXiv:1912. 07803.
A. B. De Souza et al., "Computation offloading for vehicular environments: A survey, " IEEE Access, vol. 8, pp. 198214-198243, 2020.
A. Shakarami, M. Ghobaei-Arani, M. Masdari, and M. Hosseinzadeh, "A survey on the computation offloading approaches in mobile edge/cloud computing environment: A stochastic-based perspective, " J. Grid Comput., vol. 18, no. 4, pp. 639-671, Dec. 2020.
X. Wang, Y. Han, V. C. M. Leung, D. Niyato, X. Yan, and X. Chen, "Convergence of edge computing and deep learning: A comprehensive survey, " IEEE Commun. Surveys Tuts., vol. 22, no. 2, pp. 869-904, 2nd Quart. 2020.
A. Shakarami, M. Ghobaei-Arani, and A. Shahidinejad, "A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective, " Comput. Netw., vol. 182, Dec. 2020, Art. no. 107496. [Online]. Available: https://www. sciencedirect. com/science/article/pii/S1389128620311634
K. Tan, D. Bremner, J. L. Kernec, and M. Imran, "Federated machine learning in vehicular networks: A summary of recent applications, " in Proc. Int. Conf. U. K. China Emerg. Technol. (UCET), Aug. 2020, pp. 1-4.
R. Calheiros, C. Vecchiola, D. Karunamoorthy, and R. Buyya, "The aneka platform and QoS-driven resource provisioning for elastic applications on hybrid clouds, " Future Gener. Comput. Syst., vol. 28, no. 6, pp. 861-870, Jun. 2012.
R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandíc, "Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility, " Future Gener. Comput. Syst., vol. 25, no. 6, pp. 599-616, Jun. 2009.
Z. Ning, F. Xia, N. Ullah, X. Kong, and X. Hu, "Vehicular social networks: Enabling smart mobility, " IEEE Commun. Mag., vol. 55, no. 5, pp. 16-55, May 2017.
P. Mell and T. Grance, "The NIST definition of cloud computing, " Nat. Inst. Stand. Technol., Gaithersburg, MD, USA, Rep. 800-145, Sep. 2011.
Z. Zhou, C. Gao, C. Xu, Y. Zhang, S. Mumtaz, and J. Rodriguez, "Social big-data-based content dissemination in Internet of Vehicles, " IEEE Trans. Ind. Informat., vol. 14, no. 2, pp. 768-777, Feb. 2018.
F. Spinelli and V. Mancuso, "Toward enabled industrial verticals in 5G: A survey on MEC-based approaches to provisioning and flexibility, " IEEE Commun. Surveys Tuts., vol. 23, no. 1, pp. 596-630, 1st Quart., 2021.
P. Mach and Z. Becvar, "Mobile edge computing: A survey on architecture and computation offloading, " IEEE Commun. Surveys Tuts., vol. 19, no. 3, pp. 1628-1656, 3rd Quart., 2017.
S. Raza, S. Wang, M. Ahmed, M. R. Anwar, M. A. Mirza, and W. U. Khan, "Task offloading and resource allocation for IoV using 5G NR-V2X communication, " IEEE Internet Things J., early access, Oct. 21, 2021, doi: 10. 1109/JIOT. 2021. 3121796.
S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, and A. Y. Zomaya, "Edge intelligence: The confluence of edge computing and artificial intelligence, " IEEE Internet Things J., vol. 7, no. 8, pp. 7457-7469, Aug. 2020.
S. Yu, X. Chen, L. Yang, D. Wu, M. Bennis, and J. Zhang, "Intelligent edge: Leveraging deep imitation learning for mobile edge computation offloading, " IEEE Wireless Commun., vol. 27, no. 1, pp. 92-99, Feb. 2020.
S. Yu, X. Chen, Z. Zhou, X. Gong, and D. Wu, "When deep reinforcement learning meets federated learning: Intelligent multitimescale resource management for multiaccess edge computing in 5G ultradense network, " IEEE Internet Things J., vol. 8, no. 4, pp. 2238-2251, Feb. 2021.
C. Chen, T. Qiu, J. Hu, Z. Ren, Y. Zhou, and A. K. Sangaiah, "A congestion avoidance game for information exchange on intersections in heterogeneous vehicular networks, " J. Netw. Comput. Appl., vol. 85, pp. 116-126, May 2017.
F. Lin, L. Yang, K. Xiong, and X. Gong, "Recent advances in cloudaware mobile fog computing, " Wireless Commun. Mobile Comput., vol. 2019, Jan. 2019, Art. no. 8204394.
M. Abuelela and S. Olariu, "Taking VANET to the clouds, " in Proc. MoMM, Nov. 2010, pp. 6-13.
E. Skondras, A. Michalas, and D. D. Vergados, "Mobility management on 5G vehicular cloud computing systems, " Veh. Commun., vol. 16, pp. 15-44, Apr. 2019.
S. S. Manvi and S. Tangade, "A survey on authentication schemes in VANETs for secured communication, " Veh. Commun., vol. 9, pp. 19-30, Jul. 2017.
S. Olariu, T. Hristov, and G. Yan, "The next paradigm shift: From vehicular networks to vehicular clouds" in Mobile Ad Hoc Networking, Mar. 2013, pp. 645-700, ch. 19. [Online]. Available: https://onlinelibrary. wiley. com/doi/abs/10. 1002/9781118511305. ch19
S. Arif, S. Olariu, J. Wang, G. Yan, W. Yang, and I. Khalil, "Datacenter at the airport: Reasoning about time-dependent parking lot occupancy, " IEEE Trans. Parallel Distrib. Syst., vol. 23, no. 11, pp. 2067-2080, Nov. 2012.
N. Vignesh, R. Shankar, S. Sathyamoorthy, and V. Rajam, "Value added services on stationary vehicular cloud, " in Proc. ICDCIT, Feb. 2014, pp. 92-97.
S. Li and R. Li, "Task allocation based on task deployment in autonomous vehicular cloud, " in Proc. IEEE 9th Int. Conf. Electron. Inf. Emerg. Commun. (ICEIEC), Jul. 2019, pp. 450-454.
X. Huang, R. Yu, J. Liu, and L. Shu, "Parked vehicle edge computing: Exploiting opportunistic resources for distributed mobile applications, " IEEE Access, vol. 6, pp. 66649-66663, 2018.
M. Eltoweissy, S. Olariu, and M. Younis, "Towards autonomous vehicular clouds, " in Proc. Int. Conf. Ad Hoc Netw., Victoria, BC, Canada, Aug. 2010, pp. 1-16.
G. Yan, D. Wen, S. Olariu, and M. C. Weigle, "Security challenges in vehicular cloud computing, " IEEE Trans. Intell. Transp. Syst., vol. 14, no. 1, pp. 284-294, Mar. 2013.
S. Olariu, "A survey of vehicular cloud research: Trends, applications and challenges, " IEEE Trans. Intell. Transp. Syst., vol. 21, no. 6, pp. 2648-2663, Jun. 2020.
R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 2018.
Y. He, N. Zhao, and H. Yin, "Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach, " IEEE Trans. Veh. Technol., vol. 67, no. 1, pp. 44-55, Jan. 2018.
R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, "Multiagent actor-critic for mixed cooperative-competitive environments, " in Proc. NIPS, Jun. 2017, pp. 6382-6393.
C. J. Watkins and P. Dayan, "Q-learning, " Mach. Learn., vol. 8, nos. 3-4, pp. 279-292, May. 1992.
I. Goodfellow, Y. Bengio, and A. C. Courville, "Deep learning, " Nature, vol. 521, pp. 436-444, May 2015.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " Commun. ACM, vol. 60, no. 6, pp. 84-90, Dec. 2012.
T. Young, D. Hazarika, S. Poria, and E. Cambria, "Recent trends in deep learning based natural language processing [review article], " IEEE Comput. Intell. Mag., vol. 13, no. 3, pp. 55-75, Aug. 2018.
V. Mnih et al., "Human-level control through deep reinforcement learning, " Nature, vol. 518, no. 7540, pp. 529-533, 2015. [Online]. Available: http://dblp. uni-trier. de/db/journals/nature/nature518. html# MnihKSRVBGRFOPB15
V. Mnih et al., "Playing atari with deep reinforcement learning, " Dec. 2013, arXiv:1312. 5602.
L. Lei, Y. Tan, K. Zheng, S. Liu, K. Zhang, and X. Shen, "Deep reinforcement learning for autonomous Internet of Things: Model, applications and challenges, " IEEE Commun. Surveys Tuts., vol. 22, no. 3, pp. 1722-1760, 3rd Quart., 2020.
H. V. Hasselt, A. Guez, and D. Silver, "Deep reinforcement learning with double Q-learning, " 2016, arXiv:1509. 06461.
V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, and J. Pineau, An Introduction to Deep Reinforcement Learning. Boston, MA, USA: Now Publ., Nov. 2018.
R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning, " Mach. Learn., vol. 8, nos. 3-4, pp. 229-256, May 1992.
Z. Wang et al., "Sample efficient actor-critic with experience replay, " Nov. 2016, arXiv:1611. 01224.
V. Mnih et al., "Asynchronous methods for deep reinforcement learning, " Feb. 2016, arXiv:1602. 01783.
T. Haarnoja et al., "Soft actor-critic algorithms and applications, " Dec. 2018, arXiv:1812. 05905.
T. Lillicrap et al., "Continuous control with deep reinforcement learning, " Sep. 2015, arXiv:1509. 02971.
M. Sheraz et al., "Artificial intelligence for wireless caching: Schemes, performance, and challenges, " IEEE Commun. Surveys Tuts., vol. 23, no. 1, pp. 631-661, 1st Quart., 2021.
L. Buşoniu, R. Babuka, and B. D. Schutter, "Multi-agent reinforcement learning: An overview, " Innovations in Multi-Agent Systems and Applications-1 (Studies in Computational Intelligence), vol. 310, D. Srinivasan, L. C. Jain, Eds. Berlin, Germany: Springer, 2010, pp. 183-221. [Online]. Available: https://doi. org/10. 1007/978-3-642-14435-6_7
M. Tan, "Multi-agent reinforcement learning: Independent vs. cooperative agents, " in Proc. 10th Int. Conf. Mach. Learn. (ICML), 1993, pp. 330-337.
J. A. Clouse, "Learning from an automated training, " in Adaptation and Learning in Multiagent Systems, Springer-Verlag, 1996.
C. Boutilier and B. Price, "Accelerating reinforcement learning through implicit imitation, " J. Artif. Intell. Res., vol. 19, no. 1, pp. 569-629, Jul. 2003.
J. R. Kok, M. T. J. Spaan, and N. A. Vlassis, "Non-communicative multi-robot coordination in dynamic environments, " Robot. Auton. Syst., vol. 50, nos. 2-3, pp. 99-114, Feb. 2005.
P. Sunehag et al., "Value-decomposition networks for cooperative multi-agent learning, " 2018, arXiv:1706. 05296.
T. Rashid, M. Samvelyan, C. S. D. Witt, G. Farquhar, J. N. Foerster, and S. Whiteson, "Monotonic value function factorisation for deep multiagent reinforcement learning, " J. Mach. Learn. Res., vol. 21, p. 178, Mar. 2020.
R. Powers and Y. Shoham, "New criteria and a new algorithm for learning in multi-agent systems, " in Advances in Neural Information Processing Systems, vol. 17, L. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA, USA: MIT Press, 2005.
A. Aliyu et al., "Towards video streaming in IoT environments: Vehicular communication perspective, " Comput. Commun., vol. 118, pp. 93-119, Mar. 2018.
M. Satyanarayanan, "The emergence of edge computing, " Computer, vol. 50, no. 1, pp. 30-39, Jan. 2017.
T. Refaat, B. Kantarci, and H. Mouftah, "Virtual machine migration and management for vehicular clouds, " Veh. Commun., vol. 4, pp. 47-56, Apr. 2016.
L. T. Tan, R. Q. Hu, and L. Hanzo, "Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks, " IEEE Trans. Veh. Technol., vol. 68, no. 4, pp. 3086-3099, Apr. 2019.
Z. Ning, P. Dong, X. Wang, J. J. P. C. Rodrigues, and F. Xia, "Deep reinforcement learning for vehicular edge computing: An intelligent offloading system, " ACM Trans. Intell. Syst. Technol., vol. 10, no. 6, pp. 1-24, Dec. 2019.
Z. Ning et al., "When deep reinforcement learning meets 5G-enabled vehicular networks: A distributed offloading framework for traffic big data, " IEEE Trans. Ind. Informat., vol. 16, no. 2, pp. 1352-1361, Feb. 2020.
P. Dai et al., "Multi-armed bandit learning for computation-intensive services in MEC-empowered vehicular networks, " IEEE Trans. Veh. Technol., vol. 69, no. 7, pp. 7821-7834, Jul. 2020.
M. Ibrar et al., "ARTNet: AI-based resource allocation and task offloading in a reconfigurable Internet of vehicular networks, " IEEE Trans. Netw. Sci. Eng., vol. 9, no. 1, pp. 67-77, Jan. /Feb. 2022.
K. Xiong, S. Leng, C. Huang, C. Yuen, and Y. L. Guan, "Intelligent task offloading for heterogeneous V2X communications, " IEEE Trans. Intell. Transp. Syst., vol. 22, no. 4, pp. 2226-2238, Apr. 2021.
M. Khayyat, I. A. Elgendy, A. Muthanna, A. Alshahrani, S. Alharbi, and A. Koucheryavy, "Advanced deep learning-based computational offloading for multilevel vehicular edge-cloud computing networks, " IEEE Access, vol. 8, pp. 137052-137062, 2020.
Q. Ye, W. Shi, K. Qu, H. He, W. Zhuang, and X. Shen, "Joint RAN slicing and computation offloading for autonomous vehicular networks: A learning-assisted hierarchical approach, " IEEE Open J. Veh. Technol., vol. 2, pp. 272-288, 2021.
Y. Ouyang, "Task offloading algorithm of vehicle edge computing environment based on dueling-DQN, " J. Phys. Conf. Series, vol. 1873, no. 1, Apr. 2021, Art. no. 012046, doi: 10. 1088/1742-6596/1873/1/012046.
Z. Ning et al., "Joint computing and caching in 5G-envisioned Internet of Vehicles: A deep reinforcement learning-based traffic control system, " IEEE Trans. Intell. Transp. Syst., vol. 22, no. 8, pp. 1-12, Aug. 2021.
H. Ke, J. Wang, L. Deng, Y. Ge, and H. Wang, "Deep reinforcement learning-based adaptive computation offloading for MEC in heterogeneous vehicular networks, " IEEE Trans. Veh. Technol., vol. 69, no. 7, pp. 7916-7929, Jul. 2020.
W. Zhan et al., "Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing, " IEEE Internet Things J., vol. 7, no. 6, pp. 5449-5465, Jun. 2020.
X. Zhu, Y. Luo, A. Liu, M. Z. A. Bhuiyan, and S. Zhang, "Multiagent deep reinforcement learning for vehicular computation offloading in IoT, " IEEE Internet Things J., vol. 8, no. 12, pp. 9763-9773, Jun. 2021.
M. Li, J. Gao, L. Zhao, and X. Shen, "Deep reinforcement learning for collaborative edge computing in vehicular networks, " IEEE Trans. Cogn. Commun. Netw., vol. 6, no. 4, pp. 1122-1135, Dec. 2020.
H. Peng and X. Shen, "Deep reinforcement learning based resource management for multi-access edge computing in vehicular networks, " IEEE Trans. Netw. Sci. Eng., vol. 7, no. 4, pp. 2416-2428, Oct.-Dec. 2020.
H. Peng and X. Shen, "Multi-agent reinforcement learning based resource management in MEC-and UAV-assisted vehicular networks, " IEEE J. Sel. Areas Commun., vol. 39, no. 1, pp. 131-141, Jan. 2021.
S. Yu, Q. Liu, and X. Li, "Full velocity difference and acceleration model for a car-following theory, " Commun. Nonlinear Sci. Numer. Simulat., vol. 18, pp. 1229-1234, May 2013.
S. Abdelhamid, H. S. Hassanein, and G. Takahara, "Vehicle as a resource (VaaR), " IEEE Netw., vol. 29, no. 1, pp. 12-17, Jan. /Feb. 2015.
D. Tang, X. Zhang, M. Li, and X. Tao, "Adaptive inference reinforcement learning for task offloading in vehicular edge computing systems, " in Proc. IEEE Int. Conf. Commun. Workshops (ICC Workshops), Jun. 2020, pp. 1-6.
J. Shi, J. Du, J. Wang, and J. Yuan, "Distributed V2V computation offloading based on dynamic pricing using deep reinforcement learning, " in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), 2020, pp. 1-6.
C. Chen, Y. Zhang, Z. Wang, S. Wan, and Q. Pei, "Distributed computation offloading method based on deep reinforcement learning in ICV, " Appl. Soft Comput., vol. 103, May 2021, Art. no. 107108.
L. T. Tan and R. Q. Hu, "Mobility-aware edge caching and computing in vehicle networks: A deep reinforcement learning, " IEEE Trans. Veh. Technol., vol. 67, no. 11, pp. 10190-10203, Nov. 2018.
K. Zhang, Y. Zhu, S. Leng, Y. He, S. Maharjan, and Y. Zhang, "Deep learning empowered task offloading for mobile edge computing in urban informatics, " IEEE Internet Things J., vol. 6, no. 5, pp. 7635-7647, Oct. 2019.
Y. Liu, H. Yu, S. Xie, and Y. Zhang, "Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks, " IEEE Trans. Veh. Technol., vol. 68, no. 11, pp. 11158-11168, Nov. 2019.
J. Zhao, M. Kong, Q. Li, and X. Sun, "Contract-based computing resource management via deep reinforcement learning in vehicular fog computing, " IEEE Access, vol. 8, pp. 3319-3329, 2020.
Q. Luo, C. Li, T. H. Luan, and W. Shi, "Collaborative data scheduling for vehicular edge computing via deep reinforcement learning, " IEEE Internet Things J., vol. 7, no. 10, pp. 9637-9650, Oct. 2020.
J. Zhang, H. Guo, and J. Liu, "Adaptive task offloading in vehicular edge computing networks: A reinforcement learning based scheme, " Mobile Netw. Appl., vol. 25, no. 5, pp. 1736-1745, Jun. 2020.
K. Wang, X. Wang, X. Liu, and A. Jolfaei, "Task offloading strategy based on reinforcement learning computing in edge computing architecture of Internet of Vehicles, " IEEE Access, vol. 8, pp. 173779-173789, 2020.
H. Maleki, M. Başaran, and L. Durak-Ata, "Reinforcement learningbased decision-making for vehicular edge computing, " in Proc. 29th Signal Process. Commun. Appl. Conf. (SIU), Jun. 2021, pp. 1-4.
Y. Ren, X. Yu, X. Chen, S. Guo, and X.-S. Qiu, "Vehicular network edge intelligent management: A deep deterministic policy gradient approach for service offloading decision, " in Proc. Int. Wireless Commun. Mobile Comput. (IWCMC), Jun. 2020, pp. 905-910.
X. He, H. Lu, M. Du, Y. Mao, and K. Wang, "QoE-based task offloading with deep reinforcement learning in edge-enabled Internet of Vehicles, " IEEE Trans. Intell. Transp. Syst., vol. 22, no. 4, pp. 2252-2261, Apr. 2021.
Y. Shibata, A. Sakuraba, G. Sato, and N. Uchida, "IoT based wide area road surface state sensing and communication system for future safety driving, " in Proc. Int. Conf. Adv. Inf. Netw. Appl., Mar. 2019, pp. 1123-1132.
D. Jia and D. Ngoduy, "Enhanced cooperative car-following traffic model with the combination of V2V and V2I communication, " Transp. Res. B, Methodol., vol. 90, pp. 172-191, Aug. 2016.
G. Naik, B. Choudhury, and J.-M. Park, "IEEE 802. 11 bd & 5G NR V2X: Evolution of radio access technologies for V2X communications, " IEEE Access, vol. 7, pp. 70169-70184, 2019.
Z. Jiang, S. Zhou, X. Guo, and Z. Niu, "Task replication for deadlineconstrained vehicular cloud computing: Optimal policy, performance analysis, and implications on road traffic, " IEEE Internet Things J., vol. 5, no. 1, pp. 93-107, Feb. 2018.
F. Sun, N. Cheng, S. Zhang, H. Zhou, L. Gui, and X. Shen, "Reinforcement learning based computation migration for vehicular cloud computing, " in Proc. IEEE Global Commun. Conf. (GLOBECOM), Dec. 2018, pp. 1-6.
Z. Wang, Z. Zhong, and M. Ni, "Application-aware offloading policy using SMDP in vehicular fog computing systems, " in Proc. IEEE Int. Conf. Commun. Workshops (ICC Workshops), May 2018, pp. 1-6.
C.-C. Lin, D.-J. Deng, and C.-C. Yao, "Resource allocation in vehicular cloud computing systems with heterogeneous vehicles and roadside units, " IEEE Internet Things J., vol. 5, no. 5, pp. 3692-3700, Oct. 2018.
Z. Ning et al., "Deep reinforcement learning for intelligent Internet of Vehicles: An energy-efficient computational offloading scheme, " IEEE Trans. Cogn. Commun. Netw., vol. 5, no. 4, pp. 1060-1072, Dec. 2019.
H. Liang et al., "Reinforcement learning enabled dynamic resource allocation in Internet of Vehicles, " IEEE Trans. Ind. Informat., vol. 17, no. 7, pp. 4957-4967, Jul. 2021.
Q. Qi et al., "Knowledge-driven service offloading decision for vehicular edge computing: A deep reinforcement learning approach, " IEEE Trans. Veh. Technol., vol. 68, no. 5, pp. 4192-4203, May 2019.
S. Lee and S. Lee, "Poster abstract: Deep reinforcement learningbased resource allocation in vehicular fog computing, " in Proc. IEEE INFOCOM Conf. Comput. Commun. Workshops (INFOCOM WKSHPS), May. 2019, pp. 1029-1030.
S. S. Lee and S. Lee, "Resource allocation for vehicular fog computing using reinforcement learning combined with heuristic information, " IEEE Internet Things J., vol. 7, no. 10, pp. 10450-10464, Oct. 2020.
W. Tong, A. Hussain, W. X. Bo, and S. Maharjan, "Artificial intelligence for vehicle-to-everything: A survey, " IEEE Access, vol. 7, pp. 10823-10843, 2019.
L. Ma, S. Yi, and Q. Li, "Efficient service handoff across edge servers via docker container migration, " in Proc. 2nd ACM/IEEE Symp. Edge Comput., Oct. 2017, pp. 1-13.
D. C. Marinescu, Cloud Computing: Theory and Practice. Cambridge, MA, USA: Morgan Kaufmann, May 2017.
J. Redmon and A. Farhadi, "YOLO9000: Better, faster, stronger, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 6517-6525.
D. C. Marinescu, A. Paya, J. P. Morrison, and S. Olariu, "An approach for scaling cloud resource management, " Clust. Comput., vol. 20, no. 1, pp. 909-924, 2017.
D. Kapil, E. Pilli, and R. Joshi, "Live virtual machine migration techniques: Survey and research challenges, " in Proc. 3rd IEEE Int. Adv. Comput. Conf. (IACC), Feb. 2013, pp. 963-969.
V. Hassija, V. Chamola, V. Saxena, D. Jain, P. Goyal, and B. Sikdar, "A survey on IoT security: Application areas, security threats, and solution architectures, " IEEE Access, vol. 7, pp. 82721-82743, 2019.
R. Trimananda, A. Younis, B. Wang, B. Xu, B. Demsky, and G. Xu, "Vigilia: Securing smart home edge computing, " in Proc. IEEE/ACM Symp. Edge Comput. (SEC), Seattle, WA, USA, Oct. 2018, pp. 74-89.