[en] Environmental perception is a key element of autonomous driving because the information received from the perception module influences core driving decisions. An outstanding challenge in real-time perception for autonomous driving lies in finding the best trade-off between detection quality and latency. Major constraints on both computation and power must be taken into account for real-time perception in autonomous vehicles. Larger detection models tend to produce the best results but are also slower at runtime. Since the most accurate detectors may not run in real-time locally, we investigate the possibility of offloading computation to edge and cloud platforms, which are less resource-constrained. We create a synthetic dataset to train object detection models and evaluate different offloading strategies. We measure inference and processing times for object detection on real hardware, and we rely on a network simulation framework to estimate data transmission latency. Our study compares different trade-offs between prediction quality and end-to-end delay. Following the existing literature, we aim to perform object detection at a rate of 20Hz. Since sending raw frames over the network implies additional transmission delays, we also explore the use of JPEG and H.265 compression at varying qualities and measure their impact on prediction. We show that models with adequate compression can be run in real-time on the edge/cloud while outperforming local detection performance.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Other
Precision for document type :
Review article
Disciplines :
Computer science
Author, co-author :
HAWLADER, Faisal ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Ubiquitous and Intelligent Systems (UBI-X)
ROBINET, François ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust > Ubiquitous and Intelligent Systems > Team Raphaël FRANK
FRANK, Raphaël ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Ubiquitous and Intelligent Systems (UBI-X)
External co-authors :
no
Language :
English
Title :
Leveraging the edge and cloud for V2X-based real-time object detection in autonomous driving
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
Hawlader, F., Robinet, F., Frank, R., Vehicle-to-infrastructure communication for real-time object detection in autonomous driving. 2023 18th Wireless on-Demand Network Systems and Services Conference (WONS), 2023, IEEE, 40–46.
De Souza, A.B., Rego, P.A., Carneiro, T., Rodrigues, J.D.C., Reboucas Filho, P.P., De Souza, J.N., Chamola, V., De Albuquerque, V.H.C., Sikdar, B., Computation offloading for vehicular environments: A survey. IEEE Access 8 (2020), 198214–198243.
Nardini, G., Stea, G., Virdis, A., Sabella, D., Simu5G: a system-level simulator for 5G networks. SIMULTECH 2020, 2020, INSTICC, 23.
Garcia, M.H.C., Molina-Galan, A., Boban, M., Gozalvez, J., Coll-Perales, B., Şahin, T., Kousaridas, A., A tutorial on 5G NR V2X communications. IEEE Commun. Surv. Tutor. 23:3 (2021), 1972–2026, 10.1109/COMST.2021.3057017.
Hawlader, F., Robinet, F., Frank, R., Poster: Lightweight features sharing for real-time object detection in cooperative driving. 2023 IEEE Vehicular Networking Conference (VNC), 2023, 159–160, 10.1109/VNC57357.2023.10136339.
Pan, Z., Lei, J., Zhang, Y., Sun, X., Kwong, S., Fast motion estimation based on content property for low-complexity h. 265/HEVC encoder. IEEE Trans. Broadcast. 62:3 (2016), 675–684.
Matsubara, Y., Levorato, M., Neural compression and filtering for edge-assisted real-time object detection in challenged networks. 2020 25th International Conference on Pattern Recognition (ICPR), 2021, IEEE.
Deguerre, B., Chatelain, C., Gasso, G., Fast object detection in compressed jpeg images. 2019 IEEE Intelligent Transportation Systems Conference (ITSC), 2019, IEEE, 333–338.
Abdel Hakeem, S.A., Hady, A.A., Kim, H., 5G-V2X: Standardization, architecture, use cases, network-slicing, and edge-computing. Wirel. Netw. 26:8 (2020), 6015–6041.
Jocher Glenn, J.B., Ayush Chaurasia, A.A., Stoken, A., YOLOv5 (2020). 2022 https://github.com/ultralytics/yolov5 (Accessed 10 July 2022).
Dosovitskiy, A., Ros, G., Koltun, V., CARLA: An open urban driving simulator. Conference on Robot Learning, 2017, PMLR.
H. Xu, S. Liu, G. Wang, G. Liu, B. Zeng, Omnet: Learning overlapping mask for partial-to-partial point cloud registration, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
Huang, C.-M., Lai, C.-F., The mobile edge computing (MEC)-based vehicle to infrastructure (V2I) data offloading from cellular network to VANET using the delay-constrained computing scheme. 2020 International Computer Symposium (ICS), 2020, IEEE.
Islam, A., Debnath, A., Ghose, M., Chakraborty, S., A survey on task offloading in multi-access edge computing. J. Syst. Archit., 118, 2021.
Computing, M.-a.E., Framework and reference architecture, ETSI GS MEC 003, rev. 2.2. 1, dec. 2020. 2020.
Khan, M.A., Baccour, E., Chkirbene, Z., Erbad, A., Hamila, R., Hamdi, M., Gabbouj, M., A survey on mobile edge computing for video streaming: Opportunities and challenges. IEEE Access, 2022.
Siriwardhana, Y., Porambage, P., Liyanage, M., Ylianttila, M., A survey on mobile augmented reality with 5G mobile edge computing: architectures, applications, and technical aspects. IEEE Commun. Surv. Tutor. 23:2 (2021), 1160–1192.
Tsao, S.-L.C., Enhanced GTP: an efficient packet tunneling protocol for general packet radio service. ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No. 01CH37240), Vol. 9, 2001, IEEE, 2819–2823.
P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001, http://dx.doi.org/10.1109/CVPR.2001.990517.
Dalal, N., Triggs, B., Histograms of oriented gradients for human detection. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1, 2005, 886–893, 10.1109/CVPR.2005.177.
Girshick, R., Donahue, J., Darrell, T., Malik, J., Rich feature hierarchies for accurate object detection and semantic segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014, 10.1109/CVPR.2014.81.
Girshick, R., Fast R-CNN. 2015 IEEE International Conference on Computer Vision (ICCV), 2015, 10.1109/ICCV.2015.169.
Ren, S., He, K., Girshick, R., Sun, J., Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell., 2017, 10.1109/TPAMI.2016.2577031.
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C., SSD: Single shot MultiBox detector. Leibe, B., Matas, J., Sebe, N., Welling, M., (eds.) Computer Vision – ECCV 2016, 2016, Springer International Publishing, Cham, 21–37.
Redmon, J., Divvala, S., Girshick, R., Farhadi, A., You only look once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Ehrlich, M., Davis, L., Lim, S.-N., Shrivastava, A., Analyzing and mitigating JPEG compression defects in deep learning. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021.
Dodge, S., Karam, L., Understanding how image quality affects deep neural networks. 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), 2016, 10.1109/QoMEX.2016.7498955.
Yu, R., Yang, D., Zhang, H., Edge-assisted collaborative perception in autonomous driving: A reflection on communication design. 2021 IEEE/ACM Symposium on Edge Computing (SEC), 2021, IEEE.
Kovács, G.A., Bokor, L., Integrating artery and Simu5G: A mobile edge computing use case for collective perception-based V2X safety applications. 2022 45th International Conference on Telecommunications and Signal Processing (TSP), 2022, IEEE, 360–366.
Li, J., Gao, H., Lv, T., Lu, Y., Deep reinforcement learning based computation offloading and resource allocation for MEC. 2018 IEEE Wireless Communications and Networking Conference (WCNC), 2018, IEEE.
Belogaev, A., Elokhin, A., Krasilov, A., Khorov, E., Akyildiz, I.F., Cost-effective V2X task offloading in MEC-assisted intelligent transportation systems. IEEE Access, 8, 2020.
Marvasti, E.E., Raftari, A., Marvasti, A.E., Fallah, Y.P., Guo, R., Lu, H., Feature sharing and integration for cooperative cognition and perception with volumetric sensors. 2020 Preprint arXiv:2011.08317.
E. Ye, P. Spiegel, M. Althoff, Cooperative raw sensor data fusion for ground truth generation in autonomous driving, in: IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), IEEE.
Hawlader, F., Frank, R., Towards a framework to evaluate cooperative perception for connected vehicles. 2021 IEEE Vehicular Networking Conference (VNC), 2021, 36–39, 10.1109/VNC52810.2021.9644667.
Ren, J., Guo, Y., Zhang, D., Liu, Q., Zhang, Y., Distributed and efficient object detection in edge computing: Challenges and solutions. IEEE Netw., 32(6), 2018.
Ahmed, M., Raza, S., Mirza, M.A., Aziz, A., Khan, M.A., Khan, W.U., Li, J., Han, Z., A survey on vehicular task offloading: Classification, issues, and challenges. J. King Saud Univ.-Comput. Inf. Sci., 2022.
Ndikumana, A., Nguyen, K.K., Cheriet, M., Age of processing-based data offloading for autonomous vehicles in MultiRATs open RAN. IEEE Trans. Intell. Transp. Syst., 2022.
Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., Kwon, Y., Fang, J., Michael, K., Montes, D., Nadar, J., Skalski, P., et al. Ultralytics/yolov5: v6. 1-tensorrt, tensorflow edge tpu and openvino export and inference. Zenodo, 2022.
Zhao, W., Liu, S., Li, X., Han, X., Yang, H., Fast and accurate wheat grain quality detection based on improved YOLOv5. Comput. Electron. Agric., 202, 2022, 107426.
Varga, A., Hornig, R., An overview of the OMNeT++ simulation environment. 1st International ICST Conference on Simulation Tools and Techniques for Communications, Networks and Systems, 2010.
Farasat, M., Thalakotuna, D.N., Hu, Z., Yang, Y., A review on 5G sub-6 GHz base station antenna design challenges. Electronics, 2021.
Nardini, G., Sabella, D., Stea, G., Thakkar, P., Virdis, A., Simu5G–An OMNeT++ library for end-to-end performance evaluation of 5G networks. IEEE Access, 2020, 10.1109/ACCESS.2020.3028550.
Filali, A., Abouaomar, A., Cherkaoui, S., Kobbane, A., Guizani, M., Multi-access edge computing: A survey. IEEE Access, 8, 2020.
Winick, J., Jamin, S., Inet-3.0: Internet topology generator: Tech. rep., Technical Report CSE-TR-456-02., 2002, University of Michigan.
Ali, Z., Lagén, S., Giupponi, L., Rouil, R., 3GPP NR V2X mode 2: overview, models and system-level evaluation. IEEE Access, 2021.
Frank, R., Hawlader, F., Poster: Commercial 5G performance: A V2X experiment. 2021 IEEE Vehicular Networking Conference (VNC), 2021, IEEE, 129–130.
Rappaport, T.S., Xing, Y., MacCartney, G.R., Molisch, A.F., Mellios, E., Zhang, J., Overview of millimeter wave communications for fifth-generation (5G) wireless networks—With a focus on propagation models. IEEE Trans. Antennas Propag., 65(12), 2017.
Liu, J., Zhang, Q., To improve service reliability for AI-powered time-critical services using imperfect transmission in MEC: An experimental study. IEEE Internet Things J. 7:10 (2020), 9357–9371.
Uhrina, M., Bienik, J., Vaculik, M., Coding efficiency of hevc/h. 265 and vp9 compression standards for high resolutions. 2016 26th International Conference Radioelektronika (RADIOELEKTRONIKA), 2016, IEEE, 419–423.
Gutierrez-Aguado, J., Peña-Ortiz, R., García-Pineda, M., Claver, J.M., Cloud-based elastic architecture for distributed video encoding: Evaluating H. 265, VP9, and AV1. J. Netw. Comput. Appl., 171, 2020, 102782.
Similar publications
Sorry the service is unavailable at the moment. Please try again later.