UAV Path Planning Employing MPC-Reinforcement Learning Method Considering Collision Avoidance

RAMEZANI, Mahya; HABIBI, Hamed; Sanchez-Lopez, Jose Luis; VOOS, Holger

doi:10.1109/ICUAS57906.2023.10156232

Download

Paper published in a book (Scientific congresses, symposiums and conference proceedings)

UAV Path Planning Employing MPC-Reinforcement Learning Method Considering Collision Avoidance

RAMEZANI, Mahya; HABIBI, Hamed; Sanchez-Lopez, Jose Luis et al.

2023 • In 2023 International Conference on Unmanned Aircraft Systems, ICUAS 2023

Peer reviewed

Permalink
https://hdl.handle.net/10993/59093

DOI
10.1109/ICUAS57906.2023.10156232

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

UAV_Path_Planning_Employing_MPC-Reinforcement_Learning_Method_Considering_Collision_Avoidance.pdf

Author postprint (1.73 MB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

improved DDPG; LSTM network modeling; model predictive control; path planning; reinforcement learning; Collisions avoidance; Complex environments; Deterministics; Improved DDPG; Long-short-term memory network modeling; Memory network; Model-predictive control; Network models; Reinforcement learning method; Reinforcement learnings; Control and Optimization; Aerospace Engineering

Abstract :

[en] In this paper, we tackle the problem of Unmanned Aerial (UAV) path planning in complex and uncertain environments by designing a Model Predictive Control (MPC), based on a Long-Short-Term Memory (LSTM) network integrated into the Deep Deterministic Policy Gradient algorithm. In the proposed solution, LSTM-MPC operates as a deterministic policy within the DDPG network, and it leverages a predicting pool to store predicted future states and actions for improved robustness and efficiency. The use of the predicting pool also enables the initialization of the critic network, leading to improved convergence speed and reduced failure rate compared to traditional reinforcement learning and deep reinforcement learning methods. The effectiveness of the proposed solution is evaluated by numerical simulations.

Disciplines :

Electrical & electronics engineering
Computer science

Author, co-author :

RAMEZANI, Mahya ; University of Luxembourg

HABIBI, Hamed ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Automation

Sanchez-Lopez, Jose Luis; University of Luxembourg, Centre for Security, Reliability, and Trust, Luxembourg, Luxembourg

VOOS, Holger ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Automation ; Unilu - University of Luxembourg [LU] > Faculty of Science, Technology and Medicine (FSTM), Department of Engineering

External co-authors :

Language :

English

Title :

UAV Path Planning Employing MPC-Reinforcement Learning Method Considering Collision Avoidance

Publication date :

26 June 2023

Event name :

2023 International Conference on Unmanned Aircraft Systems (ICUAS)

Event place :

Warsaw, Poland

Event date :

06-06-2023 - 09-06-2023

Main work title :

2023 International Conference on Unmanned Aircraft Systems, ICUAS 2023

Publisher :

Institute of Electrical and Electronics Engineers Inc.

ISBN/EAN :

9798350310375

Peer reviewed :

Peer reviewed

Focus Area :

Security, Reliability and Trust

Additional URL :

http://xplorestaging.ieee.org/ielx7/10155645/10155789/10156232.pdf?arnumber=10156232

Funding text :

ACKNOWLEDGMENTS This research was partially supported by the European Union’s Horizon 2020 project Secure and Safe Multi-Robot Systems (SESAME) under grant agreement no. 101017258. For the purpose of open access, the author has applied a Creative Commons Attribution 4.0 International (CC BY 4.0) license to any Author Accepted Manuscript version arising from this submission.This research was partially supported by the European Union s Horizon 2020 project Secure and Safe Multi-Robot Systems (SESAME) under grant agreement no. 101017258. For the purpose of open access, the author has applied a Creative Commons Attribution 4.0 International (CC BY 4.0) license to any Author Accepted Manuscript version arising from this submission.

Available on ORBilu :

since 22 December 2023

Statistics

Number of views

19 (1 by Unilu)

Number of downloads

26 (3 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

Bibliography

R. Raj, S. Kar, R. Nandan, and A. Jagarlapudi, "Precision agriculture and unmanned aerial Vehicles (UAVs), " Unmanned Aerial Vehicle: Applications in Agriculture and Environment, pp. 7-23, 2020.
S. A. H. Mohsan, M. A. Khan, F. Noor, I. Ullah, and M. H. Alsharif, "Towards the unmanned aerial vehicles (UAVs): A comprehensive review, " Drones, vol. 6, no. 6, p. 147, 2022.
S. Park and Y. Choi, "Applications of unmanned aerial vehicles in mining from exploration to reclamation: A review, " Minerals, vol. 10, no. 8, p. 663, 2020.
D. Wang, "Indoor mobile-robot path planning based on an improved A algorithm, " Journal of Tsinghua University Science and Technology, vol. 52, no. 8, pp. 1085-1089, 2012.
L. Li, T. Ye, M. Tan, and X.-j. Chen, "Present state and future development of mobile robot technology research, " Robot, vol. 24, no. 5, pp. 475-480, 2002.
G. Hoffmann, S. Waslander, and C. Tomlin, "Quadrotor helicopter trajectory tracking control, " in AIAA guidance, navigation and control conference and exhibit, 2008, p. 7410.
R. Bohlin and L. E. Kavraki, "Path planning using lazy PRM, " in Proceedings 2000 ICRA. Millennium conference. IEEE international conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065), 2000, vol. 1: IEEE, pp. 521-528.
Y. K. Hwang and N. Ahuja, "A potential field approach to path planning, " IEEE transactions on robotics and automation, vol. 8, no. 1, pp. 23-32, 1992.
J. Bruce and M. Veloso, "Real-time randomized path planning for robot navigation, " in IEEE/RSJ international conference on intelligent robots and systems, 2002, vol. 3: IEEE, pp. 2383-2388.
G.-T. Tu and J.-G. Juang, "UAV Path Planning and Obstacle Avoidance Based on Reinforcement Learning in 3D Environments, " in Actuators, 2023, vol. 12, no. 2: MDPI, p. 57.
B. G. Maciel-Pearson, L. Marchegiani, S. Akcay, A. Atapour-Abarghouei, J. Garforth, and T. P. Breckon, "Online deep reinforcement learning for autonomous UAV navigation and exploration of outdoor environments, " arXiv preprint arXiv: 1912. 05684, 2019.
S. Faryadi and J. Mohammadpour Velni, "A reinforcement learningbased approach for modeling and coverage of an unknown field using a team of autonomous ground vehicles, " International Journal of Intelligent Systems, vol. 36, no. 2, pp. 1069-1084, 2021.
S. Zhang, Y. Li, and Q. Dong, "Autonomous navigation of UAV in multi-obstacle environments based on a Deep Reinforcement Learning approach, " Applied Soft Computing, vol. 115, p. 108194, 2022.
B. Xin and C. He, "DRL-Based Improvement for Autonomous UAV Motion Path Planning in Unknown Environments, " in 2022 7th International Conference on Control and Robotics Engineering (ICCRE), 2022: IEEE, pp. 102-105.
Y. Li and A. H. Aghvami, "Intelligent UAV Navigation: A DRL-QiER Solution, " in ICC 2022-IEEE International Conference on Communications, 2022: IEEE, pp. 419-424.
S. Gros and M. Zanon, "Reinforcement learning based on mpc and the stochastic policy gradient method, " in 2021 American Control Conference (ACC), 2021: IEEE, pp. 1947-1952.
S. Gros and M. Zanon, "Data-driven economic NMPC using reinforcement learning, " IEEE Transactions on Automatic Control, vol. 65, no. 2, pp. 636-648, 2019.
M. L. Darby and M. Nikolaou, "MPC: Current practice and challenges, " Control Engineering Practice, vol. 20, no. 4, pp. 328-342, 2012.
A. B. Kordabad, W. Cai, and S. Gros, "MPC-based reinforcement learning for economic problems with application to battery storage, " in 2021 European Control Conference (ECC), 2021: IEEE, pp. 2573-2578.
Z. Zhang, D. Zhang, and R. C. Qiu, "Deep reinforcement learning for power system applications: An overview, " CSEE Journal of Power and Energy Systems, vol. 6, no. 1, pp. 213-225, 2019.
H. Moumouh, N. Langlois, and M. Haddad, "A Novel Tuning approach for MPC parameters based on Artificial Neural Network, " in 2019 IEEE 15th International Conference on Control and Automation (ICCA), 2019: IEEE, pp. 1638-1643.
Y. Jiao, Z. Wang, and Y. Zhang, "Prediction of air quality index based on LSTM, " in 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), 2019: IEEE, pp. 17-20.
B. Jiang, B. Li, W. Zhou, L.-Y. Lo, C.-K. Chen, and C.-Y. Wen, "Neural network based model predictive control for a quadrotor UAV, " Aerospace, vol. 9, no. 8, p. 460, 2022.
A. Sherstinsky, "Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network, " Physica D: Nonlinear Phenomena, vol. 404, p. 132306, 2020.
J. Yan, P. DiMeo, L. Sun, and X. Du, "LSTM-Based Model Predictive Control of Piezoelectric Motion Stages for High-speed Autofocus, " IEEE Transactions on Industrial Electronics, 2022.
W. Cai, A. B. Kordabad, H. N. Esfahani, A. M. Lekkas, and S. Gros, "MPC-based reinforcement learning for a simplified freight mission of autonomous surface vehicles, " in 2021 60th IEEE Conference on Decision and Control (CDC), 2021: IEEE, pp. 2990-2995.