Autonomous Navigation; Control Barrier Functions; Model Predictive Control; Pedestrian Behavior; Trust-aware Control
Abstract :
[en] A trust-aware safe control system for autonomous navigation in the presence of humans, specifically pedestrians, is presented. The system combines model predictive control (MPC) with control barrier functions (CBFs) and system-to-human trust (SHT) estimation to ensure safe and reliable navigation in human-populated environments. In the context of this article, we refer to SHT as the confidence score that a system has in an agent/pedestrian’s attentiveness. Pedestrian SHT values are computed based on features, extracted from camera sensor images, such as mutual eye contact, smartphone usage, and pose fluctuations and are integrated into the MPC controller’s CBF constraints, allowing the autonomous vehicle to make informed decisions considering pedestrian behavior. Simulations conducted in the CARLA driving simulator demonstrate the feasibility and effectiveness of the proposed system, showcasing more conservative behavior around inattentive pedestrians and vice versa. The results highlight the practicality of the system in real-world applications, providing a promising approach to enhance the safety and efficiency of autonomous navigation systems, especially self-driving vehicles.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > ARG - Automation & Robotics
Disciplines :
Engineering, computing & technology: Multidisciplinary, general & others
Author, co-author :
EJAZ, Saad ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Automation
Inoue, Masaki; Department of Applied Physics and Physico-Informatics, Keio University, Yokohama, Japan
External co-authors :
yes
Language :
English
Title :
Trust-Aware Safe Control for Autonomous Navigation: Estimation of System-to-Human Trust for Trust-Adaptive Control Barrier Functions
Publication date :
29 October 2024
Journal title :
IEEE Transactions on Control Systems Technology
ISSN :
1063-6536
Publisher :
Institute of Electrical and Electronics Engineers (IEEE)
Special issue title :
Special Issue on Intelligent Decision Making, Motion Planning and Control of Automated Vehicles in Interaction-driven Traffic Scenarios
This work was supported by Grant-in-Aid for Scientificc Research (B), No. 20H02173 JSPS. It was conducted as part of the JEMARO (Japan-Europe on Advanced Robotics) program, which is funded by the IUEP EU-Japan, MEXT, and the Erasmus+ program.
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
J. M. Beer, A. D. Fisk, and W. A. Rogers, "Toward a framework for levels of robot autonomy in human-robot interaction," J. Hum.-Robot Interact., vol. 3, no. 2, p. 74, Jun. 2014.
B. D. Adams, L. E. Bruyn, and S. Houde, "Trust in automated systems: Literature review," Ministry Nat. Defence, DRDC, Toronto. ON, Canada, Tech. Rep. CR 2003-096, 2003.
K. Schaefer, "The perception and measurement of human-robot trust," Ph.D. thesis, Dept. Model. Simul., College Sci. Univ. Central Florida, Orlando, FL, USA, 2013.
S. Nahavandi, "Trust in autonomous systems-iTrust lab: Future directions for analysis of trust with autonomous systems," IEEE Syst. Man, Cybern. Mag., vol. 5, no. 3, pp. 52-59, Jul. 2019.
X. Sun et al., "Exploring personalised autonomous vehicles to influence user trust," Cogn. Comput., vol. 12, pp. 1170-1186, Sep. 2020.
K. Akash, G. McMahon, T. Reid, and N. Jain, "Human trust-based feedback control: Dynamically varying automation transparency to optimize human-machine interactions," IEEE Control Syst. Mag., vol. 40, no. 6, pp. 98-116, Dec. 2020.
K. Akash, K. Polson, T. Reid, and N. Jain, "Improving human-machine collaboration through transparency-based feedback-Part I: Human trust and workload model," IFAC-PapersOnLine, vol. 51, no. 34, pp. 315-321, 2019.
A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, "Control barrier functions: Theory and applications," in Proc. IEEE 18th Eur. Control Conf. (ECC), Jul. 2019, pp. 3420-3431.
A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, "Control barrier function based quadratic programs for safety critical systems," IEEE Trans. Autom. Control, vol. 62, no. 8, pp. 3861-3876, Aug. 2017.
A. D. Ames, J. W. Grizzle, and P. Tabuada, "Control barrier function based quadratic programs with application to adaptive cruise control," in Proc. 53rd IEEE Conf. Decis. Control, Aug. 2014, pp. 6271-6278.
H. M. S. Ahmad, E. Sabouni, W. Xiao, C. G. Cassandras, and W. Li, "Trust-aware resilient control and coordination of connected and automated vehicles," 2023, arXiv:2305.16818.
M. F. Ozkan and Y. Ma, "Trust-aware control of automated vehicles in car-following interactions with human drivers," in Proc. IEEE 61st Conf. Decis. Control (CDC), Dec. 2022, pp. 5279-5284.
Z. Zahedi, M. Verma, S. Sreedharan, and S. Kambhampati, "Trust-aware planning: Modeling trust evolution in iterated human-robot interaction," in Proc. ACM/IEEE Int. Conf. Human-Robot Interact., Mar. 2023, pp. 281-289.
R. K. Cosner, Y. Chen, K. Leung, and M. Pavone, "Learning responsibility allocations for safe human-robot interaction with applications to autonomous driving," 2023, arXiv:2303.03504.
Y. Lyu, W. Luo, and J. M. Dolan, "Responsibility-associated multi-agent collision avoidance with social preferences," in Proc. IEEE 25th Int. Conf. Intell. Transp. Syst. (ITSC), Oct. 2022, pp. 3645-3651.
Y. Lyu, W. Luo, and J. M. Dolan, "Risk-aware safe control for decentralized multi-agent systems via dynamic responsibility allocation," 2023, arXiv:2305.13467.
H. Parwana, A. Mustafa, and D. Panagou, "Trust-based rate-tunable control barrier functions for non-cooperative multi-agent systems," in Proc. IEEE 61st Conf. Decis. Control (CDC), Dec. 2022, pp. 2222-2229.
A. Valtazanos and S. Ramamoorthy, "Intent inference and strategic escape in multi-robot games with physical limitations and uncertainty," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Sep. 2011, pp. 3679-3685.
B. Brito, A. Agarwal, and J. Alonso-Mora, "Learning interaction-aware guidance policies for motion planning in dense traffic scenarios," 2021, arXiv:2107.04538.
M. Fooladi Mahani, L. Jiang, and Y. Wang, "A Bayesian trust inference model for human-multi-robot teams," Int. J. Social Robot., vol. 13, no. 8, pp. 1951-1965, Dec. 2021.
C. Hu and J. Wang, "Trust-based and individualizable adaptive cruise control using control barrier function approach with prescribed performance," IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 6974-6984, Jul. 2022.
D. Frej, M. Jaskiewicz, M. Poliak, and Z. Zwierzewicz, "Smartphone use in traffic: A pilot study on pedestrian behavior," Appl. Sci., vol. 12, no. 24, p. 12676, Dec. 2022.
M.-I.-B. Lin and Y.-P. Huang, "The impact of walking while using a smartphone on pedestrians' awareness of roadside events," Accident Anal. Prevention, vol. 101, pp. 87-96, Apr. 2017.
F. Shinmura, Y. Kawanishi, D. Deguchi, I. Ide, H. Murase, and H. Fujiyoshi, "Pedestrian's inattention estimation based on recognition of texting while walking from in-vehicle camera images," IEICE Tech. Rep., vol. 115, no. 98, pp. 83-88, 2015.
F. Shinmura, Y. Kawanishi, D. Deguchi, I. Ide, H. Murase, and H. Fujiyoshi, "Recognition of texting-while-walking by joint features based on arm and head poses," in Computer Vision-ACCV 2016 Workshops, C.-S. Chen, J. Lu, and K.-K. Ma, Eds., Cham, Switzerland: Springer, 2017, pp. 452-462.
A. Rangesh, E. Ohn-Bar, K. Yuen, and M. M. Trivedi, "Pedestrians and their phones-detecting phone-based activities of pedestrians for autonomous vehicles," in Proc. IEEE 19th Int. Conf. Intell. Transp. Syst. (ITSC), Nov. 2016, pp. 1882-1887.
A. Rangesh and M. M. Trivedi, "When vehicles see pedestrians with phones: A multicue framework for recognizing phone-based activities of pedestrians," IEEE Trans. Intell. Vehicles, vol. 3, no. 2, pp. 218-227, Jun. 2018.
H. Saenz, H. Sun, L. Wu, X. Zhou, and H. Yu, "Detecting phonerelated pedestrian distracted behaviours via a two-branch convolutional neural network," IET Intell. Transp. Syst., vol. 15, no. 1, pp. 147-158, Jan. 2021.
E. Hatay, J. Ma, H. Sun, J. Fang, Z. Gao, and H. Yu, "Learning to detect phone-related pedestrian distracted behaviors with synthetic data," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2021, pp. 2975-2983.
Y. Uemura, Y. Kajiwara, and H. Shimakawa, "Estimating distracted pedestrian from deviated walking considering consumption of working memory," in Proc. Int. Conf. Comput. Sci. Comput. Intell. (CSCI), Dec. 2016, pp. 1164-1167.
Y. Kusakari, W. H. Chin, and N. Kubota, "A deep neural model for pedestrians detection with danger estimation," in Proc. Int. Symp. Community-Centric Syst. (CcS), Sep. 2020, pp. 1-6.
Y. Belkada, L. Bertoni, R. Caristan, T. Mordan, and A. Alahi, "Do pedestrians pay attention? Eye contact detection in the wild," 2021, arXiv:2112.04212.
R. Hata, D. Deguchi, T. Hirayama, Y. Kawanishi, and H. Murase, "Detection of distant eye-contact using spatio-temporal pedestrian skeletons," in Proc. IEEE 25th Int. Conf. Intell. Transp. Syst. (ITSC), Oct. 2022, pp. 2730-2737.
X. Zhang, X. Zhou, M. Lin, and J. Sun, "ShuffleNet: An extremely efficient convolutional neural network for mobile devices," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., May 2018, pp. 6848-6856.
S. Kreiss, L. Bertoni, and A. Alahi, "OpenPifPaf: Composite fields for semantic keypoint detection and spatio-temporal association," IEEE Trans. Intell. Transp. Syst., vol. 23, no. 8, pp. 13498-13511, Aug. 2022.
Y. Deng, P. Luo, C. C. Loy, and X. Tang, "Pedestrian attribute recognition at far distance," in Proc. 22nd ACM Int. Conf. Multimedia, Nov. 2014, pp. 789-792.
L. Zheng, H. Zhang, S. Sun, M. Chandraker, Y. Yang, and Q. Tian, "Person re-identification in the wild," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Apr. 2017, pp. 1367-1376.
L. Wang, J. Shi, G. Song, and I.-F. Shen, "Object detection combining recognition and segmentation," in Proc. Asian Conf. Comput. Vis., 2007, pp. 189-199.
M. Cordts et al., "The cityscapes dataset," in Proc. CVPR Workshop Future Datasets Vis., vol. 2, 2015, pp. 1-20.
M. Tan and Q. V. Le, "EfficientNetv2: Smaller models and faster training," in Proc. Int. Conf. Mach. Learn., 2021, pp. 10096-10106.
J. Zeng, B. Zhang, and K. Sreenath, "Safety-critical model predictive control with discrete-time control barrier function," in Proc. IEEE Amer. Control Conf. (ACC), Aug. 2021, pp. 3882-3889.
J. Zeng, Z. Li, and K. Sreenath, "Enhancing feasibility and safety of nonlinear model predictive control with discrete-time control barrier functions," in Proc. 60th IEEE Conf. Decis. Control (CDC), Aug. 2021, pp. 6137-6144.
S. Liu, J. Zeng, K. Sreenath, and C. A. Belta, "Iterative convex optimization for model predictive control with discrete-time high-order control barrier functions," in Proc. Am. Control Conf. (ACC), Aug. 2023, pp. 3368-3375.
A. Thirugnanam, J. Zeng, and K. Sreenath, "Safety-critical control and planning for obstacle avoidance between polytopes with control barrier functions," in Proc. Int. Conf. Robot. Autom. (ICRA), May 2022, pp. 286-292.
D. Kraft, A Software Package for Sequential Quadratic Programming. Wiss. Berichtswesen d. DFVLR, 1988. [Online]. Available: https://books.google.lu/books?id=4rKaGwAACAAJ
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, "CARLA: An open urban driving simulator," in Proc. 1st Annu. Conf. Robot Learn., vol. 78, 2017, pp. 1-16.
J. Fu. (2023). Autonomous Driving With Model Predictive Control. Accessed: Jun. 11, 2023. [Online]. Available: https://junshengfu. github.io/Model-Predictive-Control/
