AI; certification; digitalization; future; healthcare; risk; safety; standards; Orthopedics and Sports Medicine
Abstract :
[en] [en] UNLABELLED: Artificial intelligence (AI) has been influencing healthcare and medical research for several years and will likely become indispensable in the near future. AI is intended to support healthcare professionals to make the healthcare system more efficient and ultimately improve patient outcomes. Despite the numerous benefits of AI systems, significant concerns remain. Errors in AI systems can pose serious risks to human health, underscoring the critical need for safety, as well as adherence to ethical and moral standards, before these technologies can be integrated into clinical practice. To address these challenges, the development, certification, and deployment of medical AI systems must adhere to strict and transparent regulations. The European Commission has already established a regulatory framework for AI systems by enacting the European Union Artificial Intelligence Act. This review article, part of an AI learning series, discusses key considerations for medical AI systems such as reliability, accuracy, trustworthiness, lawfulness and legal compliance, ethical and moral alignment, sustainability, and regulatory oversight.
LEVEL OF EVIDENCE: Level V.
Disciplines :
Physical, chemical, mathematical & earth Sciences: Multidisciplinary, general & others
Author, co-author :
Winkler, Philipp W ; Department for Orthopaedics and Traumatology Kepler University Hospital GmbH, Johannes Kepler University Linz Linz Austria ; Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden ; Sahlgrenska Sports Medicine Center Göteborg Sweden
Zsidai, Bálint; Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden ; Sahlgrenska Sports Medicine Center Göteborg Sweden
Hamrin Senorski, Eric ; Sahlgrenska Sports Medicine Center Göteborg Sweden ; Department of Health and Rehabilitation, Institute of Neuroscience and Physiology, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden
Pruneski, James A; Department of Orthopaedic Surgery Tripler Army Medical Center Honolulu Hawaii USA
Hirschmann, Michael T; Department of Orthopaedic Surgery and Traumatology Kantonsspital Baselland Bruderholz Switzerland ; University of Basel Basel Switzerland
LEY, Christophe ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Mathematics (DMATH)
Tischer, Thomas; Department of Orthopaedic Surgery University Medicine Rostock Rostock Germany ; Department of Orthopaedic and Trauma Surgery Malteser Waldkrankenhaus Erlangen Erlangen Germany
Herbst, Elmar; Department of Trauma Hand and Reconstructive Surgery, University Hosptial Muenster Münster Germany
Pareek, Ayoosh; Sports Medicine and Shoulder Institute Hospital for Special Surgery New York New York USA
Musahl, Volker; Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center University of Pittsburgh Pittsburgh Pennsylvania USA
Oeding, Jacob F; Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden ; Mayo Clinic Alix School of Medicine Mayo Clinic Rochester Minnesota USA
Oettl, Felix C; Department of Orthopedic Surgery Balgrist University Hospital, University of Zürich Zurich Switzerland
Longo, Umile Giuseppe ; Fondazione Policlinico Universitario Campus Bio-Medico Roma Italy ; Department of Medicine and Surgery, Research Unit of Orthopaedic and Trauma Surgery Università Campus Bio-Medico di Roma Roma Italy
Samuelsson, Kristian ; Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden ; Sahlgrenska Sports Medicine Center Göteborg Sweden ; Department of Orthopaedics Sahlgrenska University Hospital Mölndal Sweden
Feldt, Robert; Department of Computer Science and Engineering Chalmers University of Technology Gothenburg Sweden
Abbas A. Verified AI: The Way Forward to More Reliable and Trustworthy AI Systems. Techopedia, 2023. Available from: https://www.techopedia.com/verified-ai-the-way-forward-to-more-reliable-and-trustworthy-ai-systems
Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf. 2019;28:231–237.
Chang PD, Wong TT, Rasiej MJ. Deep learning for detection of complete anterior cruciate ligament tear. J Digit Imaging. 2019;32:980–986.
Chong YY, Chan PK, Chan VWK, Cheung A, Luk MH, Cheung MH, et al. Application of machine learning in the prevention of periprosthetic joint infection following total knee arthroplasty: a systematic review. Arthroplasty. 2023;5:38.
Choudhury A. Toward an ecologically valid conceptual framework for the use of artificial intelligence in clinical settings: need for systems thinking, accountability, decision-making, trust, and patient safety considerations in safeguarding the technology and clinicians. JMIR Hum Factors. 2022;9:e35421.
Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378.
de Hond AAH, Leeuwenberg AM, Hooft L, Kant IMJ, Nijman SWJ, van Os HJA, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit Med. 2022;5:2.
European Parliament. EU AI Act: first regulation on artificial intelligence. 2023. [updated 2023 Dez 19; cited 2024 Jan 22]. Available from: https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence
Goh E, Gallo R, Hom J, Strong E, Weng Y, Kerman H, et al. Large language model influence on diagnostic reasoning: a randomized clinical trial. JAMA Netwk Open. 2024;7:e2440969.
Grace K, Stein-Perlman Z, Weinstein-Raun B, Salvatier J. Expert Survey on Progress in AI. AI Impacts. 2022. Available from: https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/
Grace K, Stewart H, Sandkühler JF, Thomas S, Weinstein-Raun B, Brauner J. Thousands of AI authors on the future of AI. arXiv preprint January 5, 2025.
Harvey HB, Gowda V. How the FDA regulates AI. Academic Radiol. 2020;27:58–61.
Helm JM, Swiergosz AM, Haeberle HS, Karnuta JM, Schaffer JL, Krebs VE, et al. Machine learning and artificial intelligence: definitions, applications, and future directions. Curr Rev Musculoskelet Med. 2020;13:69–76.
High-Level Expert Group on Artificial Intelligence. 2019. Ethics guidelines for trustworthy AI. [accesed, January 2024]. Available from: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
Li J, Li G, Li Z, Jin Z, Hu X, Zhang K, et al. CodeEditor: learning to edit source code with pre-trained models. ACM Trans Softw Eng Methodol. 2023;32:1–22.
Jones N. The world's week on AI safety: powerful computing efforts launched to boost research. Nature. 2023;623:229–230.
Jones OT, Matin RN, van der Schaar M, Prathivadi Bhayankaram K, Ranmuthu CKI, Islam MS, et al. Artificial intelligence and machine learning algorithms for early detection of skin cancer in community and primary care settings: a systematic review. Lancet Digit Health. 2022;4:e466–e476.
Joshi K, Suganthi K. Anterior cruciate ligament tear detection based on deep convolutional neural network. Diagnostics. 2022;12:2314.
Kaarre J, Feldt R, Keeling LE, Dadoo S, Zsidai B, Hughes JD, et al. Exploring the potential of ChatGPT as a supplementary tool for providing orthopaedic information. Knee Surg Sports Traumatol Arthrosc. 2023;31:5190–5198.
Karnuta JM, Churchill JL, Haeberle HS, Nwachukwu BU, Taylor SA, Ricchetti ET, et al. The value of artificial neural networks for predicting length of stay, discharge disposition, and inpatient costs after anatomic and reverse shoulder arthroplasty. J Shoulder Elbow Surg. 2020;29:2385–2394.
Kop M. EU Artificial Intelligence Act: The European Approach to AI. Transatlantic Antitrust and IPR Developments. 2021. [Accessed 17 Dec 2023]. Available from: https://futurium.ec.europa.eu/sites/default/files/2021-10/Kop_EU%20Artificial%20Intelligence%20Act%20-%20The%20European%20Approach%20to%20AI_21092021_0.pdf
Kuo RYL, Harrison C, Curran TA, Jones B, Freethy A, Cussons D, et al. Artificial intelligence in fracture detection: a systematic review and meta-analysis. Radiology. 2022;304:50–62.
Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable AI: a review of machine learning interpretability methods. Entropy. 2020;23:18.
Luu BC, Wright AL, Haeberle HS, Karnuta JM, Schickendantz MS, Makhni EC, et al. Machine learning outperforms logistic regression analysis to predict next-season NHL player injury: an analysis of 2322 players from 2007 to 2017. Orthop J Sports Med. 2020;8:2325967120953404. https://doi.org/10.1177/2325967120953404
Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digital Medicine. 2023;6:120.
Minamoto Y, Akagi R, Maki S, Shiko Y, Tozawa R, Kimura S, et al. Automated detection of anterior cruciate ligament tears using a deep convolutional neural network. BMC Musculoskelet Disord. 2022;23:577.
Mohammed TJ, Xinying C, Alnoor A, Khaw KW, Albahri AS, Teoh WL, et al. A systematic review of artificial intelligence in orthopaedic disease detection: a taxonomy for analysis and trustworthiness evaluation. Int J Comput Intell Syst. 2024;17:303.
Nwachukwu BU, Beck EC, Lee EK, Cancienne JM, Waterman BR, Paul K, et al. Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery. Am J Sports Med. 2020;48:415–423.
Oliveira e Carmo L, van den Merkhof A, Olczak J, Gordon M, Jutte PC, Jaarsma RL, et al. An increasing number of convolutional neural networks for fracture recognition and classification in orthopaedics: are these externally validated and ready for clinical application? Bone Joint Open. 2021;2:879–885.
Ortega PA, Maini V, DeepMind Safety Team. Building safe artificial intelligence: specification, robustness, and assurance. 2018. [Accessed 16 Nov 2023]. Available from: https://deepmindsafetyresearch.medium.com/building-safe-artificial-intelligence-52f5f75058f1
Ramkumar PN, Kunze KN, Haeberle HS, Karnuta JM, Luu BC, Nwachukwu BU, et al. Clinical and research medical applications of artificial intelligence. Arthroscopy. 2021;37:1694–1697.
Rudner TGJ, Toner H. Key concepts in AI safety: an overview. Washington, D.C.: Center for Security and Emerging Technology; 2021.
Rudner TGJ, Toner H. Key concepts in AI safety: interpretability in machine learning. Washington, D.C.: Center for Security and Emerging Technology; 2021.
Rudner TGJ, Toner H. Key concepts in AI safety: robustness and adversarial examples. Washington, D.C.: Center for Security and Emerging Technology; 2021.
Rudner TGJ, Toner H. Key concepts in AI Safety: specification in machine learning. Washington, D.C.: Center for Security and Emerging Technology; 2021.
Steyerberg EW. Validation of prediction models. In: Steyerberg EW (Edt.) Clinical prediction models: a practical approach to development, validation, and updating. 2019;1:329–344.
The White House. Executive order on the safe, secure, and trustworthy development and use of artificial intelligence. 2023. [cited 2024 Feb 14]. Available from: https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/
Tran A, Lassalle L, Zille P, Guillin R, Pluot E, Adam C, et al. Deep learning to detect anterior cruciate ligament tear on knee MRI: multi-continental external validation. Eur Radiol. 2022;32:8394–8403.
Turing A. Computing machinery and intelligence. Mind 59. 1950:433–460.
Ustun B, Rudin C. Supersparse linear integer models for optimized medical scoring systems. Mach Learn. 2016;102:349–391.
VanCalster, B, Collins GS, Vickers AJ, Wynants L, Kerr KF, Barreñada L, et al. Performance evaluation of predictive AI models to support medical decisions: overview and guidance. arXiv preprint, 2024.
Yu AC, Mohajer B, Eng J. External validation of deep learning algorithms for radiologic diagnosis: a systematic review. Radiol: Artif Intell. 2022;4:e210064.
Yu KH, Kohane IS. Framing the challenges of artificial intelligence in medicine. BMJ Qual Saf. 2019;28:238–241.
Zhang L, Lengersdorff L, Mikus N, Gläscher J, Lamm C. Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices. Soc Cogn Affect Neurosci. 2020;15:695–707.
Zsidai B, Hilkert AS, Kaarre J, Narup E, Senorski EH, Grassi A, et al. A practical guide to the implementation of AI in orthopaedic research - part 1: opportunities in clinical application and overcoming existing challenges. J Exp Orthop. 2023;10:117.
Zsidai B, Kaarre J, Hilkert AS, Narup E, Senorski EH, Grassi A, et al. Accelerated evidence synthesis in orthopaedics-the roles of natural language processing, expert annotation and large language models. J Exp Orthop. 2023;10:99.
