Medical image segmentation; COVID-19; Challenge; deep learning; u-net
Abstract :
[en] Artificial intelligence (AI) methods for the automatic detection and quantification of COVID-19 lesions in chest computed tomography (CT) might play an important role in the monitoring and management of the disease. We organized an international challenge and competition for the development and comparison of AI algorithms for this task, which we supported with public data and state-of-the-art benchmark methods. Board Certified Radiologists annotated 295 public images from two sources (A and B) for algorithms training (n=199, source A), validation (n=50, source A) and testing (n=23, source A; n=23, source B). There were 1,096 registered teams of which 225 and 98 completed the validation and testing phases, respectively. The challenge showed that AI models could be rapidly designed by diverse teams with the potential to measure disease or facilitate timely and patient-specific interventions. This paper provides an overview and the major outcomes of the COVID-19 Lung CT Lesion Segmentation Challenge - 2020.
Disciplines :
Engineering, computing & technology: Multidisciplinary, general & others
Author, co-author :
Roth, Holger R.
Xu, Ziyue
Diez, Carlos Tor
Jacob, Ramon Sanchez
Zember, Jonathan
Molto, Jose
Li, Wenqi
Xu, Sheng
Turkbey, Baris
Turkbey, Evrim
Yang, Dong
Harouni, Ahmed
Rieke, Nicola
Hu, Shishuai
Isensee, Fabian
Tang, Claire
Yu, Qinji
Sölter, Jan
Zheng, Tong
Liauchuk, Vitali
Zhou, Ziqi
Moltz, Jan Hendrik
Oliveira, Bruno
Xia, Yong
Maier-Hein, Klaus H.
Li, Qikai
HUSCH, Andreas ; University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > Interventional Neuroscience
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
Andrearczyk, Vincent, Oreiller, Valentin, Depeursinge, Adrien, Head and neck tumor segmentation: First challenge, HECKTOR 2020, held in conjunction with MICCAI 2020, lima, peru, october 4, 2020, proceedings. 2021, Springer Nature.
Bai, Harrison X, Wang, Robin, Xiong, Zeng, Hsieh, Ben, Chang, Ken, Halsey, Kasey, Tran, Thi My Linh, Choi, Ji Whae, Wang, Dong-Cui, Shi, Lin-Bo, et al. Artificial intelligence augmentation of radiologist performance in distinguishing COVID-19 from pneumonia of other origin at chest CT. Radiology 296:3 (2020), E156–E165.
Bao, Cuiping, Liu, Xuehuan, Zhang, Han, Li, Yiming, Liu, Jun, Coronavirus disease 2019 (COVID-19) CT findings: a systematic review and meta-analysis. J. Am. College Radiol. 17:6 (2020), 701–709.
Bernheim, Adam, Mei, Xueyan, Huang, Mingqian, Yang, Yang, Fayad, Zahi A, Zhang, Ning, Diao, Kaiyue, Lin, Bin, Zhu, Xiqi, Li, Kunwei, et al. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection. Radiology, 2020, 200463.
Cao, Yinghao, Liu, Xiaoling, Xiong, Lijuan, Cai, Kailin, Imaging and clinical features of patients with 2019 novel coronavirus SARS-CoV-2: a systematic review and meta-analysis. J. Med. Virol. 92:9 (2020), 1449–1459.
Çiçek, Özgün, Abdulkadir, Ahmed, Lienkamp, Soeren S, Brox, Thomas, Ronneberger, Olaf, 3D U-net: learning dense volumetric segmentation from sparse annotation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2016, Springer, 424–432.
Clark, Kenneth, Vendt, Bruce, Smith, Kirk, Freymann, John, Kirby, Justin, Koppel, Paul, Moore, Stephen, Phillips, Stanley, Maffitt, David, Pringle, Michael, et al. The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digital Imag. 26:6 (2013), 1045–1057.
Dayan, Ittai, Roth, Holger R, Zhong, Aoxiao, Harouni, Ahmed, Gentili, Amilcare, Abidin, Anas Z, Liu, Andrew, Costa, Anthony Beardsworth, Wood, Bradford J, Tsai, Chien-Sung, et al. Federated learning for predicting clinical outcomes in patients with COVID-19. Nat. Med., 2021, 1–9.
Desai, Shivang, Baghal, Ahmad, Wongsurawat, Thidathip, Jenjaroenpun, Piroon, Powell, Thomas, Al-Shukri, Shaymaa, Gates, Kim, Farmer, Phillip, Rutherford, Michael, Blake, Geri, et al. Chest imaging representing a COVID-19 positive rural US population. Sci. Data 7:1 (2020), 1–6.
Drozdzal, Michal, Vorontsov, Eugene, Chartrand, Gabriel, Kadoury, Samuel, Pal, Chris, The importance of skip connections in biomedical image segmentation. Deep Learning and Data Labeling for Medical Applications, 2016, Springer, 179–187.
Eugster, Manuel J.A., Hothorn, Torsten, Leisch, Friedrich, Exploratory and inferential analysis of benchmark experiments. Tech. Rep., No.30, 2008.
Isensee, Fabian, Jaeger, Paul F, Kohl, Simon AA, Petersen, Jens, Maier-Hein, Klaus H, NnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18:2 (2021), 203–211.
Li, Lin, Qin, Lixin, Xu, Zeguo, Yin, Youbing, Wang, Xin, Kong, Bin, Bai, Junjie, Lu, Yi, Fang, Zhenghan, Song, Qi, et al. Using artificial intelligence to detect COVID-19 and community-acquired pneumonia based on pulmonary CT: evaluation of the diagnostic accuracy. Radiology 296:2 (2020), E65–E71.
Lin, Tsung-Yi, Goyal, Priya, Girshick, Ross, He, Kaiming, Dollár, Piotr, Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, 2017, 2980–2988.
Long, Jonathan, Shelhamer, Evan, Darrell, Trevor, 2015. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3431–3440.
Lyu, Siwei, Fan, Yanbo, Ying, Yiming, Hu, Bao-Gang, Average top-k aggregate loss for supervised learning. IEEE Trans. Pattern Anal. Mach. Intell., 2020.
Maier-Hein, Lena, Eisenmann, Matthias, Reinke, Annika, Onogur, Sinan, Stankovic, Marko, Scholz, Patrick, Arbel, Tal, Bogunovic, Hrvoje, Bradley, Andrew P, Carass, Aaron, et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Commun. 9:1 (2018), 1–13.
Mei, Xueyan, Lee, Hao-Chih, Diao, Kai-yue, Huang, Mingqian, Lin, Bin, Liu, Chenyu, Xie, Zongyu, Ma, Yixuan, Robson, Philip M, Chung, Michael, et al. Artificial intelligence–enabled rapid diagnosis of patients with COVID-19. Nat. Med. 26:8 (2020), 1224–1228.
Merkus, Peter J.F.M., Klein, Willemijn M., The value of chest CT as a COVID-19 screening tool in children. Eur. Respir. J., 55(6), 2020.
Milletari, Fausto, Navab, Nassir, Ahmadi, Seyed-Ahmad, V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV), 2016, IEEE, 565–571.
Nino, Gustavo, Zember, Jonathan, Sanchez-Jacob, Ramon, Gutierrez, Maria J, Sharma, Karun, Linguraru, Marius George, Pediatric lung imaging features of COVID-19: a systematic review and meta-analysis. Pediatr. Pulmonol. 56:1 (2021), 252–263.
Ojha, Vineeta, Mani, Avinash, Pandey, Niraj Nirmal, Sharma, Sanjiv, Kumar, Sanjeev, CT in coronavirus disease 2019 (COVID-19): a systematic review of chest CT findings in 4410 adult patients. Euro. Radiol. 30 (2020), 6129–6138.
Oulefki, Adel, Agaian, Sos, Trongtirakul, Thaweesak, Laouar, Azzeddine Kassah, Automatic COVID-19 lung infected region segmentation and measurement using CT-scans images. Pattern Recognit., 114, 2021, 107747.
Patrick, Sanjana, Birur, N Praveen, Gurushanth, Keerthi, Raghavan, A Shubhasini, Gurudath, Shubha, et al. Comparison of gray values of cone-beam computed tomography with hounsfield units of multislice computed tomography: an in vitro study. Indian J. Dental Res., 28(1), 2017, 66.
Project MONAI, Project MONAI. 2021 https://monai.io/. (Accessed September 22, 2022).
Roberts, Michael, Driggs, Derek, Thorpe, Matthew, Gilbey, Julian, Yeung, Michael, Ursprung, Stephan, Aviles-Rivero, Angelica I, Etmann, Christian, McCague, Cathal, Beer, Lucian, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat. Mach. Intell. 3:3 (2021), 199–217.
Ronneberger, Olaf, Fischer, Philipp, Brox, Thomas, U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, Springer, 234–241.
Rubin, Geoffrey D, Ryerson, Christopher J, Haramati, Linda B, Sverzellati, Nicola, Kanne, Jeffrey P, Raoof, Suhail, Schluger, Neil W, Volpi, Annalisa, Yim, Jae-Joon, Martin, Ian BK, et al. The role of chest imaging in patient management during the COVID-19 pandemic: a multinational consensus statement from the Fleischner Society. Radiology 296:1 (2020), 172–180.
Shan, Fei, Gao, Yaozong, Wang, Jun, Shi, Weiya, Shi, Nannan, Han, Miaofei, Xue, Zhong, Shen, Dinggang, Shi, Yuxin, Abnormal lung quantification in chest CT images of COVID-19 patients with deep learning and its application to severity prediction. Med. Phys. 48:4 (2021), 1633–1645.
Sun, Dong, Li, Xiang, Guo, Dajing, Wu, Lan, Chen, Ting, Fang, Zheng, Chen, Linli, Zeng, Wenbing, Yang, Ran, CT quantitative analysis and its relationship with clinical features for assessing the severity of patients with COVID-19. Korean J. Radiol., 21(7), 2020, 859.
Tajbakhsh, Nima, Jeyaseelan, Laura, Li, Qian, Chiang, Jeffrey N, Wu, Zhihao, Ding, Xiaowei, Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Med. Image Anal., 63, 2020, 101693.
Wan, Shang, Li, Mingqi, Ye, Zheng, Yang, Caiwei, Cai, Qian, Duan, Shaofeng, Song, Bin, CT manifestations and clinical characteristics of 1115 patients with coronavirus disease 2019 (COVID-19): a systematic review and meta-analysis. Acad. Radiol. 27:7 (2020), 910–921.
Wang, Guotai, Liu, Xinglong, Li, Chaoping, Xu, Zhiyong, Ruan, Jiugen, Zhu, Haifeng, Meng, Tao, Li, Kang, Huang, Ning, Zhang, Shaoting, A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images. IEEE Trans. Med. Imaging 39:8 (2020), 2653–2663.
Wiesenfarth, Manuel, Reinke, Annika, Landman, Bennett A, Eisenmann, Matthias, Saiz, Laura Aguilera, Cardoso, M Jorge, Maier-Hein, Lena, Kopp-Schneider, Annette, Methods and open-source toolkit for analyzing and visualizing challenge results. Sci. Rep. 11:1 (2021), 1–15.
Wynants, Laure, Van Calster, Ben, Collins, Gary S, Riley, Richard D, Heinze, Georg, Schuit, Ewoud, Bonten, Marc MJ, Dahly, Darren L, Damen, Johanna A, Debray, Thomas PA, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. Bmj, 369, 2020.
Yang, Dong, Roth, Holger, Xu, Ziyue, Milletari, Fausto, Zhang, Ling, Xu, Daguang, Searching learning strategy with reinforcement learning for 3d medical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019, Springer, 3–11.
Yang, Dong, Xu, Ziyue, Li, Wenqi, Myronenko, Andriy, Roth, Holger R, Harmon, Stephanie, Xu, Sheng, Turkbey, Baris, Turkbey, Evrim, Wang, Xiaosong, et al. Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan. Med. Image Anal., 70, 2021, 101992.
Yu, Qihang, Yang, Dong, Roth, Holger, Bai, Yutong, Zhang, Yixiao, Yuille, Alan L, Xu, Daguang, 2020. C2fnas: Coarse-to-fine neural architecture search for 3d medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4126–4135.
Yushkevich, Paul A, Piven, Joseph, Hazlett, Heather Cody, Smith, Rachel Gimpel, Ho, Sean, Gee, James C, Gerig, Guido, User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31:3 (2006), 1116–1128.
Zang, Si-Tian, Han, Xu, Cui, Qi, Chang, Qing, Wu, Qi-Jun, Zhao, Yu-Hong, Imaging characteristics of coronavirus disease 2019 (COVID-19) in pediatric cases: a systematic review and meta-analysis. Transl. Pediatrics, 10(1), 2021, 1.
Zhang, Kang, Liu, Xiaohong, Shen, Jun, Li, Zhihuan, Sang, Ye, Wu, Xingwang, Zha, Yunfei, Liang, Wenhua, Wang, Chengdi, Wang, Ke, et al. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell 181:6 (2020), 1423–1433.
Zhou, Tongxue, Canu, Stéphane, Ruan, Su, Automatic COVID-19 CT segmentation using U-net integrated spatial and channel attention mechanism. Int. J. Imaging Syst. Technol. 31:1 (2021), 16–27.
Zhu, Zhuotun, Liu, Chenxi, Yang, Dong, Yuille, Alan, Xu, Daguang, V-nas: Neural architecture search for volumetric medical image segmentation. 2019 International Conference on 3D Vision (3DV), 2019, IEEE, 240–248.
Zhu, Jieyun, Zhong, Zhimei, Li, Hongyuan, Ji, Pan, Pang, Jielong, Li, Bocheng, Zhang, Jianfeng, CT imaging features of 4121 patients with COVID-19: A meta-analysis. J. Med. Virol. 92:7 (2020), 891–902.
Zoph, Barret, Vasudevan, Vijay, Shlens, Jonathon, Le, Quoc V, 2018. Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8697–8710.
