F. Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger, "Simplifying graph convolutional networks, " in Proc. Int. Conf. Mach. Learn., PMLR, 2019, pp. 6861-6871.
M. Jiang et al., "Drug-target affinity prediction using graph neural network and contact maps, " RSC Adv., vol. 10, no. 35, pp. 20701-20712, 2020.
P. Bongini, M. Bianchini, and F. Scarselli, "Molecular generative graph neural networks for drug discovery, " Neurocomputing, vol. 450, pp. 242-252, 2021.
C. Shi, M. Xu, Z. Zhu, W. Zhang, M. Zhang, and J. Tang, "GraphAF: A flow-based autoregressive model for molecular graph generation, " 2020, arXiv:2001.09382.
S. Wu, F. Sun, W. Zhang, X. Xie, and B. Cui, "Graph neural networks in recommender systems: A survey, " ACM Comput. Surv., vol. 55, no. 5, pp. 1-37, 2022.
R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, "Graph convolutional neural networks for web-scale recommender systems, " in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2018, pp. 974-983.
W. Fan et al., "Graph neural networks for social recommendation, " in Proc. World Wide Web Conf., 2019, pp. 417-426.
C. Li, J. Ma, X. Guo, and Q. Mei, "DeepCas: An end-to-end predictor of information cascades, " in Proc. 26th Int. Conf. World Wide Web, 2017, pp. 577-586.
Z. Li, X. Ma, C. Xu, C. Cao, J. Xu, and J. Lü, "Boosting operational DNN testing efficiency through conditioning, " in Proc. 27th ACM Joint Meet. Eur. Softw. Eng. Conf. Symp. Found. Softw. Eng., 2019, pp. 499-509.
Y. Feng, Q. Shi, X. Gao, J. Wan, C. Fang, and Z. Chen, "DeepGini: Prioritizing massive tests to enhance the robustness of deep neural networks, " in Proc. 29th ACM SIGSOFT Int. Symp. Softw. Test. Anal., 2020, pp. 177-188.
Z. Wang, H. You, J. Chen, Y. Zhang, X. Dong, and W. Zhang, "Prioritizing test inputs for deep neural networks via mutation analysis, " in Proc. IEEE/ACM 43rd Int. Conf. Softw. Eng. (ICSE), Piscataway, NJ, USA: IEEE Press, 2021, pp. 397-409.
L. Zhang, X. Sun, Y. Li, and Z. Zhang, "A noise-sensitivity-analysisbased test prioritization technique for deep neural networks, " 2019, arXiv:1901.00054.
T. Byun, V. Sharma, A. Vijayakumar, S. Rayadurgam, and D. Cofer, "Input prioritization for testing neural networks, " in Proc. IEEE Int. Conf. Artif. Intell. Test. (AITest), Piscataway, NJ, USA: IEEE Press, 2019, pp. 63-70.
S. Yoo and M. Harman, "Regression testing minimization, selection and prioritization: A survey, " Softw. Test. Verification Rel., vol. 22, no. 2, pp. 67-120, 2012.
J. Kim, R. Feldt, and S. Yoo, "Guiding deep learning system testing using surprise adequacy, " in Proc. IEEE/ACM 41st Int. Conf. Softw. Eng. (ICSE), Piscataway, NJ, USA: IEEE Press, 2019, pp. 1039-1049.
W. Ma, M. Papadakis, A. Tsakmalis, M. Cordy, and Y. L. Traon, "Test selection for deep learning systems, " ACM Trans. Softw. Eng. Method., vol. 30, no. 2, pp. 1-22, 2021.
X. Dang, Y. Li, M. Papadakis, J. Klein, T. F. Bissyandé, and Y. L. Traon, "GraphPrior: Mutation-based test input prioritization for graph neural networks, " ACM Trans. Softw. Eng. Method., vol. 33, no. 1, pp. 1-40, 2023.
I. D. Mienye and Y. Sun, "A survey of ensemble learning: Concepts, algorithms, applications, and prospects, " IEEE Access, vol. 10, pp. 99129-99149, 2022.
O. Sagi and L. Rokach, "Ensemble learning: A survey, " Wiley Interdiscip. Rev., Data Mining Knowl. Discovery, vol. 8, no. 4, 2018, Art. no. e1249.
R. Polikar, "Ensemble learning, " in Ensemble Machine Learning. Berlin, Germany: Springer-Verlag, 2012, pp. 1-34.
N. Humbatova, G. Jahangirova, and P. Tonella, "DeepCrime: Mutation testing of deep learning systems based on real faults, " in Proc. 30th ACM SIGSOFT Int. Symp. Softw. Test. Anal., 2021, pp. 67-78.
Y. Jia and M. Harman, "An analysis and survey of the development of mutation testing, " IEEE Trans. Softw. Eng., vol. 37, no. 5, pp. 649-678, Sep./Oct. 2011.
T. Loise, X. Devroey, G. Perrouin, M. Papadakis, and P. Heymans, "Towards security-aware mutation testing, " in Proc. IEEE Int. Conf. Softw. Test. Verification Validation Workshops (ICSTW), Piscataway, NJ, USA: IEEE Press, 2017, pp. 97-102.
Z.-H. Zhou and Z.-H. Zhou, "Ensemble learning, " in Machine Learning, Singapore: Springer, 2021, pp. 181-210.
A. Mohammed and R. Kora, "An effective ensemble deep learning framework for text classification, " J. King Saud Univ.-Comput. Inf. Sci., vol. 34, no. 10, pp. 8825-8837, 2022.
P. Goel, R. Jain, A. Nayyar, S. Singhal, and M. Srivastava, "Sarcasm detection using deep learning and ensemble learning, " Multimedia Tools Appl., vol. 81, no. 30, pp. 43229-43252, 2022.
D. Che, Q. Liu, K. Rasheed, and X. Tao, "Decision tree and ensemble learning algorithms with their applications in bioinformatics, " in Software Tools and Algorithms for Biological Systems, New York, NY, USA: Springer, 2011, pp. 191-199.
J. Chen, Z. Li, and S. Qin, "Ensemble learning for assessing degree of humor, " in Proc. Int. Conf. Big Data Inf. Comput. Netw. (BDICN), Piscataway, NJ, USA: IEEE Press, 2022, pp. 492-498.
F. Divina, A. Gilson, F. Goméz-Vela, M. García Torres, and J. F. Torres, "Stacking ensemble learning for short-term electricity consumption forecasting, " Energies, vol. 11, no. 4, 2018, Art. no. 949.
N. Littlestone and M. K. Warmuth, "The weighted majority algorithm, " Inf. Comput., vol. 108, no. 2, pp. 212-261, 1994.
D. Zügner and S. Günnemann, "Adversarial attacks on graph neural networks via meta learning, " 2019, arXiv:1902.08412.
K. Xu et al., "Topology attack and defense for graph neural networks: An optimization perspective, " 2019, arXiv:1906.04214.
A. Bojchevski and S. Günnemann, "Adversarial attacks on node embeddings via graph poisoning, " in Proc. Int. Conf. Mach. Learn., PMLR, 2019, pp. 695-704.
Y. Li, W. Jin, H. Xu, and J. Tang, "DeepRobust: A pyTorch library for adversarial attacks and defenses, " 2020, arXiv:2005.06149.
Z. Liu, Y. Dou, P. S. Yu, Y. Deng, and H. Peng, "Alleviating the inconsistency problem of applying graph neural network to fraud detection, " in Proc. 43rd Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2020, pp. 1569-1572.
M. Gori, G. Monfardini, and F. Scarselli, "A new model for learning in graph domains, " in Proc. IEEE Int. Joint Conf. Neural Netw., 2005, vol. 2, pp. 729-734.
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model, " IEEE Trans. Neural Netw., vol. 20, no. 1, pp. 61-80, Jan. 2009.
P. Velicković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, "Graph attention networks, " in Proc. Int. Conf. Learn. Representations, 2018, pp. 10-48550.
C. Sun, A. Shrivastava, C. Vondrick, R. Sukthankar, K. Murphy, and C. Schmid, "Relational action forecasting, " in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 273-283.
W. Pian, Y. Wu, X. Qu, J. Cai, and Z. Kou, "Spatial-temporal dynamic graph attention networks for ride-hailing demand prediction, " 2020, arXiv:2006.05905.
T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks, " 2016, arXiv:1609.02907.
P. Velicković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, "Graph attention networks, " 2017, arXiv:1710.10903.
W. Hamilton, Z. Ying, and J. Leskovec, "Inductive representation learning on large graphs, " in Proc. Adv. Neural Inf. Process. Syst., 2017, vol. 30, pp. 1025-1035.
J. Du, S. Zhang, G. Wu, J. M. Moura, and S. Kar, "Topology adaptive graph convolutional networks, " 2017, arXiv:1710.10370.
T.-Y. Liu et al., "Learning to rank for information retrieval, " Found. Trends® Inf. Retrieval, vol. 3, no. 3, pp. 225-331, 2009.
R. E. Wright, "Logistic regression, " 1995. [Online]. Available: https://psycnet.apa.org/record/1995-97110-007
L. Breiman, "Random forests, " Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
T. Chen and C. Guestrin, "XGBoost: A scalable tree boosting system, " in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2016, pp. 785-794.
G. Ke et al., "LightGBM: A highly efficient gradient boosting decision tree, " in Proc. Adv. Neural Inf. Process. Syst., 2017, vol. 30, pp. 3149-3157.
Q. H. Nguyen et al., "Influence of data splitting on performance of machine learning models in prediction of shear strength of soil, " Math. Prob. Eng., vol. 2021, pp. 1-15, 2021.
D. Shin, S. Yoo, M. Papadakis, and D.-H. Bae, "Empirical evaluation of mutation-based test case prioritization techniques, " Softw. Test. Verification Rel., vol. 29, nos. 1-2, 2019, Art. no. e1695.
Y. Lou, D. Hao, and L. Zhang, "Mutation-based test-case prioritization in software evolution, " in Proc. IEEE 26th Int. Symp. Softw. Rel. Eng. (ISSRE), Piscataway, NJ, USA: IEEE Press, 2015, pp. 46-57.
R. Meyes, M. Lu, C. W. de Puiseau, and T. Meisen, "Ablation studies to uncover structure of learned representations in artificial neural networks, " in Proc. Int. Conf. Artif. Intell. (ICAI), VA, USA, 2019, pp. 185-191.
M. Papadakis, M. Kintis, J. Zhang, Y. Jia, Y. Le Traon, and M. Harman, "Mutation testing advances: An analysis and survey, " in Advances in Computers, vol. 112. Elsevier, 2019, pp. 275-378.
M. Weiss and P. Tonella, "Simple techniques work surprisingly well for neural network test prioritization and active learning (replicability study), " in Proc. 31st ACM SIGSOFT Int. Symp. Softw. Test. Anal., 2022, pp. 139-150.
Q. Hu et al., "Towards exploring the limitations of active learning: An empirical study, " in Proc. 36th IEEE/ACM Int. Conf. Autom. Softw. Eng. (ASE), Piscataway, NJ, USA: IEEE Press, 2021, pp. 917-929.
S. Elbaum, A. G. Malishevsky, and G. Rothermel, "Test case prioritization: A family of empirical studies, " IEEE Trans. Softw. Eng., vol. 28, no. 2, pp. 159-182, Feb. 2002.
S. Geisler, T. Schmidt, H. Sirin, D. Zügner, A. Bojchevski, and S. Günnemann, "Robustness of graph neural networks at scale, " in Proc. Adv. Neural Inf. Process. Syst., 2021, vol. 34, pp. 7637-7649.
X. Zou et al., "TDGIA: Effective injection attacks on graph neural networks, " in Proc. 27th ACM SIGKDD Conf. Knowl. Discovery Data Mining, 2021, pp. 2461-2471.
D. Zügner, A. Akbarnejad, and S. Günnemann, "Adversarial attacks on neural networks for graph data, " in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2018, pp. 2847-2856.
J. Ma, S. Ding, and Q. Mei, "Towards more practical adversarial attacks on graph neural networks, " in Proc. Adv. Neural Inf. Process. Syst., 2020, vol. 33, pp. 4756-4766.
H. Wu, C. Wang, Y. Tyshetskiy, A. Docherty, K. Lu, and L. Zhu, "Adversarial examples on graph data: Deep insights into attack and defense, " 2019, arXiv:1903.01610.
Z. Yang, W. Cohen, and R. Salakhudinov, "Revisiting semi-supervised learning with graph embeddings, " in Proc. Int. Conf. Mach. Learn., PMLR, 2016, pp. 40-48.
B. Rozemberczki and R. Sarkar, "Characteristic functions on graphs: Birds of a feather, from statistical descriptors to parametric models, " in Proc. 29th ACM Int. Conf. Inf. Knowl. Manage., 2020, pp. 1325-1334.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks, " 2017, arXiv:1706.06083.
T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, "Meta-learning in neural networks: A survey, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 9, pp. 5149-5169, Sep. 2022.
X. Dong, Z. Yu, W. Cao, Y. Shi, and Q. Ma, "A survey on ensemble learning, " Front. Comput. Sci., vol. 14, no. 2, pp. 241-258, 2020.
A. Paszke et al., "PyTorch: An imperative style, high-performance deep learning library, " in Proc. Adv. Neural Inf. Process. Syst., 2019, vol. 32, pp. 8026-8037.
D. Wang and Y. Shang, "A new active labeling method for deep learning, " in Proc. Int. Joint Conf. Neural Netw. (IJCNN), Piscataway, NJ, USA: IEEE Press, 2014, pp. 112-119.
A. Arcuri and L. Briand, "A practical guide for using statistical tests to assess randomized algorithms in software engineering, " in Proc. 33rd Int. Conf. Softw. Eng., 2011, pp. 1-10.
P. E. McKnight and J. Najab, "Mann-Whitney U test, " in The Corsini Encyclopedia of Psychology, Hoboken, NJ, USA: Wiley, 2010, pp. 1-1.
G. Chryssolouris, M. Lee, and A. Ramsey, "Confidence interval prediction for neural network models, " IEEE Trans. Neural Netw., vol. 7, no. 1, pp. 229-232, Jan. 1996.
K. Kelley and K. J. Preacher, "On effect size, " Psychol. Methods, vol. 17, no. 2, pp. 137-152, 2012. [Online]. Available: https://psycnet.apa.org/record/2015-32022-018
J. Cohen, "A power primer, " 2016.
H. Do and G. Rothermel, "On the use of mutation faults in empirical assessments of test case prioritization techniques, " IEEE Trans. Softw. Eng., vol. 32, no. 9, pp. 733-752, Sep. 2006.
C. Henard, M. Papadakis, M. Harman, Y. Jia, and Y. Le Traon, "Comparing white-box and black-box test prioritization, " in Proc. IEEE/ACM 38th Int. Conf. Softw. Eng. (ICSE), Piscataway, NJ, USA: IEEE Press, 2016, pp. 523-534.
J. Chen, Y. Bai, D. Hao, Y. Xiong, H. Zhang, and B. Xie, "Learning to prioritize test programs for compiler testing, " in Proc. IEEE/ACM 39th Int. Conf. Softw. Eng. (ICSE), Piscataway, NJ, USA: IEEE Press, 2017, pp. 700-711.
J. Chen et al., "Coverage prediction for accelerating compiler testing, " IEEE Trans. Softw. Eng., vol. 47, no. 2, pp. 261-278, Feb. 2021.
D. Di Nardo, N. Alshahwan, L. Briand, and Y. Labiche, "Coveragebased test case prioritisation: An industrial case study, " in Proc. IEEE 6th Int. Conf. Softw. Test. Verification Validation, Piscataway, NJ, USA: IEEE Press, 2013, pp. 302-311.
M. Papadakis, C. Henard, and Y. L. Traon, "Sampling program inputs with mutation analysis: Going beyond combinatorial interaction testing, " in Proc. 7th IEEE Int. Conf. Softw. Test. Verification Validation (ICST), Cleveland, OH, USA. Liverpool, U.K. IEEE Comput. Soc. Press, 2014, pp. 1-10, doi: 10.1109/ICST.2014.11.
N. Gökce, M. Eminov, and F. Belli, "Coverage-based, prioritized testing using neural network clustering, " in Proc. Comput. Inf. Sci., 21st Int. Symp., Berlin, Germany: Springer-Verlag, 2006, pp. 1060-1071.
F. Belli and C. J. Budnik, "Test minimization for human-computer interaction, " Appl. Intell., vol. 26, pp. 161-174, 2007.
N. Gökçe, F. Belli, M. Eminli, and B. T. Dincer, "Model-based test case prioritization using cluster analysis: A soft-computing approach, " Turkish J. Elect. Eng. Comput. Sci., vol. 23, no. 3, pp. 623-640, 2015.
L. Deng, J. Offutt, P. Ammann, and N. Mirzaei, "Mutation operators for testing android apps, " Inf. Softw. Technol., vol. 81, pp. 154-168, 2017.
L. Deng, N. Mirzaei, P. Ammann, and J. Offutt, "Towards mutation analysis of android apps, " in Proc. IEEE 8th Int. Conf. Softw. Test. Verification Validation Workshops (ICSTW), Piscataway, NJ, USA: IEEE Press, 2015, pp. 1-10.
L. Ma et al., "Deepmutation: Mutation testing of deep learning systems, " in Proc. IEEE 29th Int. Symp. Softw. Rel. Eng. (ISSRE), Piscataway, NJ, USA: IEEE Press, 2018, pp. 100-111.
Q. Hu, L. Ma, X. Xie, B. Yu, Y. Liu, and J. Zhao, "Deepmutation++: A mutation testing framework for deep learning systems, " in Proc. 34th IEEE/ACM Int. Conf. Autom. Softw. Eng. (ASE), Piscataway, NJ, USA: IEEE Press, 2019, pp. 1158-1161.
W. Shen, J. Wan, and Z. Chen, "MuNN: Mutation analysis of neural networks, " in Proc. IEEE Int. Conf. Softw. Qual., Rel. Secur. Companion (QRS-C), Piscataway, NJ, USA: IEEE Press, 2018, pp. 108-115.
J. Chen, Z. Wu, Z. Wang, H. You, L. Zhang, and M. Yan, "Practical accuracy estimation for efficient deep neural network testing, " ACM Trans. Softw. Eng. Method., vol. 29, no. 4, pp. 1-35, 2020.
A. Guerriero, R. Pietrantuono, and S. Russo, "Operation is the hardest teacher: Estimating DNN accuracy looking for mispredictions, " in Proc. IEEE/ACM 43rd Int. Conf. Softw. Eng. (ICSE), Piscataway, NJ, USA: IEEE Press, 2021, pp. 348-358.
X. Dang, Y. Li, M. Papadakis, J. Klein, T. F. Bissyandé, and Y. Le Traon, "Test input prioritization for machine learning classifiers, " IEEE Trans. Softw. Eng., vol. 50, no. 3, pp. 413-442, Mar. 2024.
Y. Li, X. Dang, L. Ma, J. Klein, Y. L. Traon, and T. F. Bissyandé, "Test input prioritization for 3D point clouds, " ACM Trans. Softw. Eng. Method., 2023.
B. Kim, R. Khanna, and O. O. Koyejo, "Examples are not enough, learn to criticize! Criticism for interpretability, " in Proc. Adv. Neural Inf. Process. Syst., 2016, vol. 29, pp. 2288-2296.
L. Ma et al., "DeepCT: Tomographic combinatorial testing for deep learning systems, " in Proc. IEEE 26th Int. Conf. Softw. Anal., Evol. Reengineering (SANER), Piscataway, NJ, USA: IEEE Press, 2019, pp. 614-618.
K. Pei, Y. Cao, J. Yang, and S. Jana, "DeepXplore: Automated whitebox testing of deep learning systems, " in Proc. 26th Symp. Oper. Syst. Princ., 2017, pp. 1-18.
L. Ma et al., "DeepGauge: Multi-granularity testing criteria for deep learning systems, " in Proc. 33rd ACM/IEEE Int. Conf. Autom. Softw. Eng., 2018, pp. 120-131.
