M. Abadi et al., “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015. [Online]. Available: tensorflow. org.
A. T. Acree, “On Mutation,” Ph.D. thesis, School of Information and Computer Science, Georgia Inst. Technol., Atlanta, Georgia, 1980.
P. Ammann, M. E. Delamaro, and J. Offutt, “Establishing theoretical minimal sets of mutants,” in Proc. IEEE 7th Int. Conf. Softw. Testing, Verification Valid., 2014, pp. 21–30.
P. Ammann, J. Offutt, “Introduction to Software Testing,” 1st ed. New York, NY, USA: Cambridge Univ. Press, 2008.
J. H. Andrews, L. C. Briand, Y. Labiche, and A. S. Namin, “Using mutation analysis for assessing and comparing testing coverage criteria,” IEEE Trans. Softw. Eng., vol. 32, no. 8, pp. 608–624, Aug. 2006.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate, in Proc. 3rd Int. Conf. Learn. Representations, 2015.
D. Bahdanau, J. Chorowski, D. Serdyuk, P. Brakel, and Y. Bengio, “End-to-end attention-based large vocabulary speech recognition,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2016, pp. 4945–4949.
D. Britz, A. Goldie, M.-T. Luong, and Q. Le, “Massive exploration of neural machine translation architectures,” in Proc. Conf. Empir. Methods Natural Lang. Process., 2017, pp. 1442–1451.
T. A. Budd and D. Angluin, “Two notions of correctness and their relation to testing,“Acta Inf., vol. 18, no. 1, pp. 31–45, Mar. 1982.
C. Cadar, D. Dunbar, and D. Engler, “KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs,” in Proc. 8th USENIX Conf. Oper. Syst. Des. Implementation, 2008, pp. 209–224.
T. T. Chekam, M. Papadakis, T. F. Bissyande, Yves Le Traon, and K. Sen, “Selecting fault revealing mutants,“Empir. Softw. Eng., vol. 25, no. 1 pp. 434–487, 2020.
T. T. Chekam, M. Papadakis, M. Cordy, and Y. Le Traon, “Killing stubborn mutants with symbolic execution,“ACM Trans. Softw. Eng. Methodol., vol. 30, no. 2, Jan. 2021.
T. T. Chekam, M. Papadakis, and Y. Le Traon, “Mart: A mutant generation tool for LLVM,” in Proc. 27th ACM Joint Meeting Eur. Softw. Eng. Conf. Symp. Found. Softw. Eng., 2019, page 1080–1084.
T. T. Chekam, M. Papadakis, Y. Le Traon, and M. Harman, “An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption,” in Proc. 39th Int. Conf. Softw. Eng., 2017, pp. 597–608.
L. Chen, L. Zhang, “Speeding up mutation testing via regression test selection: An extensive study,” in Proc. 11th IEEE Int. Conf. Softw. Testing Verification Valid., 2018, pp. 58–69.
K. Cho et al., “Learning phrase representations using RNN encoder–decoder for statistical machine translation,” in Proc. Conf. Empir. Methods Natural Lang. Process., 2014, pp. 1724–1734.
H. Coles, T. Laurent, C. Henard, M. Papadakis, and A. Ventresque, “PIT: A practical mutation testing tool for java (demo),” in Proc. 25th Int. Symp. Softw. Testing Anal., 2016, pp. 449–452.
M. L. Collard, J. I. Maletic, “srcML 1.0: Explore, analyze, and manipulate source code,” in Proc. IEEE Int. Conf. Softw. Maintenance Evolution, 2016, pp. 649–649.
R. A. DeMillo, R. J. Lipton, and F. G. Sayward, “Hints on test data selection: Help for the practicing programmer,” IEEE Computer, vol. 11, no. 4, pp. 34–41, Apr. 1978.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pretraining of deep bidirectional transformers for language understanding,” in Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol., 2019, pp. 4171–4186.
J. J. Domınguez-Jimenez, A. Estero-Botaro, A. Garcıa-Domınguez, and I. Medina-Bulo, “Evolutionary mutation testing,” Inf. Softw. Technol., vol. 53, no. 10, pp. 1108–1123, 2011.
Z. Feng et al., “CodeBERT: A pre-trained model for programming and natural languages,” in Proc. Findings Assoc. Comput. Linguistics: EMNLP, 2020, pp. 1536–1547.
G. Fraser, A. Zeller, “Mutation-driven generation of unit tests and oracles,” in Proc. ACM Int. Symp. Softw. Testing Anal., 2010, pp. 147–158.
D. Gong, G. Zhang, X. Yao, and F. Meng, “Mutant reduction based on dominance relation for weak mutation testing,” Inf. Softw. Technol., vol. 81, pp. 82–96, 2017.
F. Hariri, A. Shi, V. Fernando, S. Mahmood, and D. Marinov, “Comparing mutation testing at the levels of source code and compiler intermediate representation,” in Proc. 12th IEEE Conf. Softw. Testing, Valid. Verification, 2019, pp. 114–124.
J. Hestness et al., “Deep learning scaling is predictable, empirically,” 2017, arXiv:1712.00409.
Y. Jia, M. Harman, “Higher order mutation testing,” Inf. Softw. Technol., vol. 51, no. 10, pp. 1379–1393, 2009.
R. Jozefowicz, O. Vinyals, M. Schuster, N. Shazeer, and Y. Wu, “Exploring the limits of language modeling,” 2016, arXiv:1602.02410.
Rene Just, B. Kurtz, and P. Ammann, “Inferring mutant utility from program context,” in Proc. 26th ACM SIGSOFT Int. Symp. Softw. Testing Anal., 2017, pp. 284–294.
M. Kintis, M. Papadakis, and N. Malevris, “Evaluating mutation testing alternatives: A collateral experiment,” in Proc. Asia Pacific Softw. Eng. Conf., 2010. pp. 300–309.
M. Kintis, M. Papadakis, Y. Jia, N. Malevris, Y. Le Traon, and M. Harman, “Detecting trivial mutant equivalences via compiler optimisations,” IEEE Trans. Softw. Eng., vol. 44, no. 4, pp. 308–333, Apr. 2018.
M. Kintis, M. Papadakis, and N. Malevris, “Employing second-order mutation for isolating first-order equivalent mutants,” Softw. Testing Verification Rel., vol. 25, no. 5/7, pp. 508–535, 2015.
B. Kurtz, P. Ammann, M. E. Delamaro, J. Offutt, and L. Deng, “Mutant subsumption graphs,” in IEEE 7th Int. Conf. Softw. Testing Verification Valid. Workshops, 2014, pp. 176–185.
B. Kurtz, P. Ammann, J. Offutt, Marcio, E. Delamaro, M. Kurtz, and Nida Gokçe, “Analyzing the validity of selective mutation with dominator mutants,” in Proc. 24th ACM SIGSOFT Int. Symp. Found. Softw. Eng., 2016, pp. 571–582.
Linux in 2020: 27.8 million lines of code in the kernel, 1.3 million in systemd,” Accessed: Oct. 12, 2020. [Online]. Available: https://www.theregister.com/2020/01/06/linux_2020_kernel_systemd_code/
Z. Liu, W. Lin, Ya Shi, and J. Zhao, “A robustly optimized bert pre-training approach with post-training,” in Proc. China Nat. Conf. Chin. Comput. Linguistics, 2021, pp. 471–484.
D. Mahajan et al., “Exploring the limits of weakly supervised pre-training,” in Proc. Eur. Conf. Comput. Vis, 2018, pp. 181–196.
M. Marcozzi, S. Bardin, N. Kosmatov, M. Papadakis, V. Prevosto, and Loïc Correnson, “Time to clean your test objectives,” in Proc. 40th Int. Conf. Softw. Eng., 2018, pp. 456–467.
A. Mastropaolo et al., “Studying the usage of text-to-text transfer transformer to support code-related tasks,” in Proc. IEEE/ACM 43rd Int. Conf. Softw. Eng., 2021, pp. 336–347.
B. W. Matthews, “Comparison of the predicted and observed secondary structure of t4 phage lysozyme,” Biochimica et Biophysica Acta (BBA) - Protein Struct., vol. 405, no. 2, pp. 442–451.
S. Mirshokraie, A. Mesbah, and K. Pattabiraman, “Guided mutation testing for javascript web applications,” IEEE Trans. Softw. Eng., vol. 41, no. 5, pp. 429–444, May 2015.
P. Morrison, K. Herzig, B. Murphy, and L. Williams, “Challenges with applying vulnerability prediction models,” in Proc. Symp. Bootcamp Sci. Secur., 2015, pp. 1–9.
A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf, “An experimental determination of sufficient mutant operators,” ACM Trans. Softw. Eng. Methodol., vol. 5, no. 2, pp. 99–118, 1996.
M. Oquab, L. Bottou, I. Laptev, and J. Sivic, “Learning and transferring mid-level image representations using convolutional neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 1717–1724.
M. Papadakis, T. T. Chekam, and Y. Le Traon, “Mutant quality indicators,” in Proc. IEEE Int. Conf. Softw. Testing Verification Valid. Workshops, 2018, pp. 32–39.
M. Papadakis, C. Henard, M. Harman, Y. Jia, and Y. Le Traon, “Threats to the validity of mutation-based test assessment,” in Proc. 25th Int. Symp. Softw. Testing Anal., 2016, pp. 354–365.
M. Papadakis, M. Kintis, J. Zhang, Y. Jia, Y. Le Traon, and M. Harman, “Chapter six - Mutation testing advances: An analysis and survey,” Advances Comput., vol. 112, pp. 275–378, 2019.
M. Papadakis and N. Malevris, “Automatic mutation test case generation via dynamic symbolic execution,” in Proc. IEEE 21st Int. Symp. Softw. Rel. Eng., 2010, pp. 121–130.
M. Papadakis and N. Malevris, “An empirical evaluation of the first and second order mutation testing strategies,” in Proc. 3rd Int. Conf. Softw. Testing Verification Valid., 2010, pp. 90–99.
M. Papadakis, D. Shin, S. Yoo, and D.-H. Bae, “Are mutation scores correlated with real fault detection?: A large scale empirical study on the relationship between mutants and real faults,” in Proc. 40th Int. Conf. Softw. Eng., 2018, pp. 537–548.
G. Petrovic and M. Ivankovic, “State of mutation testing at google,” in Proc. 40th IEEE/ACM Int. Conf. Softw. Eng. Softw. Eng. Pract. Track, 2018, pp. 163–171.
C. Raffel et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” 2019, arXiv:1910.10683.
O. Russakovsky et al., “Imagenet large scale visual recognition challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, 2015.
N. Shazeer, et al., “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” in Proc. 5th Int. Conf. Learn. Representations, 2017.
M. Shepperd, D. Bowes, and T. Hall, “Researcher bias: The use of machine learning in software defect prediction,” IEEE Trans. Softw. Eng., vol. 40, no. 6, pp. 603–616, Jun. 2014.
A. Shewalkar, D. Nyavanandi, and S. Ludwig, “Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU,” J. Artif. Intell. Soft Comput. Res., vol. 9, pp. 235–245, 2019.
C.-AI Sun, F. Xue, H. Liu, and X. Zhang, “A path-aware approach to mutant reduction in mutation testing,” Inf. Softw. Technol., vol. 81, pp. 65–81, 2017.
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks, in Proc. 27th Int. Conf. Neural Inf. Process. Syst., 2014, pp. 3104–3112.
G. Tang, M. Muller, € A. Rios, and R. Sennrich,”Why self-attention? A targeted evaluation of neural machine translation architectures,” in Proc. 2018 Conf. Empir. Methods Natural Lang. Process., 2018, pp. 4263–4272.
M. Tufano, C. Watson, G. Bavota, M. Di Penta, M. White, and D. Poshyvanyk,”Learning how to mutate source code from bug-fixes,” in Proc. IEEE Int. Conf. Softw. Maintenance Evolution, 2019, pp. 301–312.
M. Tufano, C. Watson, G. Bavota, M. Di Penta, M. White, and D. Poshyvanyk,”An empirical study on learning bug-fixing patches in the wild via neural machine translation,” ACM Trans. Softw. Eng. Methodol., vol. 28, no. 4, pp. 19:1–19:29, 2019.
A. Vargha and H. D. Delaney,”A critique and improvement of the”CL” common language effect size statistics of McGraw and wong,” J. Educ. Behav. Statist., vol. 25, no. 2, pp. 101–132, 2000.
J. Yangqing et al.,”Convolutional architecture for fast feature embedding,” in Proc. ACM Int. Conf. Multimedia, 2014, pp. 675–678.
B. Wang, Y. Xiong, Y. Shi, Lu Zhang, and D. Hao,”Faster mutation analysis via equivalence modulo states,“in Proc. 26th ACM SIGSOFT Int. Symp. Softw. Testing Anal., 2017, pp. 295–306.
X. Yao, M. Harman, and Y. Jia, “A study of equivalent and stubborn mutation operators using human analysis of equivalence,” in Proc. 36th Int. Conf. Softw. Eng., 2014, pp. 919–930.
J. Zhang, L. Zhang, M. Harman, D. Hao, Y. Jia, and L. Zhang, “Predictive mutation testing,” IEEE Trans. Softw. Eng., vol. 45, no. 9, pp. 898–918, Sep. 2019.
L. Zhang, M. Gligoric, D. Marinov, and S. Khurshid, “Operator-based and random mutant selection: Better together,” in 28th IEEE/ACM Int. Conf. Automated Softw. Eng., 2013, pp. 92–102.
L. Zhang, D. Marinov, and S. Khurshid, “Faster mutation testing inspired by test prioritization and reduction,” in Proc. Int. Symp. Softw. Testing Anal, 2013, pp. 235–245.
