Marcozzi, Michaël; Imperial College London > Department of Computing
Bardin, Sébastien; CEA-LIST Paris
Kosmatov, Nikolai; CEA-LIST Paris
PAPADAKIS, Mike ; University of Luxembourg > Faculty of Science, Technology and Communication (FSTC) > Computer Science and Communications Research Unit (CSC)
Prevosto, Virgile; CEA-LIST Paris
Correnson, Loïc; CEA-LIST Paris
External co-authors :
yes
Language :
English
Title :
Time to Clean Your Test Objectives
Publication date :
May 2018
Event name :
40th International Conference on Software Engineering (ICSE'18)
Event date :
from 27-5-2018 t0 3-6-2018
Audience :
International
Main work title :
40th International Conference on Software Engineering, May 27 - 3 June 2018, Gothenburg, Sweden
Paul Ammann, Márcio Eduardo Delamaro, and Jeff Offutt. 2014. Establishing Theoretical Minimal Sets of Mutants. In Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014, March 31 2014-April 4, 2014, Cleveland, Ohio, USA. 21–30.
Paul Ammann and Jeff Offutt. 2008. Introduction to Software Testing (1 ed.). Cambridge University Press.
Saswat Anand, Edmund K. Burke, Tsong Yueh Chen, John A. Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, and Phil McMinn. 2013. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86, 8 (2013), 1978–2001.
Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2010. Structural coverage of feasible code. In The 5th Workshop on Automation of Software Test, AST 2010, May 3-4, 2010, Cape Town, South Africa. 59–66.
Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2011. Enhancing structural software coverage by incrementally computing branch executability. Software Quality Journal 19, 4 (2011), 725–751.
Mauro Baluda, Giovanni Denaro, and Mauro Pezzè. 2016. Bidirectional Symbolic Analysis for Effective Branch Testing. IEEE Trans. Software Eng. 42, 5 (2016), 403–426.
Sébastien Bardin, Omar Chebaro, Mickaël Delahaye, and Nikolai Kosmatov. 2014. An All-in-One Toolkit for Automated White-Box Testing. In TAP. Springer.
Sébastien Bardin, Mickaël Delahaye, Robin David, Nikolai Kosmatov, Mike Papadakis, Yves Le Traon, and Jean-Yves Marion. 2015. Sound and Quasi-Complete Detection of Infeasible Test Requirements. In ICST.
Sébastien Bardin, Nikolai Kosmatov, and François Cheynier. 2014. Efficient Leveraging of Symbolic Execution to Advanced Coverage Criteria. In Software Testing, Verification and Validation (ICST), 2014 IEEE Seventh International Conference on (pp. 173-182). IEEE.
Mike Barnett and Rustan Leino. 2005. Weakest-Precondition of Unstructured Programs. In ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering (PASTE). 82–87.
Patrick Baudin, Pascal Cuoq, Jean C. Filliâtre, Claude Marché, Benjamin Monate, Yannick Moy, and Virgile Prevosto. [n. d.]. ACSL: ANSI/ISO C Specification Language. http://frama-c.com/acsl.html
Nels E. Beckman, Aditya V. Nori, Sriram K. Rajamani, Robert J. Simmons, SaiDeep Tetali, and Aditya V. Thakur. 2010. Proofs from Tests. IEEE Trans. Software Eng. 36, 4 (2010), 495–508.
Antonia Bertolino and Martina Marré. 1994. Automatic Generation of Path Covers Based on the Control Flow Analysis of Computer Programs. IEEE Trans. Software Eng. 20, 12 (1994), 885–899.
Dirk Beyer, Adam Chlipala, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2004. Generating Tests from Counterexamples. In 26th International Conference on Software Engineering (ICSE 2004), 23-28 May 2004, Edinburgh, United Kingdom. 326–335.
Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2007. The software model checker Blast. STTT 9, 5-6 (2007), 505–525.
Thierry Titcheu Chekam, Mike Papadakis, Yves Le Traon, and Mark Harman. 2017. An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption. In Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, Buenos Aires, Argentina, May 20-28, 2017. 597–608.
Takeshi Chusho. 1987. Test Data Selection and Quality Estimation Based on the Concept of Esssential Branches for Path Testing. IEEE Trans. Software Eng. 13, 5 (1987), 509–517.
Leonardo De Moura and Nikolaj Bjørner. 2011. Satisfiability Modulo Theories: Introduction and Applications. Commun. ACM 54, 9 (Sept. 2011), 69–77.
Mickaël Delahaye, Bernard Botella, and Arnaud Gotlieb. 2015. Infeasible path generalization in dynamic symbolic execution. Information & Software Technology 58 (2015), 403–418.
Richard A. DeMillo and A. Jefferson Offutt. 1991. Constraint-Based Automatic Test Data Generation. IEEE Trans. Software Eng. 17, 9 (1991), 900–910.
E. W. Dijkstra. 1976. A Discipline of Programming. Prentice Hall.
Phyllis G. Frankl and Oleg Iakounenko. 1998. Further Empirical Studies of Test Effectiveness. In Proceedings of the 6th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT’98/FSE-6). ACM, New York, NY, USA, 153–162.
Milos Gligoric, Alex Groce, Chaoqiang Zhang, Rohan Sharma, Mohammad Amin Alipour, and Darko Marinov. 2015. Guidelines for Coverage-Based Comparisons of Non-Adequate Test Suites. ACM Trans. Softw. Eng. Methodol. 24, 4 (2015), 22:1–22:33.
Allen Goldberg, Tie-Cheng Wang, and David Zimmerman. 1994. Applications of Feasible Path Analysis to Program Testing. In Proceedings of the 1994 International Symposium on Software Testing and Analysis, ISSTA 1994, Seattle, WA, USA, August 17-19, 1994. 80–94.
Dunwei Gong, Gongjie Zhang, Xiangjuan Yao, and Fanlin Meng. 2017. Mutant reduction based on dominance relation for weak mutation testing. Information & Software Technology 81 (2017), 82–96.
Bernhard J. M. Grün, David Schuler, and Andreas Zeller. 2009. The Impact of Equivalent Mutants. In Second International Conference on Software Testing Verification and Validation, ICST 2009, Denver, Colorado, USA, April 1-4, 2009, Workshops Proceedings. 192–199.
C. A. R. Hoare. 1969. An axiomatic basis for computer programming. Commun. ACM 12, 10 (October 1969), 576–580 and 583.
Monica Hutchins, Herb Foster, Tarak Goradia, and Thomas Ostrand. 1994. Experiments of the Effectiveness of Dataflow- and Controlflow-based Test Adequacy Criteria. In Proceedings of the 16th International Conference on Software Engineering (ICSE’94). IEEE Computer Society Press, Los Alamitos, CA, USA, 191–200.
Monica Hutchins, Herbert Foster, Tarak Goradia, and Thomas J. Ostrand. 1994. Experiments of the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria. In Proceedings of the 16th International Conference on Software Engineering. 191–200.
M. Kintis, M. Papadakis, Y. Jia, N. Malevris, Y. Le Traon, and M. Harman. 2017. Detecting Trivial Mutant Equivalences via Compiler Optimisations. IEEE Transactions on Software Engineering PP, 99 (2017), 1–1.
Marinos Kintis, Mike Papadakis, and Nicos Malevris. 2010. Evaluating Mutation Testing Alternatives: A Collateral Experiment. In 17th Asia Pacific Software Engineering Conference, APSEC 2010, Sydney, Australia, November 30 - December 3, 2010. 300–309.
Marinos Kintis, Mike Papadakis, and Nicos Malevris. 2015. Employing second-order mutation for isolating first-order equivalent mutants. Softw. Test., Verif. Reliab. 25, 5-7 (2015), 508–535.
Florent Kirchner, Nikolai Kosmatov, Virgile Prevosto, Julien Signoles, and Boris Yakobowski. 2015. Frama-C: A Program Analysis Perspective. Formal Aspects of Computing Journal (2015).
Nikolai Kosmatov. 2008. All-Paths Test Generation for Programs with Internal Aliases. In 19th International Symposium on Software Reliability Engineering (ISSRE 2008), 11-14 November 2008, Seattle/Redmond, WA, USA. 147–156.
Bob Kurtz, Paul Ammann, and Jeff Offutt. 2015. Static analysis of mutant subsumption. In Eighth IEEE International Conference on Software Testing, Verification and Validation, ICST 2015 Workshops, Graz, Austria, April 13-17, 2015. 1–10.
Bob Kurtz, Paul Ammann, Jeff Offutt, Márcio Eduardo Delamaro, Mariet Kurtz, and Nida Gökçe. 2016. Analyzing the validity of selective mutation with dominator mutants. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, November 13-18, 2016. 571–582.
Bob Kurtz, Paul Ammann, Jeff Offutt, and Mariet Kurtz. 2016. Are We There Yet? How Redundant and Equivalent Mutants Affect Determination of Test Completeness. In Ninth IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops 2016, Chicago, IL, USA, April 11-15, 2016. 142–151.
Sébastien Lapierre, Ettore Merlo, Gilles Savard, Giuliano Antoniol, Roberto Fiutem, and Paolo Tonella. 1999. Automatic Unit Test Data Generation Using Mixed-Integer Linear Programming and Execution Trees. In 1999 International Conference on Software Maintenance, ICSM 1999, Oxford, England, UK, August 30 - September 3, 1999. 189–198.
Michaël Marcozzi, Sébastien Bardin, Mickaël Delahaye, Nikolai Kosmatov, and Virgile Prevosto. 2017. Taming Coverage Criteria Heterogeneity with LTest. In 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, March 13-17, 2017. 500–507.
Michaël Marcozzi, Mickaël Delahaye, Sébastien Bardin, Nikolai Kosmatov, and Virgile Prevosto. 2017. Generic and Effective Specification of Structural Test Objectives. In 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, March 13-17, 2017. 436–441.
Aditya P. Mathur. 2008. Foundations of Software Testing. Addison-Wesley Prof.
Glenford J. Myers and Corey Sandler. 2004. The Art of Software Testing. John Wiley & Sons.
Glenford J. Myers, Corey Sandler, and Tom Badgett. 2011. The Art of Software Testing (3 ed.). Wiley.
Minh Ngoc Ngo and Hee Beng Kuan Tan. 2008. Heuristics-based infeasible path detection for dynamic test data generation. Information & Software Technology 50, 7-8 (2008), 641–655.
A. Jefferson Offutt, Ammei Lee, Gregg Rothermel, Roland H. Untch, and Christian Zapf. 1996. An Experimental Determination of Sufficient Mutant Operators. ACM Trans. Softw. Eng. Methodol. 5, 2 (April 1996), 99–118.
A. Jefferson Offutt and Stephen D. Lee. 1994. An Empirical Evaluation of Weak Mutation. IEEE Trans. Software Eng. 20, 5 (1994).
A. Jefferson Offutt and Jie Pan. 1997. Automatically Detecting Equivalent Mutants and Infeasible Paths. Softw. Test., Verif. Reliab. 7, 3 (1997), 165–192.
OpenSSL 2018. https://www.openssl.org. (2018).
Rahul Pandita, Tao Xie, Nikolai Tillmann, and Jonathan de Halleux. 2010. Guided Test Generation for Coverage Criteria. In ICSM.
Mike Papadakis, Márcio Eduardo Delamaro, and Yves Le Traon. 2014. Mitigating the effects of equivalent mutants with mutant classification strategies. Sci. Comput. Program. 95 (2014), 298–319.
Mike Papadakis, Christopher Henard, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Threats to the Validity of Mutation-based Test Assessment. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). ACM, New York, NY, USA, 354–365.
Mike Papadakis, Yue Jia, Mark Harman, and Yves Le Traon. 2015. Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Effective Equivalent Mutant Detection Technique. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (ICSE’15). IEEE Press, Piscataway, NJ, USA, 936–946.
Mike Papadakis and Nicos Malevris. 2012. Mutation based test case generation via a path selection strategy. Information & Software Technology 54, 9 (2012), 915–932.
Radio Technical Commission for Aeronautics. 1992. RTCA DO178-B Software Considerations in Airborne Systems and Equipment Certification. (1992).
Stuart C. Reid. 1995. The Software Testing Standard — How you can use it. In 3rd European Conference on Software Testing, Analysis and Review (EuroSTAR ’95). London.
David Schuler, Valentin Dallmeier, and Andreas Zeller. 2009. Efficient mutation testing by checking invariant violations. In Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, ISSTA 2009, Chicago, IL, USA, July 19-23, 2009. 69–80.
David Schuler and Andreas Zeller. 2013. Covering and Uncovering Equivalent Mutants. Softw. Test., Verif. Reliab. 23, 5 (2013), 353–374.
Ting Su, Zhoulai Fu, Geguang Pu, Jifeng He, and Zhendong Su. 2015. Combining Symbolic Execution and Model Checking for Data Flow Testing. In 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16-24, 2015, Volume 1. 654–665.
E. J. Weyuker. 1993. More Experience with Data Flow Testing. IEEE Trans. Softw. Eng. 19, 9 (Sept. 1993), 912–919.
M. R. Woodward, D. Hedley, and M. A. Hennell. 1980. Experience with Path Analysis and Testing of Programs. IEEE Trans. Softw. Eng. 6, 3 (May 1980), 278–286.
Martin R. Woodward, David Hedley, and Michael A. Hennell. 1980. Experience with Path Analysis and Testing of Programs. IEEE Trans. Software Eng. 6, 3 (1980), 278–286.
D. Yates and N. Malevris. 1989. Reducing the Effects of Infeasible Paths in Branch Testing. In Proceedings of the ACM SIGSOFT’89 Third Symposium on Software Testing, Analysis, and Verification (TAV3). ACM, New York, NY, USA, 48–54.
Hong Zhu, Patrick A. V. Hall, and John H. R. May. 1997. Software Unit Test Coverage and Adequacy. ACM Comput. Surv. 29, 4 (1997).
