An Empirical Evaluation of Evolutionary Algorithms for Unit Test Suite Generation

Context: Evolutionary algorithms have been shown to be e ective at generating
unit test suites optimised for code coverage. While many speci c aspects
of these algorithms have been evaluated in detail (e.g., test length and di erent
kinds of techniques aimed at improving performance, like seeding), the in
uence
of the choice of evolutionary algorithm has to date seen less attention in
the literature.
Objective: Since it is theoretically impossible to design an algorithm that is
the best on all possible problems, a common approach in software engineering
problems is to rst try the most common algorithm, a Genetic Algorithm, and
only afterwards try to re ne it or compare it with other algorithms to see if any
of them is more suited for the addressed problem. The objective of this paper
is to perform this analysis, in order to shed light on the in
uence of the search
algorithm applied for unit test generation.
Method: We empirically evaluate thirteen di erent evolutionary algorithms
and two random approaches on a selection of non-trivial open source classes.
All algorithms are implemented in the EvoSuite test generation tool, which includes recent optimisations such as the use of an archive during the search
and optimisation for multiple coverage criteria.
Results: Our study shows that the use of a test archive makes evolutionary
algorithms clearly better than random testing, and it con rms that the DynaMOSA
many-objective search algorithm is the most e ective algorithm for
unit test generation.
Conclusions: Our results show that the choice of algorithm can have a substantial
in
uence on the performance of whole test suite optimisation. Although
we can make a recommendation on which algorithm to use in practice, no algorithm
is clearly superior in all cases, suggesting future work on improved search
algorithms for unit test generation