Many Independent Objective (MIO) Algorithm for Test Suite Generation

Automatically generating test suites is intrinsically a multi-
objective problem, as any of the testing targets (e.g, statements to exe-
cute or mutants to kill) is an objective on its own. Test suite generation
has peculiarities that are quite di erent from other more regular optimi-
sation problems. For example, given an existing test suite, one can add
more tests to cover the remaining objectives. One would like the smallest
number of small tests to cover as many objectives as possible, but that
is a secondary goal compared to covering those targets in the rst place.
Furthermore, the amount of objectives in software testing can quickly
become unmanageable, in the order of (tens/hundreds of) thousands, es-
pecially for system testing of industrial size systems. Traditional multi-
objective optimisation algorithms can already start to struggle with just
four or ve objectives to optimize. To overcome these issues, di erent
techniques have been proposed, like for example the Whole Test Suite
(WTS) approach and the Many-Objective Sorting Algorithm (MOSA).
However, those techniques might not scale well to very large numbers
of objectives and limited search budgets (a typical case in system test-
ing). In this paper, we propose a novel algorithm, called Many Indepen-
dent Objective (MIO) algorithm. This algorithm is designed and tailored
based on the speci c properties of test suite generation. An empirical
study, on a set of arti cial and actual software, shows that the MIO al-
gorithm can achieve higher coverage compared to WTS and MOSA, as
it can better exploit the peculiarities of test suite generation.