MOTIF: A tool for Mutation Testing with Fuzzing

Mutation testing consists of generating test cases that detect faults
injected into software (generating mutants) which its original test suite could
not. By running such an augmented set of test cases, it may discover actual
faults that may have gone unnoticed with the original test suite. It is thus a
desired practice for embedded software running in safety-critical
cyber-physical systems (CPS). Unfortunately, the state-of-the-art tool
targeting C, a typical language for CPS software, relies on symbolic execution,
whose limitations often prevent its application. MOTIF overcomes such
limitations by leveraging grey-box fuzzing tools to generate unit test cases in
C that detect injected faults in mutants. Indeed, fuzzing tools automatically
generate inputs by exercising the compiled version of the software under test
guided by coverage feedback, thus overcoming the limitations of symbolic
execution. Our empirical assessment has shown that it detects more faults than
symbolic execution (i.e., up to 47 percentage points), when the latter is
applicable.