DESIGNATOR: a Toolset for Automated GAN-enhanced Search-based Testing and Retraining of DNNs in Martian Environments

We present DESIGNATOR, a toolset for
generating datasets for testing and retraining deep neu-
ral networks (DNNs) performing computer vision tasks
in Martian-like environments. The toolset integrates
Marsim, a simulator of the Mars environment, and
DESIGNATE, a search-based approach combining sim-
ulation, generative adversarial networks (GANs), and
search-based test input generation. The tool enables
users to select a search strategy, launch simulations
in MarsSim, and observe the evolution of simulated
images, corresponding realistic images, ground truth
labels, model predictions, and fitness values. Beyond
supporting researchers and practitioners in generating
datasets capable of identifying failures and retraining
DNNs, DESIGNATOR can be used as a didactic tool
to explain how image datasets can be generated using
meta-heuristic search. Furthermore, MarsSim can be
used standalone, through its API and GUI. MarsSim
enables researchers to assess search-based approaches
beyond the predominantly studied automotive context.
A demo video of DESIGNATOR is available at https:
//youtu.be/fGxgpgViPEo.