A multifactorial evaluation framework for gene regulatory network reconstruction

In the past years, many computational methods have been developed to infer the
structure of gene regulatory networks from time series data. However, the applicability and
accuracy presumptions of such algorithms remain unclear due to experimental heterogeneity.
This paper assesses the performance of recent and successful network inference strategies
under a novel, multifactorial evaluation framework in order to highlight pragmatic tradeoffs
in experimental design. The effects of data quantity and systems perturbations are addressed,
thereby formulating guidelines for efficient resource management.
Realistic data were generated from six widely used benchmark models of rhythmic and nonrhythmic gene regulatory systems with random perturbations mimicking the effect of gene
knock-out or chemical treatments. Then, time series data of increasing lengths were provided
to five state-of-the-art network inference algorithms representing distinctive mathematical
paradigms. The performances of such network reconstruction methodologies are uncovered under
various experimental conditions. We report that the algorithms do not benefit equally from
data increments. Furthermore, at least for the studied rhythmic system, it is more profitable
for network inference strategies to be run on long time series rather than short time series
with multiple perturbations. By contrast, for the non-rhythmic systems, increasing the number
of perturbation experiments yielded better results than increasing the sampling frequency.
We expect that future benchmark and algorithm design would integrate such multifactorial
considerations to promote their widespread and conscientious usage.