Achieving Long-term Time Series Forecasting Models with Fewer Than 1k Parameters through Dynamic Sparse Training

Recently, there has been a surge in linear-based solutions
for the long-term time series forecasting (LTSF) task, driven by their
effectiveness and efficiency. Building on these models, this paper in-
troduces dynamic sparse training to explore the potential of creating
lightweight models for LTSF in scenarios with extremely limited compu-
tational resources, without the need for cumbersome architectural design.
To achieve this, we extensively explore the search space during sparse
training to achieve a very high sparsity ratio for linear-based models.
This approach results in models with fewer than 1k parameters and saves
up to 33 × training computational cost with only a slight performance
loss. Experimental results on six real-life multivariate and univariate time
series datasets demonstrate the effectiveness of our approach, achieving
a better trade-off between computational efficiency and performance. We
hope this finding opens up new research directions for the LTSF task,
enabling the development of extremely lightweight models that can oper-
ate effectively in resource-constrained environments. Our code is publicly
available at: https://github.com/QiaoXiao7282/LTSF-DST.