Transformer Multivariate Forecasting: Less is More?

In the domain of multivariate forecasting, transformer models stand out as
powerful apparatus, displaying exceptional capabilities in handling messy
datasets from real-world contexts. However, the inherent complexity of these
datasets, characterized by numerous variables and lengthy temporal sequences,
poses challenges, including increased noise and extended model runtime. This
paper focuses on reducing redundant information to elevate forecasting accuracy
while optimizing runtime efficiency. We propose a novel transformer forecasting
framework enhanced by Principal Component Analysis (PCA) to tackle this
challenge. The framework is evaluated by five state-of-the-art (SOTA) models
and four diverse real-world datasets. Our experimental results demonstrate the
framework's ability to minimize prediction errors across all models and
datasets while significantly reducing runtime. From the model perspective, one
of the PCA-enhanced models: PCA+Crossformer, reduces mean square errors (MSE)
by 33.3% and decreases runtime by 49.2% on average. From the dataset
perspective, the framework delivers 14.3% MSE and 76.6% runtime reduction on
Electricity datasets, as well as 4.8% MSE and 86.9% runtime reduction on
Traffic datasets. This study aims to advance various SOTA models and enhance
transformer-based time series forecasting for intricate data.