Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. (arXiv:2106.13008v5 [cs.LG] UPDATED)

Extending the forecasting time is a critical demand for real applications,
such as extreme weather early warning and long-term energy consumption
planning. This paper studies the long-term forecasting problem of time series.
Prior Transformer-based models adopt various self-attention mechanisms to
discover the long-range dependencies. However, intricate temporal patterns of
the long-term future prohibit the model from finding reliable dependencies.
Also, Transformers have to adopt the sparse versions of point-wise
self-attentions for long series efficiency, resulting in the information
utilization bottleneck. …

arxiv correlation forecasting transformers