iTransformer: Inverted Transformers Are Effective for Time Series Forecasting

arXiv:2310.06625v3 Announce Type: replace
Abstract: The recent boom of linear forecasting models questions the ongoing passion for architectural modifications of Transformer-based forecasters. These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp. However, Transformers are challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion. Besides, the embedding for each temporal token fuses multiple variates that represent potential delayed …

arxiv cs.lg forecasting series time series time series forecasting transformers type