[R] Tsinghua University: Inverting Transformers Significantly Improves Time Series Forecasting

Transformers are great at NLP and computer vision tasks, but I was surprised to learn they still lag behind simple linear models at time series forecasting.

The issue is how most Transformer architectures treat each timestamp as a token and fuse all the variable data from that moment. This makes two big problems:

* **Variables recorded at slightly different times get blurred together,** losing important timing info
* **Each token can only see a single moment,** no long-term dependencies

So …

architectures computer computer vision data forecasting issue learn linear machinelearning nlp series simple tasks time series time series forecasting token transformer transformers tsinghua university university vision