Fourier-Mixed Window Attention: Accelerating Informer for Long Sequence Time-Series Forecasting

arXiv:2307.00493v2 Announce Type: replace
Abstract: We study a fast local-global window-based attention method to accelerate Informer for long sequence time-series forecasting. While window attention is local and a considerable computational saving, it lacks the ability to capture global token information which is compensated by a subsequent Fourier transform block. Our method, named FWin, does not rely on query sparsity hypothesis and an empirical approximation underlying the ProbSparse attention of Informer. Through experiments on univariate and multivariate datasets, we show that …

abstract arxiv attention block computational cs.ai cs.lg forecasting fourier global information mixed saving series study token type