Rough Transformers for Continuous and Efficient Time-Series Modelling

arXiv:2403.10288v1 Announce Type: cross
Abstract: Time-series data in real-world medical settings typically exhibit long-range dependencies and are observed at non-uniform intervals. In such contexts, traditional sequence-based recurrent models struggle. To overcome this, researchers replace recurrent architectures with Neural ODE-based models to model irregularly sampled data and use Transformer-based architectures to account for long-range dependencies. Despite the success of these two approaches, both incur very high computational costs for input sequences of moderate lengths and greater. To mitigate this, we introduce …

abstract architectures arxiv continuous cs.ai cs.lg data dependencies medical modelling researchers series stat.ml struggle transformer transformers type uniform world