Learning to Embed Time Series Patches Independently

arXiv:2312.16427v2 Announce Type: replace
Abstract: Masked time series modeling has recently gained much attention as a self-supervised representation learning strategy for time series. Inspired by masked image modeling in computer vision, recent works first patchify and partially mask out time series, and then train Transformers to capture the dependencies between patches by predicting masked patches from unmasked patches. However, we argue that capturing such patch dependencies might not be an optimal strategy for time series representation learning; rather, learning to …

arxiv cs.ai cs.lg embed series stat.ml time series type