Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptation. (arXiv:2208.10734v1 [cs.CL])

Dynamic contextualised word embeddings represent the temporal semantic
variations of words. We propose a method for learning dynamic contextualised
word embeddings by time-adapting a pretrained Masked Language Model (MLM) using
time-sensitive templates. Given two snapshots $C_1$ and $C_2$ of a corpora
taken respectively at two distinct timestamps $T_1$ and $T_2$, we first propose
an unsupervised method to select (a) pivot terms related to both $C_1$ and
$C_2$, and (b) anchor terms that are associated with a specific pivot term in …

arxiv learning template temporal word embeddings