Time Machine GPT

arXiv:2404.18543v1 Announce Type: cross
Abstract: Large language models (LLMs) are often trained on extensive, temporally indiscriminate text corpora, reflecting the lack of datasets with temporal metadata. This approach is not aligned with the evolving nature of language. Conventional methods for creating temporally adapted language models often depend on further pre-training static models on time-specific data. This paper presents a new approach: a series of point-in-time LLMs called Time Machine GPT (TiMaGPT), specifically designed to be nonprognosticative. This ensures they remain …

abstract arxiv cs.ce cs.cl cs.lg data datasets gpt language language models large language large language models llms machine metadata nature paper pre-training temporal text training type