Continual-T0: Progressively Instructing 50+ Tasks to Language Models Without Forgetting. (arXiv:2205.12393v1 [cs.CL])

Recent work on large language models relies on the intuition that most
natural language processing tasks can be described via natural language
instructions. Language models trained on these instructions show strong
zero-shot performance on several standard datasets. However, these models even
though impressive still perform poorly on a wide range of tasks outside of
their respective training and evaluation sets. To address this limitation, we
argue that a model should be able to keep extending its knowledge and
abilities, without …

arxiv continual language language models