Controlling Forgetting with Test-Time Data in Continual Learning

arXiv:2406.13653v1 Announce Type: new
Abstract: Foundational vision-language models have shown impressive performance on various downstream tasks. Yet, there is still a pressing need to update these models later as new tasks or domains become available. Ongoing Continual Learning (CL) research provides techniques to overcome catastrophic forgetting of previous information when new knowledge is acquired. To date, CL techniques focus only on the supervised training sessions. This results in significant forgetting yielding inferior performance to even the prior model zero shot …

abstract arxiv become catastrophic forgetting continual cs.lg data domains foundational information knowledge language language models later performance research tasks test time data type update vision vision-language vision-language models