[D] LLMs are known for catastrophic forgetting during continual fine-tuning

But how is Chatgpt-4 able to remember all the factual data that it learned?

In other words, how can LLMs remember the data that they learned in the initial training batches (in both, during pre-training and continual fine-tuning)?

catastrophic forgetting chatgpt chatgpt-4 continual data fine-tuning llms machinelearning pre-training training words