[D] LLMs: intelligence or data leakage?

Hey everyone, ML Engineer here, I've trained LLMs in real life on a decent amount of data from scratch and I have a point here that I'd like to discuss with you.

As you know LLMs are trained at very big scales both in terms of model size and data.

As you know from ML basics, we want to avoid data leakage: the test set must NOT see samples from the training set. That would be cheating.

Someday, maybe if …

big data data leakage discuss engineer hey intelligence life llms machinelearning ml engineer terms