Everything We've Been Taught About ML is Wrong // Emmanuel Ameisen // LLMs in Prod Con Part 2

// Abstract
Machine learning has long been guided by a set of well-established principles around model design and evaluation. But recent progress in language models is challenging many of the rules that have shaped ML practice for years.

Powerful LLMs can now generate their own high-quality training data, evaluate themselves, and achieve strong performance in new domains. As a result, foundational rules of thumb like “never evaluate on the test set”, "don't use model generated data" and “start with the …

abstract design evaluation everything language language models llms machine machine learning model design part practice prod progress rules set