Why Longer Context Isn't Enough

Recent advances in large language models (LLMs) have centered around more data, larger models, and larger context lengths. The ability of LLMs to learn in-context (i.e. in-context learning) makes longer context lengths extremely valuable. However, there are some problems with relying on just in-context learning to learn during inference.

Outline
0:00 - Intro

Social Media
YouTube - https://youtube.com/c/EdanMeyer
Twitter - https://twitter.com/ejmejm1

advances context context learning data however in-context learning inference intro isn language language models large language large language models larger models learn llms media social social media