GPT-4, 128K context - it is not big enough

OpenAI has recently introduced the GPT-4 Turbo model with a 128K context window. A 4x boost to the previous maximum of 32K in GPT-4.

Did you know that this context size is enough to hold 1684 tweets or 123 StackOverflow questions? Or that the largest source file from the Linux kernel would require a context window that is 540 times larger than 128k? Hold on for more comparisons.

I've taken a few artefacts, converted them to a textual representation, …

128k context big boost chatgpt context context window gpt gpt-4 gpt4 kernel linux llm openai questions stackoverflow turbo tweets