Do Large Language Models learn world models or just surface statistics?

A mystery

Large Language Models (LLM) are on fire, capturing public attention by their ability to provide seemingly impressive completions to user prompts (NYT coverage). They are a delicate combination of a radically simplistic algorithm with massive amounts of data and computing power. They are trained by playing a guess-the-next-word

algorithm attention combination computing computing power data fire language language models large language models learn llm massive next playing power prompts public statistics word world world models