LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law

Pretrained large language models (LLMs) are surprisingly effective at performing zero-shot tasks, including time-series forecasting. However, understanding the mechanisms behind such capabilities remains highly challenging due to the complexity of the models. In this paper, we study LLMs' ability to extrapolate the behavior of dynamical systems whose evolution is governed by principles of physical interest. Our results show that LLaMA 2, a language model trained primarily on texts, achieves accurate predictions of dynamical system time series without fine-tuning or prompt …

behavior capabilities complexity context cs.ai cs.lg forecasting language language models large language large language models law learn llms paper scaling scaling law series study systems tasks understanding zero-shot