In-Context Learning Dynamics with Random Binary Sequences

arXiv:2310.17639v3 Announce Type: replace-cross
Abstract: Large language models (LLMs) trained on huge corpora of text datasets demonstrate intriguing capabilities, achieving state-of-the-art performance on tasks they were not explicitly trained for. The precise nature of LLM capabilities is often mysterious, and different prompts can elicit different capabilities through in-context learning. We propose a framework that enables us to analyze in-context learning dynamics to understand latent concepts underlying LLMs' behavioral patterns. This provides a more nuanced understanding than success-or-failure evaluation benchmarks, but …

abstract art arxiv binary capabilities context cs.ai cs.cl cs.lg datasets dynamics framework in-context learning language language models large language large language models llm llms nature performance prompts random state tasks text through type