A Mathematical Theory for Learning Semantic Languages by Abstract Learners

arXiv:2404.07009v1 Announce Type: cross
Abstract: Recent advances in Large Language Models (LLMs) have demonstrated the emergence of capabilities (learned skills) when the number of system parameters and the size of training data surpass certain thresholds. The exact mechanisms behind such phenomena are not fully understood and remain a topic of active research. Inspired by the skill-text bipartite graph model presented in [1] for modeling semantic language, we develop a mathematical theory to explain the emergence of learned skills, taking the …

abstract advances arxiv capabilities cs.cl cs.it cs.lg data emergence language language models languages large language large language models llms math.it parameters semantic skills theory training training data type