Linguistic Structure Induction from Language Models

arXiv:2403.09714v1 Announce Type: new
Abstract: Linear sequences of words are implicitly represented in our brains by hierarchical structures that organize the composition of words in sentences. Linguists formalize different frameworks to model this hierarchy; two of the most common syntactic frameworks are Constituency and Dependency. Constituency represents sentences as nested groups of phrases, while dependency represents a sentence by assigning relations between its words. Recently, the pursuit of intelligent machines has produced Language Models (LMs) capable of solving many language …

abstract arxiv brains cs.ai cs.cl frameworks hierarchical language language models linear organize type words