Emergent Word Order Universals from Cognitively-Motivated Language Models

arXiv:2402.12363v1 Announce Type: new
Abstract: The world's languages exhibit certain so-called typological or implicational universals; for example, Subject-Object-Verb (SOV) word order typically employs postpositions. Explaining the source of such biases is a key goal in linguistics. We study the word-order universals through a computational simulation with language models (LMs). Our experiments show that typologically typical word orders tend to have lower perplexity estimated by LMs with cognitively plausible biases: syntactic biases, specific parsing strategies, and memory limitations. This suggests that …

abstract arxiv biases computational cs.cl example key language language models languages linguistics lms show simulation study through type word world