Eliciting Better Multilingual Structured Reasoning from LLMs through Code

arXiv:2403.02567v1 Announce Type: new
Abstract: Development of large language models (LLM) have shown progress on reasoning, though studies have been limited to English or simple reasoning tasks. We thus introduce a multilingual structured reasoning and explanation dataset, termed xSTREET, that covers four tasks across six languages. xSTREET exposes a gap in base LLM performance between English and non-English reasoning tasks. We then propose two methods to remedy this gap, building on the insight that LLMs trained on code are better …

abstract arxiv code cs.ai cs.cl dataset development english gap language language models languages large language large language models llm llms multilingual progress reasoning simple six studies tasks through type