Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT

arXiv:2404.02403v1 Announce Type: cross
Abstract: This paper explores the efficacy of large language models (LLMs) for Persian. While ChatGPT and consequent LLMs have shown remarkable performance in English, their efficiency for more low-resource languages remains an open question. We present the first comprehensive benchmarking study of LLMs across diverse Persian language tasks. Our primary focus is on GPT-3.5-turbo, but we also include GPT-4 and OpenChat-3.5 to provide a more holistic evaluation. Our assessment encompasses a diverse set of tasks categorized …

abstract arxiv benchmarking chatgpt cs.cl cs.lg efficiency english language language models languages large language large language models llms low paper performance question study type