BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English

arXiv:2403.10900v1 Announce Type: new
Abstract: In this study, we introduce BEnQA, a dataset comprising parallel Bengali and English exam questions for middle and high school levels in Bangladesh. Our dataset consists of approximately 5K questions covering several subjects in science with different types of questions, including factual, application, and reasoning-based questions. We benchmark several Large Language Models (LLMs) with our parallel dataset and observe a notable performance disparity between the models in Bengali and English. We also investigate some prompting …

abstract application arxiv bangladesh benchmark cs.cl dataset english exam question question answering questions reasoning school science study type types