IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context

arXiv:2403.20147v1 Announce Type: new
Abstract: The pervasive influence of social biases in language data has sparked the need for benchmark datasets that capture and evaluate these biases in Large Language Models (LLMs). Existing efforts predominantly focus on English language and the Western context, leaving a void for a reliable dataset that encapsulates India's unique socio-cultural nuances. To bridge this gap, we introduce IndiBias, a comprehensive benchmarking dataset designed specifically for evaluating social biases in the Indian context. We filter and …

abstract arxiv benchmark biases context cs.cl data dataset datasets english english language focus indian influence language language data language models large language large language models llms social type