DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures

arXiv:2403.14651v1 Announce Type: cross
Abstract: Generative models are increasingly being used in various applications, such as text generation, commonsense reasoning, and question-answering. To be effective globally, these models must be aware of and account for local socio-cultural contexts, making it necessary to have benchmarks to evaluate the models for their cultural familiarity. Since the training data for LLMs is web-based and the Web is limited in its representation of information, it does not capture knowledge present within communities that are …

abstract applications arxiv benchmarks cs.cl cs.cy dataset generative generative models indian making question reasoning social text text generation type