SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes

arXiv:2403.05696v1 Announce Type: cross
Abstract: While generative multilingual models are rapidly being deployed, their safety and fairness evaluations are largely limited to resources collected in English. This is especially problematic for evaluations targeting inherently socio-cultural phenomena such as stereotyping, where it is important to build multi-lingual resources that reflect the stereotypes prevalent in respective language communities. However, gathering these resources, at scale, in varied languages and regions pose a significant challenge as it requires broad socio-cultural knowledge and can also …

abstract arxiv build cs.cl cs.cv dataset english fairness generative geo multilingual multilingual models resources safety stereotypes targeting type