NORMAD: A Benchmark for Measuring the Cultural Adaptability of Large Language Models

arXiv:2404.12464v1 Announce Type: new
Abstract: The integration of Large Language Models (LLMs) into various global cultures fundamentally presents a cultural challenge: LLMs must navigate interactions, respect social norms, and avoid transgressing cultural boundaries. However, it is still unclear if LLMs can adapt their outputs to diverse cultural norms. Our study focuses on this aspect. We introduce NormAd, a novel dataset, which includes 2.6k stories that represent social and cultural norms from 75 countries, to assess the ability of LLMs to …

abstract adapt adaptability arxiv benchmark challenge cs.cl diverse global however integration interactions language language models large language large language models llms measuring social type