From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings

arXiv:2402.11512v1 Announce Type: new
Abstract: Embeddings play a pivotal role in the efficacy of Large Language Models. They are the bedrock on which these models grasp contextual relationships and foster a more nuanced understanding of language and consequently perform remarkably on a plethora of complex tasks that require a fundamental understanding of human language. Given that these embeddings themselves often reflect or exhibit bias, it stands to reason that these models may also inadvertently learn this bias. In this work, …

abstract arxiv bedrock cs.cl cs.cy embeddings language language model language models large language large language model large language models pivotal relationships role tasks type understanding word word embeddings