Debiasing Algorithm through Model Adaptation

arXiv:2310.18913v3 Announce Type: replace-cross
Abstract: Large language models are becoming the go-to solution for the ever-growing number of tasks. However, with growing capacity, models are prone to rely on spurious correlations stemming from biases and stereotypes present in the training data. This work proposes a novel method for detecting and mitigating gender bias in language models. We perform causal analysis to identify problematic model components and discover that mid-upper feed-forward layers are most prone to convey bias. Based on the …

abstract algorithm arxiv bias biases capacity correlations cs.ai cs.cl data gender gender bias however language language models large language large language models model adaptation novel solution stat.ml stemming stereotypes tasks through training training data type work