Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruning. (arXiv:2207.02463v1 [cs.CL])

Language model debiasing has emerged as an important field of study in the
NLP community. Numerous debiasing techniques were proposed, but bias ablation
remains an unaddressed issue. We demonstrate a novel framework for inspecting
bias in pre-trained transformer-based language models via movement pruning.
Given a model and a debiasing objective, our framework finds a subset of the
model containing less bias than the original model. We implement our framework
by pruning the model while fine-tuning it on the debiasing objective. …

arxiv bias biases gender gender bias language language models pruning transformer