Parameter Efficient Diff Pruning for Bias Mitigation. (arXiv:2205.15171v2 [cs.LG] UPDATED)

In recent years language models have achieved state of the art performance on
a wide variety of natural language processing tasks. As these models are
continuously growing in size it becomes increasingly important to explore
methods to make them more storage efficient. At the same time their increase
cognitive abilities increase the danger that societal bias existing in datasets
are implicitly encoded in the model weights. We propose an architecture which
deals with these two challenges at the same time …

arxiv bias diff lg pruning