Causal ATE Mitigates Unintended Bias in Controlled Text Generation

arXiv:2311.11229v2 Announce Type: replace
Abstract: We study attribute control in language models through the method of Causal Average Treatment Effect (Causal ATE). Existing methods for the attribute control task in Language Models (LMs) check for the co-occurrence of words in a sentence with the attribute of interest, and control for them. However, spurious correlation of the words with the attribute in the training dataset, can cause models to hallucinate the presence of the attribute when presented with the spurious correlate …

arxiv bias controlled text generation cs.cl text text generation type