Sampling with Attribute-Related Information for Controlling Language Models. (arXiv:2205.06036v1 [cs.CL])

Web: http://arxiv.org/abs/2205.06036

The dominant approaches for controlling language models are based on
fine-tuning large language models or prompt engineering. However, these methods
often require condition-specific data or considerable hand-crafting. We propose
a new simple guided decoding method, Gamma Sampling, which does not require
complex engineering and any extra data. Gamma Sampling introduces
attribute-related information (provided by humans or language models
themselves) into the sampling process to guide language models to generate
texts with desired attributes. Experiments on controlling topics and sentiments
of …

arxiv information language language models models sampling