Tuning Language Models by Proxy

arXiv:2401.08565v2 Announce Type: replace
Abstract: Despite the general capabilities of large pretrained language models, they consistently benefit from further adaptation to better achieve desired behaviors. However, tuning these models has become increasingly resource-intensive, or impossible when model weights are private. We introduce proxy-tuning, a lightweight decoding-time algorithm that operates on top of black-box LMs to achieve the same end as direct tuning, but by accessing only its predictions over the output vocabulary, not its parameters. Our method tunes a smaller …

abstract algorithm arxiv become benefit box capabilities cs.cl decoding general however language language models lms type