Anti-LM Decoding for Zero-shot In-context Machine Translation

arXiv:2311.08324v2 Announce Type: replace
Abstract: Zero-shot In-context learning is the phenomenon where models can perform the task simply given the instructions. However, pre-trained large language models are known to be poorly calibrated for this task. One of the most effective approaches to handling this bias is to adopt a contrastive decoding objective, which accounts for the prior probability of generating the next token by conditioning on some context. This work introduces an Anti-Language Model objective with a decay factor designed …

abstract arxiv bias context cs.ai cs.cl decoding however in-context learning language language models large language large language models machine machine translation translation type zero-shot