On the True Distribution Approximation of Minimum Bayes-Risk Decoding

arXiv:2404.00752v1 Announce Type: new
Abstract: Minimum Bayes-risk (MBR) decoding has recently gained renewed attention in text generation. MBR decoding considers texts sampled from a model as pseudo-references and selects the text with the highest similarity to the others. Therefore, sampling is one of the key elements of MBR decoding, and previous studies reported that the performance varies by sampling methods. From a theoretical standpoint, this performance variation is likely tied to how closely the samples approximate the true distribution of …

abstract approximation arxiv attention bayes cs.ai cs.cl decoding distribution key risk sampling text text generation the key true type