EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling

arXiv:2403.14541v1 Announce Type: new
Abstract: Recently, Large Language Models (LLMs) have demonstrated outstanding performance across a wide range of downstream language tasks. Temperature sampling is a commonly used decoding strategy for LLMs' generation process. However, a fixed temperature parameter is used in most cases, which may not always be an optimal choice for balancing generation quality and diversity. In this paper, we propose an effective Entropy-based Dynamic Temperature (EDT) Sampling method, to achieve a more balanced performance in terms of …

abstract arxiv cases cs.cl decoding dynamic entropy however improving language language models large language large language models llms performance process sampling strategy tasks type