Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis

arXiv:2305.14877v2 Announce Type: replace
Abstract: Previous works in prompt engineering for large language models have introduced different gradient-free probability-based prompt selection methods that aim to choose the optimal prompt among the candidates for a given task but have failed to provide a comprehensive and fair comparison between each other. In this paper, we propose a unified framework to interpret and evaluate the existing probability-based prompt selection methods by performing extensive experiments on 13 common and diverse NLP tasks. We find …

abstract aim analysis and analysis arxiv comparison cs.cl engineering evaluation fair free gradient language language models large language large language models probability prompt through type