Strings from the Library of Babel: Random Sampling as a Strong Baseline for Prompt Optimisation

arXiv:2311.09569v2 Announce Type: replace
Abstract: Recent prompt optimisation approaches use the generative nature of language models to produce prompts -- even rivaling the performance of human-curated prompts. In this paper, we demonstrate that randomly sampling tokens from the model vocabulary as ``separators'' can be as effective as language models for prompt-style text classification. Our experiments show that random separators are competitive baselines, having less than a 1% difference compared to previous self-optimisation methods and showing a 12% average relative improvement …

abstract arxiv cs.ai cs.cl generative human language language models library nature optimisation paper performance prompt prompts random sampling strings tokens type