The Unreasonable Effectiveness of Eccentric Automatic Prompts

arXiv:2402.10949v1 Announce Type: cross
Abstract: Large Language Models (LLMs) have demonstrated remarkable problem-solving and basic mathematics abilities. However, their efficacy is highly contingent on the formulation of the prompt. This study endeavors to quantify the influence of incorporating "positive thinking" into the system message of the prompt, then compare that to systematic prompt optimization. We assess the performance of 60 combinations of system message snippets, tested with and without Chain of Thought prompting, across three models with parameters ranging from …

abstract arxiv basic cs.ai cs.cl cs.lg influence language language models large language large language models llms mathematics positive problem-solving prompt prompts study the prompt thinking type