Large Language Models As Evolution Strategies

arXiv:2402.18381v1 Announce Type: cross
Abstract: Large Transformer models are capable of implementing a plethora of so-called in-context learning algorithms. These include gradient descent, classification, sequence completion, transformation, and improvement. In this work, we investigate whether large language models (LLMs), which never explicitly encountered the task of black-box optimization, are in principle capable of implementing evolutionary optimization algorithms. While previous works have solely focused on language-based task specification, we move forward and focus on the zero-shot application of LLMs to black-box …

abstract algorithms arxiv box classification context cs.ai cs.lg cs.ne evolution gradient improvement in-context learning language language models large language large language models llms optimization strategies transformation transformer transformer models type work