Hypothesis Generation with Large Language Models

arXiv:2404.04326v1 Announce Type: cross
Abstract: Effective generation of novel hypotheses is instrumental to scientific progress. So far, researchers have been the main powerhouse behind hypothesis generation by painstaking data analysis and thinking (also known as the Eureka moment). In this paper, we examine the potential of large language models (LLMs) to generate hypotheses. We focus on hypothesis generation based on data (i.e., labeled examples). To enable LLMs to handle arbitrarily long contexts, we generate initial hypotheses from a small number …

abstract analysis arxiv cs.ai cs.cl cs.cy cs.lg data data analysis eureka generate hypothesis language language models large language large language models llms moment novel paper progress researchers scientific thinking type