Approaching Human-Level Forecasting with Language Models

arXiv:2402.18563v1 Announce Type: new
Abstract: Forecasting future events is important for policy and decision making. In this work, we study whether language models (LMs) can forecast at the level of competitive human forecasters. Towards this goal, we develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions. To facilitate our study, we collect a large dataset of questions from competitive forecasting platforms. Under a test set published after the knowledge cut-offs of our …

abstract arxiv cs.ai cs.cl cs.ir cs.lg decision decision making events forecast forecasting future generate human information language language models lms making policy predictions retrieval retrieval-augmented search study type work