MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation

arXiv:2310.03302v2 Announce Type: replace
Abstract: A central aspect of machine learning research is experimentation, the process of designing and running experiments, analyzing the results, and iterating towards some positive outcome (e.g., improving accuracy). Could agents driven by powerful language models perform machine learning experimentation effectively? To answer this question, we introduce MLAgentBench, a suite of 13 tasks ranging from improving model performance on CIFAR-10 to recent research problems like BabyLM. For each task, an agent can perform actions like reading/writing …

agents arxiv cs.ai cs.lg experimentation language machine machine learning type