Feb. 12, 2024, 5:46 a.m. | Katharina Stein Daniel Fi\v{s}er J\"org Hoffmann Alexander Koller

cs.CL updates on arXiv.org arxiv.org

LLMs are being increasingly used for planning-style tasks, but their capabilities for planning and reasoning are poorly understood. We present AutoPlanBench, a novel method for automatically converting planning benchmarks written in PDDL into textual descriptions and offer a benchmark dataset created with our method. We show that while the best LLM planners do well on some planning tasks, others remain out of reach of current methods.

benchmark benchmarks capabilities cs.ai cs.cl dataset llm llms novel planning reasoning show style tasks textual

Doctoral Researcher (m/f/div) in Automated Processing of Bioimages

@ Leibniz Institute for Natural Product Research and Infection Biology (Leibniz-HKI) | Jena

Research Scholar (Technical Research)

@ Centre for the Governance of AI | Hybrid; Oxford, UK

Backend Spark Developer

@ Talan | Warsaw, Poland

Pricing & Data Management Intern

@ Novelis | Atlanta, GA, United States

Sr Data Engineer

@ Visa | Bengaluru, India

Customer Analytics / Data Science - Lead Analyst - Analytics US Timezone

@ dentsu international | Bengaluru, India