all AI news
Do Large Language Models Solve ARC Visual Analogies Like People Do?
March 18, 2024, 4:47 a.m. | Gustaw Opie{\l}ka, Hannes Rosenbusch, Veerle Vijverberg, Claire E. Stevenson
cs.CL updates on arXiv.org arxiv.org
Abstract: The Abstraction Reasoning Corpus (ARC) is a visual analogical reasoning test designed for humans and machines (Chollet, 2019). We compared human and large language model (LLM) performance on a new child-friendly set of ARC items. Results show that both children and adults outperform most LLMs on these tasks. Error analysis revealed a similar "fallback" solution strategy in LLMs and young children, where part of the analogy is simply copied. In addition, we found two other …
abstract abstraction arc arxiv child children cs.ai cs.cl human humans language language model language models large language large language model large language models llm machines people performance reasoning results set show solve test type visual
More from arxiv.org / cs.CL updates on arXiv.org
Benchmarking LLMs via Uncertainty Quantification
1 day, 15 hours ago |
arxiv.org
CARE: Extracting Experimental Findings From Clinical Literature
1 day, 15 hours ago |
arxiv.org
Jobs in AI, ML, Big Data
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
Intern Large Language Models Planning (f/m/x)
@ BMW Group | Munich, DE
Data Engineer Analytics
@ Meta | Menlo Park, CA | Remote, US