MacGyver: Are Large Language Models Creative Problem Solvers?

arXiv:2311.09682v2 Announce Type: replace
Abstract: We explore the creative problem-solving capabilities of modern LLMs in a novel constrained setting. To this end, we create MACGYVER, an automatically generated dataset consisting of over 1,600 real-world problems deliberately designed to trigger innovative usage of objects and necessitate out-of-the-box thinking. We then present our collection to both LLMs and humans to compare and contrast their problem-solving abilities. MACGYVER is challenging for both groups, but in unique and complementary ways. For instance, humans excel …

abstract arxiv box capabilities collection creative cs.ai cs.cl dataset explore generated language language models large language large language models llms modern novel objects problem-solving thinking type usage world