Assessing Logical Puzzle Solving in Large Language Models: Insights from a Minesweeper Case Study

arXiv:2311.07387v2 Announce Type: replace
Abstract: Large Language Models (LLMs) have shown remarkable proficiency in language understanding and have been successfully applied to a variety of real-world tasks through task-specific fine-tuning or prompt engineering. Despite these advancements, it remains an open question whether LLMs are fundamentally capable of reasoning and planning, or if they primarily rely on recalling and synthesizing information from their training data. In our research, we introduce a novel task -- Minesweeper -- specifically designed in a format …

abstract arxiv case case study cs.cl engineering fine-tuning insights language language models language understanding large language large language models llms prompt puzzle question study tasks through type understanding world