Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?

There is increasing interest in employing large language models (LLMs) as cognitive models. For such purposes, it is central to understand which cognitive properties are well-modeled by LLMs, and which are not. In this work, we study the biases of LLMs in relation to those known in children when solving arithmetic word problems. Surveying the learning science literature, we posit that the problem-solving process can be split into three distinct steps: text comprehension, solution planning and solution execution. We construct …

biases cognitive cognitive biases cs.ai cs.cl cs.lg human language language models large language large language models llms study work