(Ir)rationality and Cognitive Biases in Large Language Models

arXiv:2402.09193v1 Announce Type: new
Abstract: Do large language models (LLMs) display rational reasoning? LLMs have been shown to contain human biases due to the data they have been trained on; whether this is reflected in rational reasoning remains less clear. In this paper, we answer this question by evaluating seven language models using tasks from the cognitive psychology literature. We find that, like humans, LLMs display irrationality in these tasks. However, the way this irrationality is displayed does not reflect …

abstract arxiv biases clear cognitive cognitive biases cs.ai cs.cl cs.hc data human language language models large language large language models llms paper question reasoning type