Rationality Report Cards: Assessing the Economic Rationality of Large Language Models

arXiv:2402.09552v1 Announce Type: new
Abstract: There is increasing interest in using LLMs as decision-making "agents." Doing so includes many degrees of freedom: which model should be used; how should it be prompted; should it be asked to introspect, conduct chain-of-thought reasoning, etc? Settling these questions -- and more broadly, determining whether an LLM agent is reliable enough to be trusted -- requires a methodology for assessing such an agent's economic rationality. In this paper, we provide one. We begin by …

abstract agents arxiv cards cs.cl decision econ.gn economic etc freedom language language models large language large language models llms making q-fin.ec questions reasoning report thought type