Can Large Language Models put 2 and 2 together? Probing for Entailed Arithmetical Relationships

arXiv:2404.19432v1 Announce Type: new
Abstract: Two major areas of interest in the era of Large Language Models regard questions of what do LLMs know, and if and how they may be able to reason (or rather, approximately reason). Since to date these lines of work progressed largely in parallel (with notable exceptions), we are interested in investigating the intersection: probing for reasoning about the implicitly-held knowledge. Suspecting the performance to be lacking in this area, we use a very simple …

abstract arxiv cs.ai cs.cl language language models large language large language models llms major questions reason regard relationships together type work