Go Beyond The Obvious: Probing the gap of INFORMAL reasoning ability between Humanity and LLMs by Detective Reasoning Puzzle Benchmark. (arXiv:2307.05113v2 [cs.CL] UPDATED)

Informal reasoning ability is the ability to reason based on common sense,
experience, and intuition.Humans use informal reasoning every day to extract
the most influential elements for their decision-making from a large amount of
life-like information.With the rapid development of language models, the
realization of general artificial intelligence has emerged with hope. Given the
outstanding informal reasoning ability of humans, how much informal reasoning
ability language models have has not been well studied by scholars.In order to
explore the gap …

arxiv benchmark beyond common sense decision experience extract gap humanity humans information intuition life llms making puzzle reason reasoning sense