When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment. (arXiv:2210.01478v2 [cs.CL] UPDATED)

AI systems are becoming increasingly intertwined with human life. In order to
effectively collaborate with humans and ensure safety, AI systems need to be
able to understand, interpret and predict human moral judgments and decisions.
Human moral judgments are often guided by rules, but not always. A central
challenge for AI safety is capturing the flexibility of the human moral mind --
the ability to determine when a rule should be broken, especially in novel or
unusual situations. In this …

arxiv human judgment language language models