Procedural Dilemma Generation for Evaluating Moral Reasoning in Humans and Language Models

arXiv:2404.10975v1 Announce Type: new
Abstract: As AI systems like language models are increasingly integrated into decision-making processes affecting people's lives, it's critical to ensure that these systems have sound moral reasoning. To test whether they do, we need to develop systematic evaluations. We provide a framework that uses a language model to translate causal graphs that capture key aspects of moral dilemmas into prompt templates. With this framework, we procedurally generated a large and diverse set of moral dilemmas -- …

abstract ai systems arxiv cs.cl decision framework humans language language models making people processes reasoning sound systems test type