GreedLlama: Performance of Financial Value-Aligned Large Language Models in Moral Reasoning

arXiv:2404.02934v1 Announce Type: cross
Abstract: This paper investigates the ethical implications of aligning Large Language Models (LLMs) with financial optimization, through the case study of GreedLlama, a model fine-tuned to prioritize economically beneficial outcomes. By comparing GreedLlama's performance in moral reasoning tasks to a base Llama2 model, our results highlight a concerning trend: GreedLlama demonstrates a marked preference for profit over ethical considerations, making morally appropriate decisions at significantly lower rates than the base model in scenarios of both low …

abstract arxiv case case study cs.ai cs.cl cs.cy cs.lg ethical ethical implications financial language language models large language large language models llama2 llms optimization paper performance reasoning results s performance study tasks through type value