Steering Conversational Large Language Models for Long Emotional Support Conversations

arXiv:2402.10453v1 Announce Type: new
Abstract: In this study, we address the challenge of consistently following emotional support strategies in long conversations by large language models (LLMs). We introduce the Strategy-Relevant Attention (SRA) metric, a model-agnostic measure designed to evaluate the effectiveness of LLMs in adhering to strategic prompts in emotional support contexts. By analyzing conversations within the Emotional Support Conversations dataset (ESConv) using LLaMA models, we demonstrate that SRA is significantly correlated with a model's ability to sustain the outlined …

abstract arxiv attention challenge conversational conversations cs.cl language language models large language large language models llms model-agnostic prompts strategies strategy study support type