Concept -- An Evaluation Protocol on Conversation Recommender Systems with System- and User-centric Factors

arXiv:2404.03304v1 Announce Type: new
Abstract: The conversational recommendation system (CRS) has been criticized regarding its user experience in real-world scenarios, despite recent significant progress achieved in academia. Existing evaluation protocols for CRS may prioritize system-centric factors such as effectiveness and fluency in conversation while neglecting user-centric aspects. Thus, we propose a new and inclusive evaluation protocol, Concept, which integrates both system- and user-centric factors. We conceptualise three key characteristics in representing such factors and further divide them into six primary …

abstract academia arxiv concept conversation conversational cs.ai cs.cl evaluation experience progress protocol recommendation recommender systems systems type world