What Are We Optimizing For? A Human-centric Evaluation of Deep Learning-based Movie Recommenders

arXiv:2401.11632v2 Announce Type: replace-cross
Abstract: In the past decade, deep learning (DL) models have gained prominence for their exceptional accuracy on benchmark datasets in recommender systems (RecSys). However, their evaluation has primarily relied on offline metrics, overlooking direct user perception and experience. To address this gap, we conduct a human-centric evaluation case study of four leading DL-RecSys models in the movie domain. We test how different DL-RecSys models perform in personalized recommendation generation by conducting survey study with 445 real …

abstract accuracy arxiv benchmark cs.hc cs.ir cs.lg datasets deep learning evaluation experience gap however human human-centric metrics movie movıe offline perception recommenders recommender systems recsys systems type