Your Model Is Not Predicting Depression Well And That Is Why: A Case Study of PRIMATE Dataset

arXiv:2403.00438v1 Announce Type: new
Abstract: This paper addresses the quality of annotations in mental health datasets used for NLP-based depression level estimation from social media texts. While previous research relies on social media-based datasets annotated with binary categories, i.e. depressed or non-depressed, recent datasets such as D2S and PRIMATE aim for nuanced annotations using PHQ-9 symptoms. However, most of these datasets rely on crowd workers without the domain knowledge for annotation. Focusing on the PRIMATE dataset, our study reveals concerns …

abstract annotations arxiv binary case case study cs.cl dataset datasets depression health media mental health nlp paper quality research social social media study type