ConceptPsy:A Benchmark Suite with Conceptual Comprehensiveness in Psychology

arXiv:2311.09861v3 Announce Type: replace
Abstract: The critical field of psychology necessitates a comprehensive benchmark to enhance the evaluation and development of domain-specific Large Language Models (LLMs). Existing MMLU-type benchmarks, such as C-EVAL and CMMLU, include psychology-related subjects, but their limited number of questions and lack of systematic concept sampling strategies mean they cannot cover the concepts required in psychology. Consequently, despite their broad subject coverage, these benchmarks lack the necessary depth in the psychology domain, making them inadequate as psychology-specific …

abstract arxiv benchmark benchmarks concept cs.ai cs.cl development domain evaluation language language models large language large language models llms mean mmlu psychology questions replace sampling strategies type