A Literature Review and Framework for Human Evaluation of Generative Large Language Models in Healthcare

arXiv:2405.02559v1 Announce Type: new
Abstract: As generative artificial intelligence (AI), particularly Large Language Models (LLMs), continues to permeate healthcare, it remains crucial to supplement traditional automated evaluations with human expert evaluation. Understanding and evaluating the generated texts is vital for ensuring safety, reliability, and effectiveness. However, the cumbersome, time-consuming, and non-standardized nature of human evaluation presents significant obstacles to the widespread adoption of LLMs in practice. This study reviews existing literature on human evaluation methodologies for LLMs within healthcare. We …

abstract artificial artificial intelligence arxiv automated cs.ai cs.cl evaluation expert framework generated generative generative artificial intelligence healthcare human intelligence language language models large language large language models literature llms reliability review safety type understanding vital