Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator

arXiv:2403.08495v1 Announce Type: new
Abstract: Large Language Models (LLMs) have demonstrated remarkable proficiency in human interactions, yet their application within the medical field remains insufficiently explored. Previous works mainly focus on the performance of medical knowledge with examinations, which is far from the realistic scenarios, falling short in assessing the abilities of LLMs on clinical tasks. In the quest to enhance the application of Large Language Models (LLMs) in healthcare, this paper introduces the Automated Interactive Evaluation (AIE) framework and …

abstract application arxiv cs.cl evaluation focus human human interactions interactions interactive knowledge language language models large language large language models llms medical medical field patient performance simulator state type