Can LLMs Correct Physicians, Yet? Investigating Effective Interaction Methods in the Medical Domain

arXiv:2403.20288v1 Announce Type: new
Abstract: We explore the potential of Large Language Models (LLMs) to assist and potentially correct physicians in medical decision-making tasks. We evaluate several LLMs, including Meditron, Llama2, and Mistral, to analyze the ability of these models to interact effectively with physicians across different scenarios. We consider questions from PubMedQA and several tasks, ranging from binary (yes/no) responses to long answer generation, where the answer of the model is produced after an interaction with a physician. Our …

abstract analyze arxiv cs.ai cs.cl decision domain explore language language models large language large language models llama2 llms making medical mistral physicians tasks type