DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generation

arXiv:2311.09581v2 Announce Type: replace
Abstract: Medical text generation aims to assist with administrative work and highlight salient information to support decision-making. To reflect the specific requirements of medical text, in this paper, we propose a set of metrics to evaluate the completeness, conciseness, and attribution of the generated text at a fine-grained level. The metrics can be computed by various types of evaluators including instruction-following (both proprietary and open-source) and supervised entailment models. We demonstrate the effectiveness of the resulting …

abstract arxiv attribution cs.cl decision evaluation fine-grained generated highlight information making medical metrics paper requirements set support text text generation type work