ANLS* -- A Universal Document Processing Metric for Generative Large Language Models

Traditionally, discriminative models have been the predominant choice for tasks like document classification and information extraction. These models make predictions that fall into a limited number of predefined classes, facilitating a binary true or false evaluation and enabling the direct calculation of metrics such as the F1 score. However, recent advancements in generative large language models (GLLMs) have prompted a shift in the field due to their enhanced zero-shot capabilities, which eliminate the need for a downstream dataset and computationally …

binary classification cs.ai cs.cl discriminative models document document processing enabling evaluation extraction false generative information information extraction language language models large language large language models metrics predictions processing tasks true