Perceptual Quality-based Model Training under Annotator Label Uncertainty

arXiv:2403.10190v1 Announce Type: cross
Abstract: Annotators exhibit disagreement during data labeling, which can be termed as annotator label uncertainty. Annotator label uncertainty manifests in variations of labeling quality. Training with a single low-quality annotation per sample induces model reliability degradations. In this work, we first examine the effects of annotator label uncertainty in terms of the model's generalizability and prediction uncertainty. We observe that the model's generalizability and prediction uncertainty degrade with the presence of low-quality noisy labels. Meanwhile, our …

abstract annotation arxiv cs.ai cs.cv cs.lg data data labeling effects labeling low per quality reliability sample training type uncertainty work