Pearls from Pebbles: Improved Confidence Functions for Auto-labeling

arXiv:2404.16188v1 Announce Type: new
Abstract: Auto-labeling is an important family of techniques that produce labeled training sets with minimum manual labeling. A prominent variant, threshold-based auto-labeling (TBAL), works by finding a threshold on a model's confidence scores above which it can accurately label unlabeled data points. However, many models are known to produce overconfident scores, leading to poor TBAL performance. While a natural idea is to apply off-the-shelf calibration methods to alleviate the overconfidence issue, such methods still fall short. …

abstract arxiv auto confidence cs.ai cs.lg data family functions however labeling minimum stat.ml threshold training type