Debiased Pseudo Labeling in Self-Training. (arXiv:2202.07136v1 [cs.LG])

Deep neural networks achieve remarkable performances on a wide range of tasks
with the aid of large-scale labeled datasets. However, large-scale annotations
are time-consuming and labor-exhaustive to obtain on realistic tasks. To
mitigate the requirement for labeled data, self-training is widely used in both
academia and industry by pseudo labeling on readily-available unlabeled data.
Despite its popularity, pseudo labeling is well-believed to be unreliable and
often leads to training instability. Our experimental studies further reveal
that the performance of self-training …

arxiv labeling training