Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learning. (arXiv:2110.00926v2 [cs.LG] UPDATED)

Using information-theoretic principles, we consider the generalization error
(gen-error) of iterative semi-supervised learning (SSL) algorithms that
iteratively generate pseudo-labels for a large amount of unlabelled data to
progressively refine the model parameters. In contrast to most previous works
that {\em bound} the gen-error, we provide an {\em exact} expression for the
gen-error and particularize it to the binary Gaussian mixture model. Our
theoretical results suggest that when the class conditional variances are not
too large, the gen-error decreases with the …

arxiv information learning semi-supervised learning supervised learning