Assessing Generalization of SGD via Disagreement

The variable \(y\) refers to the average generalization error of the two models and the variable \(x\) refers to the disagreement of the two models. Estimating the generalization error of a model — how well the model performs on unseen data — is a fundamental component in any machine learning system. Generalization performance is traditionally estimated in a supervised manner, by dividing the labeled data into a training set and test set. However, high-quality labels are usually costly and, ideally, …

machine learning