Black box tests for algorithmic stability. (arXiv:2111.15546v4 [cs.LG] UPDATED)

Algorithmic stability is a concept from learning theory that expresses the
degree to which changes to the input data (e.g., removal of a single data
point) may affect the outputs of a regression algorithm. Knowing an algorithm's
stability properties is often useful for many downstream applications -- for
example, stability is known to lead to desirable generalization properties and
predictive inference guarantees. However, many modern algorithms currently used
in practice are too complex for a theoretical analysis of their stability …

arxiv black box tests