Benefits of Additive Noise in Composing Classes with Bounded Capacity. (arXiv:2206.07199v1 [stat.ML])

We observe that given two (compatible) classes of functions $\mathcal{F}$ and
$\mathcal{H}$ with small capacity as measured by their uniform covering
numbers, the capacity of the composition class $\mathcal{H} \circ \mathcal{F}$
can become prohibitively large or even unbounded. We then show that adding a
small amount of Gaussian noise to the output of $\mathcal{F}$ before composing
it with $\mathcal{H}$ can effectively control the capacity of $\mathcal{H}
\circ \mathcal{F}$, offering a general recipe for modular design. To prove our
results, we …

arxiv benefits capacity ml noise