Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features

We investigate the test risk of continuous-time stochastic gradient flow dynamics in learning theory. Using a path integral formulation we provide, in the regime of a small learning rate, a general formula for computing the difference between test risk curves of pure gradient and stochastic gradient flows. We apply the general theory to a simple model of weak features, which displays the double descent phenomenon, and explicitly compute the corrections brought about by the added stochastic term in the dynamics, …

computing cond-mat.dis-nn continuous cs.lg difference dynamics features flow general gradient integral path rate risk small solution stat.ml stochastic test theory