Statistical inference with implicit SGD: proximal Robbins-Monro vs. Polyak-Ruppert. (arXiv:2206.12663v2 [stat.ML] UPDATED)

The implicit stochastic gradient descent (ISGD), a proximal version of SGD,
is gaining interest in the literature due to its stability over (explicit) SGD.
In this paper, we conduct an in-depth analysis of the two modes of ISGD for
smooth convex functions, namely proximal Robbins-Monro (proxRM) and proximal
Poylak-Ruppert (proxPR) procedures, for their use in statistical inference on
model parameters. Specifically, we derive non-asymptotic point estimation error
bounds of both proxRM and proxPR iterates and their limiting distributions, and
propose …

arxiv inference ml statistical