Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression

arXiv:2302.09727v2 Announce Type: replace-cross
Abstract: Stochastic gradient descent (SGD) has emerged as the quintessential method in a data scientist's toolbox. Using SGD for high-stakes applications requires, however, careful quantification of the associated uncertainty. Towards that end, in this work, we establish a high-dimensional Central Limit Theorem (CLT) for linear functionals of online SGD iterates for overparametrized least-squares regression with non-isotropic Gaussian inputs. Our result shows that a CLT holds even when the dimensionality is of order exponential in the number …

abstract applications arxiv data data scientist gradient however inference linear linear regression math.oc math.pr math.st quantification regression statistical stat.ml stat.th stochastic theorem type uncertainty work