Uncertainty quantification for iterative algorithms in linear models with application to early stopping

arXiv:2404.17856v1 Announce Type: cross
Abstract: This paper investigates the iterates $\hbb^1,\dots,\hbb^T$ obtained from iterative algorithms in high-dimensional linear regression problems, in the regime where the feature dimension $p$ is comparable with the sample size $n$, i.e., $p \asymp n$. The analysis and proposed estimators are applicable to Gradient Descent (GD), proximal GD and their accelerated variants such as Fast Iterative Soft-Thresholding (FISTA). The paper proposes novel estimators for the generalization error of the iterate $\hbb^t$ for any fixed iteration $t$ …

abstract algorithms analysis application arxiv cs.lg feature iterative linear linear regression math.st paper quantification regression sample stat.co stat.me stat.ml stat.th type uncertainty