Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuning

arXiv:2310.04357v3 Announce Type: replace-cross
Abstract: We employ random matrix theory to establish consistency of generalized cross validation (GCV) for estimating prediction risks of sketched ridge regression ensembles, enabling efficient and consistent tuning of regularization and sketching parameters. Our results hold for a broad class of asymptotically free sketches under very mild data assumptions. For squared prediction risk, we provide a decomposition into an unsketched equivalent implicit ridge bias and a sketching-based variance, and prove that the risk can be globally …

abstract arxiv class consistent enabling free generalized math.st matrix parameters prediction random regression regularization results ridge risks sketches stat.ml stat.th theory type validation