Adaptive Data Analysis for Growing Data

arXiv:2405.13375v1 Announce Type: new
Abstract: Reuse of data in adaptive workflows poses challenges regarding overfitting and the statistical validity of results. Previous work has demonstrated that interacting with data via differentially private algorithms can mitigate overfitting, achieving worst-case generalization guarantees with asymptotically optimal data requirements. However, such past work assumes data is static and cannot accommodate situations where data grows over time. In this paper we address this gap, presenting the first generalization bounds for adaptive analysis in the dynamic …

abstract algorithms analysis arxiv case challenges cs.lg data data analysis however overfitting requirements results statistical stat.ml type via work workflows