Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution

arXiv:2405.00424v1 Announce Type: cross
Abstract: Ridge regression is an indispensable tool in big data econometrics but suffers from bias issues affecting both statistical efficiency and scalability. We introduce an iterative strategy to correct the bias effectively when the dimension $p$ is less than the sample size $n$. For $p>n$, our method optimally reduces the bias to a level unachievable through linear transformations of the response. We employ a Ridge-Screening (RS) method to handle the remaining bias when $p>n$, creating a …

abstract arxiv bias big big data data econ.em econometrics efficiency form inference iterative regression ridge sample scalability solution statistical stat.me stat.ml strategy tool type