Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests

arXiv:2402.12668v1 Announce Type: cross
Abstract: We study the often overlooked phenomenon, first noted in \cite{breiman2001random}, that random forests appear to reduce bias compared to bagging. Motivated by an interesting paper by \cite{mentch2020randomization}, where the authors argue that random forests reduce effective degrees of freedom and only outperform bagging ensembles in low signal-to-noise ratio (SNR) settings, we explore how random forests can uncover patterns in the data missed by bagging. We empirically demonstrate that in the presence of such patterns, random …

abstract arxiv authors bias case case study cs.lg forests freedom paper random random forests randomization reduce stat.ml study type variance