The Statistics Behind Random Forests

A peek into the inner workings of randomized ensembles

There is something about random forests that is deeply unique.

Our usual variance reduction techniques — like limiting the depth of a decision tree, or adding a norm penalty to a logistic regression — work by penalizing the complexity of a single hypothesis. The variance reduction in a random forest, however, comes from simultaneously exploring multiple competing hypotheses.

Note the …

data science editors pick ensemble-learning random random-forest random forests statistics