[R] A small bias in CART decision trees and random forests

Decision trees and consequently random forests are invariant to the scaling of attributes. Interestingly, they are not invariant to the mirroring of the attributes (i.e. multiplying by -1). To be precise, if there are features which are likely to take values coinciding with thresholds in binary CART trees, then the mirroring of the feature leads to a bias in inference time. It is not a big bias, but it can reach about 0.1-0.2 percentage points of r2 and AUC.

The …

bias binary cart decision decision trees features forests machinelearning random random forests scaling small trees values