Rethinking Fano's Inequality in Ensemble Learning. (arXiv:2205.12683v1 [cs.LG])

We propose a fundamental theory on ensemble learning that evaluates a given
ensemble system by a well-grounded set of metrics. Previous studies used a
variant of Fano's inequality of information theory and derived a lower bound of
the classification error rate on the basis of the accuracy and diversity of
models. We revisit the original Fano's inequality and argue that the studies
did not take into account the information lost when multiple model predictions
are combined into a final prediction. …

arxiv ensemble inequality learning