Towards a Systematic Approach to Design New Ensemble Learning Algorithms

Ensemble learning has been a focal point of machine learning research due to its potential to improve predictive performance. This study revisits the foundational work on ensemble error decomposition, historically confined to bias-variance-covariance analysis for regression problems since the 1990s. Recent advancements introduced a "unified theory of diversity," which proposes an innovative bias-variance-diversity decomposition framework. Leveraging this contemporary understanding, our research systematically explores the application of this decomposition to guide the creation of new ensemble learning algorithms. Focusing on regression …

algorithms analysis bias bias-variance covariance cs.lg design diversity ensemble error machine machine learning performance predictive regression research study theory variance work