View article

Ensemble methods that train and combine multiple learners have always been among the state-of-the-art learning methods, and ensemble pruning aims at generating a smaller-sized ensemble with even better generalization performance. Abundant ensemble pruning methods that use evaluation criteria such as diversity or margin together with validation error have been proposed. However, as these evaluation criteria are used together with the validation error, their effect on generalization performance is less clear. In this paper, we propose a margin distribution and structural diversity guided ensemble pruning framework, called Decoupled Ensemble Pruning (DEP). It decouples the optimization of margin distribution and structural diversity and the optimization of validation error into two stages. Our information-theoretic analysis reveals that the expected generalization gap is related to the combination distribution, i …