Abstract by Mike Brodie
Optimal Dieting: Improving Diversity in Multiple Choice Learning Ensembles
Many tasks in computer vision, natural language processing, and computational biology require end users to select the best option from multiple possible solutions. Recent work in this area seeks to produce more diverse sets of possible solutions using model ensembles. These existing approaches often lead to 'alpha-model' ensemble domination, which weakens overall ensemble performance. We introduce a multinomial pooling layer for neural network outputs that improves instance sharing during the initial stages of ensemble training. We also introduce a novel, inexpensive loss term that discourages alpha-model domination and improves ensemble diversity. We demonstrate empirically that our contributions yield greater ensemble model diversity and improved performance on image classification, segmentation, and captioning tasks.