Entropy-MCMC: Sampling from Flat Basins with Ease

Bayesian deep learning counts on the quality of posterior distribution estimation. However, the posterior of deep neural networks is highly multi-modal in nature, with local modes exhibiting varying generalization performance. Given a practical budget, targeting at the original posterior can lead to suboptimal performance, as some samples may become trapped in "bad" modes and suffer from overfitting. Leveraging the observation that "good" modes with low generalization error often reside in flat basins of the energy landscape, we propose to bias …

bayesian bayesian deep learning become budget cs.lg deep learning distribution entropy mcmc modal multi-modal nature networks neural networks performance posterior practical quality samples sampling stat.ml targeting