Gradient-based Discrete Sampling with Automatic Cyclical Scheduling

arXiv:2402.17699v1 Announce Type: new
Abstract: Discrete distributions, particularly in high-dimensional deep models, are often highly multimodal due to inherent discontinuities. While gradient-based discrete sampling has proven effective, it is susceptible to becoming trapped in local modes due to the gradient information. To tackle this challenge, we propose an automatic cyclical scheduling, designed for efficient and accurate sampling in multimodal discrete distributions. Our method contains three key components: (1) a cyclical step size schedule where large steps discover new modes and …

abstract arxiv challenge cs.lg gradient information multimodal sampling scheduling stat.ml type