Tuning diagonal scale matrices for HMC

arXiv:2403.07495v1 Announce Type: cross
Abstract: Three approaches for adaptively tuning diagonal scale matrices for HMC are discussed and compared. The common practice of scaling according to estimated marginal standard deviations is taken as a benchmark. Scaling according to the mean log-target gradient (ISG), and a scaling method targeting that the frequency of when the underlying Hamiltonian dynamics crosses the respective medians should be uniform across dimensions, are taken as alternatives. Numerical studies suggest that the ISG method leads in many …

abstract arxiv benchmark gradient mean practice scale scaling standard stat.co stat.me stat.ml targeting type