Bayesian Additive Regression Networks

arXiv:2404.04425v1 Announce Type: cross
Abstract: We apply Bayesian Additive Regression Tree (BART) principles to training an ensemble of small neural networks for regression tasks. Using Markov Chain Monte Carlo, we sample from the posterior distribution of neural networks that have a single hidden layer. To create an ensemble of these, we apply Gibbs sampling to update each network against the residual target value (i.e. subtracting the effect of the other networks). We demonstrate the effectiveness of this technique on several …

abstract apply arxiv bart bayesian cs.lg distribution ensemble gibbs hidden layer markov networks neural networks posterior regression sample sampling small stat.ml tasks training tree type update