PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations. (arXiv:2110.15137v3 [cs.LG] UPDATED)

Considering a probability distribution over parameters is known as an
efficient strategy to learn a neural network with non-differentiable activation
functions. We study the expectation of a probabilistic neural network as a
predictor by itself, focusing on the aggregation of binary activated neural
networks with normal distributions over real-valued weights. Our work leverages
a recent analysis derived from the PAC-Bayesian framework that derives tight
generalization bounds and learning procedures for the expected output value of
such an aggregation, which is …

aggregation analysis arxiv bayesian binary distribution framework learn network networks neural network neural networks normal probability strategy study value work