Decorrelating neurons using persistence. (arXiv:2308.04870v1 [cs.LG])

We propose a novel way to improve the generalisation capacity of deep
learning models by reducing high correlations between neurons. For this, we
present two regularisation terms computed from the weights of a minimum
spanning tree of the clique whose vertices are the neurons of a given network
(or a sample of those), where weights on edges are correlation dissimilarities.
We provide an extensive set of experiments to validate the effectiveness of our
terms, showing that they outperform popular ones. …

arxiv capacity correlations deep learning network neurons novel persistence terms tree