Hyperplane Arrangements of Trained ConvNets Are Biased. (arXiv:2003.07797v2 [cs.CV] UPDATED)

We investigate the geometric properties of the functions learned by trained
ConvNets in the preactivation space of their convolutional layers, by
performing an empirical study of hyperplane arrangements induced by a
convolutional layer. We introduce statistics over the weights of a trained
network to study local arrangements and relate them to the training dynamics.
We observe that trained ConvNets show a significant statistical bias towards
regular hyperplane configurations. Furthermore, we find that layers showing
biased configurations are critical to validation …

architectures arxiv bias dynamics hyperplane network observe performance space statistical statistics study training validation