Koopman-based generalization bound: New aspect for full-rank weights

arXiv:2302.05825v3 Announce Type: replace
Abstract: We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight matrices. Our bound is tighter than existing norm-based bounds when the condition numbers of weight matrices are small. Especially, it is completely independent of the width of the network if the weight matrices are orthogonal. Our bound does not contradict to the existing bounds but is a …

abstract arxiv cs.lg focus low math.fa networks neural networks norm numbers operators small stat.ml type