Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression

arXiv:2305.16534v2 Announce Type: replace-cross
Abstract: This paper introduces a novel theoretical framework for the analysis of vector-valued neural networks through the development of vector-valued variation spaces, a new class of reproducing kernel Banach spaces. These spaces emerge from studying the regularization effect of weight decay in training networks with activations like the rectified linear unit (ReLU). This framework offers a deeper understanding of multi-output networks and their function-space characteristics. A key contribution of this work is the development of a …

abstract analysis arxiv class compression cs.lg development framework insights kernel multi-task learning network networks neural networks novel paper regularization spaces stat.ml studying through training type variation vector