Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression

arXiv:2305.00608v3 Announce Type: replace-cross
Abstract: We study the properties of differentiable neural networks activated by rectified power unit (RePU) functions. We show that the partial derivatives of RePU neural networks can be represented by RePUs mixed-activated networks and derive upper bounds for the complexity of the function class of derivatives of RePUs networks. We establish error bounds for simultaneously approximating $C^s$ smooth functions and their derivatives using RePU-activated deep neural networks. Furthermore, we derive improved approximation error bounds when data …

abstract applications arxiv complexity cs.lg derivatives differentiable functions mixed networks neural networks power regression show stat.ml study type