Gradient Networks

arXiv:2404.07361v1 Announce Type: new
Abstract: Directly parameterizing and learning gradients of functions has widespread significance, with specific applications in optimization, generative modeling, and optimal transport. This paper introduces gradient networks (GradNets): novel neural network architectures that parameterize gradients of various function classes. GradNets exhibit specialized architectural constraints that ensure correspondence to gradient functions. We provide a comprehensive GradNet design framework that includes methods for transforming GradNets into monotone gradient networks (mGradNets), which are guaranteed to represent gradients of convex functions. …

abstract applications architectures arxiv constraints cs.lg cs.ne eess.sp function functions generative generative modeling gradient math.oc modeling network networks neural network novel optimization paper significance transport type