Rademacher Complexity of Neural ODEs via Chen-Fliess Series

We show how continuous-depth neural ODE models can be framed as single-layer, infinite-width nets using the Chen--Fliess series expansion for nonlinear ODEs. In this net, the output ''weights'' are taken from the signature of the control input -- a tool used to represent infinite-dimensional paths as a sequence of tensors -- which comprises iterated integrals of the control input over a simplex. The ''features'' are taken to be iterated Lie derivatives of the output function with respect to the vector …

chen complexity continuous control cs.lg cs.sy eess.sy expansion layer math.oc series show stat.ml tool via