Rademacher Complexity of Neural ODEs via Chen-Fliess Series. (arXiv:2401.16655v1 [stat.ML])

We show how continuous-depth neural ODE models can be framed as single-layer,
infinite-width nets using the Chen--Fliess series expansion for nonlinear ODEs.
In this net, the output ''weights'' are taken from the signature of the control
input -- a tool used to represent infinite-dimensional paths as a sequence of
tensors -- which comprises iterated integrals of the control input over a
simplex. The ''features'' are taken to be iterated Lie derivatives of the
output function with respect to the vector …

arxiv chen complexity continuous control expansion layer series show stat.ml tool via