A Derivation of Feedforward Neural Network Gradients Using Fr\'echet Calculus. (arXiv:2209.13234v1 [cs.LG])

We present a derivation of the gradients of feedforward neural networks using
Fr\'echet calculus which is arguably more compact than the ones usually
presented in the literature. We first derive the gradients for ordinary neural
networks working on vectorial data and show how these derived formulas can be
used to derive a simple and efficient algorithm for calculating a neural
networks gradients. Subsequently we show how our analysis generalizes to more
general neural network architectures including, but not limited to, …

arxiv calculus derivation network neural network