Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks

arXiv:2405.05236v1 Announce Type: cross
Abstract: This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive …

abstract analysis arxiv class constraints cs.lg cs.sy eess.sy functions general math.oc networks neural networks paper performance performance analysis recurrent neural networks relu stability theory type