An Analytic Solution to Covariance Propagation in Neural Networks

arXiv:2403.16163v1 Announce Type: new
Abstract: Uncertainty quantification of neural networks is critical to measuring the reliability and robustness of deep learning systems. However, this often involves costly or inaccurate sampling methods and approximations. This paper presents a sample-free moment propagation technique that propagates mean vectors and covariance matrices across a network to accurately characterize the input-output distributions of neural networks. A key enabler of our technique is an analytic solution for the covariance of random variables passed through nonlinear activation …

abstract arxiv covariance cs.ai cs.lg deep learning free however learning systems mean measuring network networks neural networks paper propagation quantification reliability robustness sample sampling solution stat.ml systems type uncertainty vectors