Training ReLU networks to high uniform accuracy is intractable. (arXiv:2205.13531v1 [cs.LG])

Statistical learning theory provides bounds on the necessary number of
training samples needed to reach a prescribed accuracy in a learning problem
formulated over a given target class. This accuracy is typically measured in
terms of a generalization error, that is, an expected value of a given loss
function. However, for several applications -- for example in a
security-critical context or for problems in the computational sciences --
accuracy in this sense is not sufficient. In such cases, one would …

accuracy arxiv networks relu training uniform