A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks. (arXiv:2201.08652v1 [stat.ML])

To fit sparse linear associations, a LASSO sparsity inducing penalty with a
single hyperparameter provably allows to recover the important features
(needles) with high probability in certain regimes even if the sample size is
smaller than the dimension of the input vector (haystack). More recently
learners known as artificial neural networks (ANN) have shown great successes
in many machine learning tasks, in particular fitting nonlinear associations.
Small learning rate, stochastic gradient descent algorithm and large training
set help to cope …

artificial arxiv lasso ml networks neural networks transition