On the Implicit Bias in Deep-Learning Algorithms. (arXiv:2208.12591v1 [cs.LG])

Gradient-based deep-learning algorithms exhibit remarkable performance in
practice, but it is not well-understood why they are able to generalize despite
having more parameters than training examples. It is believed that implicit
bias is a key factor in their ability to generalize, and hence it has been
widely studied in recent years. In this short survey, we explain the notion of
implicit bias, review main results and discuss their implications.

algorithms arxiv bias learning lg