Research Highlights: Scaling MLPs: A Tale of Inductive Bias

Multi-layer Perceptrons (MLPs) are the most fundamental type of neural network, so they play an important role in many machine learning systems and are the most theoretically studied type of neural network. A new paper from researchers at ETH Zurich pushes the limits of pure MLPs, and shows that scaling them up allows much better performance than expected from MLPs in the past. These findings may have important implications for the study of inductive biases, the theory of deep learning, …

analysis arxiv bias data science eth zurich google news feed highlights inductive machine machine learning main feature mlp network neural network paper reports research researchers research highlight role scaling shows systems them type weekly newsletter articles zurich