µTransfer: A technique for hyperparameter tuning of enormous neural networks

Great scientific achievements cannot be made by trial and error alone. Every launch in the space program is underpinned by centuries of fundamental research in aerodynamics, propulsion, and celestial bodies. In the same way, when it comes to building large-scale AI systems, fundamental research forms the theoretical insights that drastically reduce the amount of trial […]

The post µTransfer: A technique for hyperparameter tuning of enormous neural networks appeared first on Microsoft Research.

networks neural networks research blog