Why Using Learning Rate Schedulers In NNs May Be a Waste of Time

Why Using Learning Rate Schedulers in NNs May Be a Waste of Time

Hint: Batch size is the key, and it might not be what you think!

Photo by Andrik Langfield on UnsplashTL;DR: instead of decreasing the learning rate by a factor, increase the batch size using the same factor to achieve faster convergence and, if not better, training results.

In recent years, the continuous development of neural networks has led to an increasing amount of applications of them …

data science deep learning learning machine learning neural networks nns rate tensorflow time waste