[D] Critical batch size and LLMs

In a video about ["A little guide to building Large Language Models in 2024" at 41:38](https://youtu.be/2-SPH9hIKT8?t=2498) the author starts to talk about the limits of how big the batch size can be.

​

>Well, if you start to have a very large batch size, the model for each optimization step makes less efficient use of each token, because the batch size is so big that each token is kind of washed out in the optimization step. And roughly, it's a …

big call kind machinelearning optimization token