[Discussion] What kind of problems do you run into training your models?

It must be really frustrating for many to try and train or fine-tune an open-source model only to fail because of just how complicated it is.

Recently heard a conversation of the huggingface developers on a podcast talking about how they identify and debug the activation to apply normalization to stabilize training of LLMs.

​

I am curious to know, what kind of problems do you run into while training your models (even non-LLM) and how do you usually solve …

apply conversation debug developers huggingface identify kind machinelearning podcast train training