How do you keep track of all the math?

I'm fairly new to deep learning. I've familiarized myself with the basics and I'm currently building my first neural network from scratch. The issue is, while the math itself isn't complicated, there are too many operations happening at once and it's hard to keep track of (especially the things like matrix multiplication and chain rule derivatives).

basics building deep learning deeplearning isn issue math matrix network neural network operations