How do I know when to stop hyper parameter tuning and try something else?

Edit: its for deep learning just to clarify; im referencing stuff like messing around with a CNN's architecture, activation, optimizer, learning rate, regularizers, etc

I feel like i understand the math and algorithm behind model architectures quite well; i take care to preprocess and clean data, but in practice i struggle to get good performance. I always just end up manually tuning hyper parameters or using gridsearch for days or weeks with minimal improvement in erformance.

I guess my question …

algorithm architecture architectures clean data cnn data datascience deep learning edit etc math preprocess rate something