How do AI researchers know create novel architectures? What do they know which I don't?

For example take transformer architecture or attention mechanism. How did they know that by combining self attention with layer normalisation, positional encoding we can have models that will outperform lstm, CNNs?

I am asking this from the perspective of mathematics. Currently I feel like I can never come up with something new, and there is something missing which ai researchers know which I don't.

So what do I need to know that will allow me to solve problems in new …

ai researchers architecture architectures attention cnns deeplearning encoding example layer lstm mathematics novel perspective positional encoding researchers transformer transformer architecture will