[D] Why is Vaswani et al still the SOTA when the attention mechanism is O(n²)?

There are so many attention mechanisms (guided attention, apple's aft, nystromformer, etc) that alleviate the O(n²) to something like O(n). Why don't recent LMs use these techniques to speed up training and matrix multiplication in the SA layer?

attention machinelearning sota