Attention Mechanisms in transformers

Hi All,

I have been a bit out of touch of the attention mechanisms. I know the core multihead attention used in attention is all you need paper but I think there have been some new developments in the field. Can someone help le with a list of the new attention mechanisms that I can start reading up on. I know of Flash Attention but I think that are even newer methods now. I tried googling but with not much …

attention attention is all you need attention mechanisms core deeplearning list paper think transformers