FlashAttention 2: making Transformers 800% faster w/o approximation - with Tri Dao of Together AI

Listen now (55 min) | How FlashAttention became the new industry standard architecture, how FlashAttention 2 is 2x faster still, life inside the Stanford Hazy Research lab, and hints of the post-Transformers future

approximation architecture dao faster future industry inside lab life making research standard stanford together transformers