[D] Interview with Tri Dao, Stanford: On FlashAttention and sparsity, quantization, and efficient inference

[New episode of Imbue's *Generally Intelligent* podcast](https://imbue.com/podcast/2024-02-08-podcast-episode-33-tri-dao/) with Tri Dao, author of [FlashAttention](https://arxiv.org/abs/2205.14135) and Chief Scientist at Together AI.

Some topics covered:

* Taking a contrarian bet on recurrent connections over attention
* Using data augmentation to encode knowledge into models
* Designing algorithms that take advantage of hardware

Listen to the conversation:

* [Spotify](https://open.spotify.com/show/1hikWa5LWDQJwXtz5LoeVn)
* [Apple Podcasts](https://podcasts.apple.com/us/podcast/generally-intelligent/id1544921720)
* [Pocket Casts](https://pca.st/ewh266dr)
* [Highlights and referenced papers](https://imbue.com/podcast/2024-02-08-podcast-episode-33-tri-dao/)

algorithms attention augmentation conversation data designing encode hardware knowledge machinelearning the conversation topics