3Blue1Brown: Attention in transformers, visually explained

3Blue1Brown: Attention in transformers, visually explained

Grant Sanderson publishes animated explainers of mathematical topics on YouTube, to over 6 million subscribers. His latest shows how the attention mechanism in transformers (the algorithm behind most LLMs) works and is by far the clearest explanation I've seen of the topic anywhere.

I was intrigued to find out what tool he used to produce the visualizations. It turns out Grant built his own open source Python animation library, manim, to enable his YouTube …

ai algorithm animated attention explained generativeai grant llms shows subscribers the algorithm topics transformers youtube