How do Vision Transformers work? – Paper explained | multi-head self-attention & convolutions

It turns out that multi-head self-attention and convolutions are complementary. So, what makes multi-head self-attention different from convolutions? How and why do Vision Transformers work? In this video, we will find out by explaining the paper “How Do Vision Transformers Work?” by Namuk & Kim, 2021.

SPONSOR: Weights & Biases 👉 https://wandb.me/ai-coffee-break

⏩ Vision Transformers explained playlist: https://youtube.com/playlist?list=PLpZBeKTZRGPMddKHcsJAOIghV8MwzwQV6
📺 ViT: An image is worth 16x16 pixels: https://youtu.be/DVoHvmww2lQ
📺 Swin Transformer: https://youtu.be/SndHALawoag
📺 ConvNext: https://youtu.be/QqejV0LNDHA
📺 DeiT: https://youtu.be/-FbV2KgRM8A
📺 Adversarial attacks: …

attention explained head multi-head paper self-attention transformers vision work