Scratch Implementation of Vision Transformer in PyTorch

I am sharing my scratch PyTorch implementation of Vision Transformer. It has a detailed step-by-step guide of Self-attention and model specifics for learning Vision Transformers. The network is a small scaled-down version of the original architecture and achieves around 99.4% test Accuracy on MNIST and 92.5% on FashionMNIST.

Hope you find it helpful. Feedbacks appreciated.

GitHub: [https://github.com/s-chh/PyTorch-Vision-Transformer-ViT-MNIST](https://github.com/s-chh/PyTorch-Vision-Transformer-ViT-MNIST)

accuracy architecture attention computervision guide implementation mnist network pytorch self-attention small test transformer transformers vision