How I Created an Animation Of the Embeddings During Fine-Tuning

Using Cleanlab, PCA, and Procrustes to visualize ViT fine-tuning on CIFAR-10

In the field of machine learning, Vision Transformers (ViT) are a type of model used for image classification. Unlike traditional convolutional neural networks, ViTs use the transformer architecture, which was originally designed for natural language processing tasks, to process images. Fine-tuning these models, for optimal performance can be a complex process.

In a previous article, I used an animation to demonstrate changes in the embeddings during the fine-tuning …

animation architecture artificial intelligence classification convolutional neural networks embeddings fine-tuning fine-tuning-transformer image image processing images language language processing machine machine learning natural natural language natural language processing networks neural networks process processing python transformer transformer architecture transformers type vision vision transformers vit