I am heartbroken 💔. Random Forest classifier beat Transformer Encoder classifier by about 2.5%

I wanted to learn the transformer from scratch. So I decided to try on a Twitter sentiment classification problem on Kaggle.

I have been working on it for the last 2 3 weeks. Using the transformer encoder I got training accuracy of about 97% and validation accuracy of Max 95.45%.

The dataset was big enough and I was overly optimistic. I thought maybe I could get an accuracy of more than 98% on both training and validation.

Turns out someone …

accuracy classification classifier deeplearning encoder kaggle learn random scratch sentiment training transformer transformer encoder twitter validation