[D] Sentiment Analyzer with Transformer Architecture?

I was wondering how one could utilize the transformer architecture to make a sentiment analyzer from scratch. I was thinking of taking only the encoder sections of the transformer neural network and then plugging it into densely connected layers. The idea would be to get the encoding/latent space representation and then have the dense layers find sentiment from that. I've tried this with some mixed results. Any ideas on how one would go about doing this? I'd like any kind …

architecture machinelearning transformer