Pre trained vs custom models

So I’m trying to build a text classifier, specifically for Twitter. When I started that project I had little to no knowledge about deep learning so I stuck with a combo of TF-ID with a logistic regression. With that I got around 65% accuracy, which for the task at hand is ok.

Now I’m learning about deep learning, RNNs and pretty soon will be learning about transformers.

My question is: should I make another model using a pretrained model, like …

custom models datascience