Is there an agreed-upon hierarchy of which LMs are better for word embeddings?

For context: my research is in applied ML, but not NLP. I'm running experiments to test if, in my field, word/document embeddings from transformers are better than word2vec. As of now, I'm using BERT, but it seems [that was a poor choice](https://www.reddit.com/r/MachineLearning/comments/vm2sti/n_inverse_scaling_prize_250k_in_prizes_for/ie2jy90/). Which brings me to this sub's expertise--is there a class of transformers (T5, RoBERTa, etc.) that are better for embeddings that are used in downstream tasks?

languagetechnology word embeddings