[D] Is huggingface transformers like really slow on TPUs for anyone else?

I’ve been trying to run experiments using the T5 implementation from huggingface transformers, but it’s absurdly slow compared to GPU. I’m using Google colab. Is there anything I can do?

huggingface machinelearning tpus transformers