Why is it said transformers are more parallelizable than RNN's?

The parallelization of transformers and RNNs (Recurrent Neural Networks) is often discussed. It's commonly said that transformers are more parallelizable than RNNs. However, this is a rather vague statement that merits further discussion.

One could argue that an RNN can be made as parallelizable as desired by simply adding more instances to each batch.

What is generally meant by saying transformers are more parallelizable is that transformers lack time-dependent operations. In other words, given an input, all operations can be …

networks neural networks neuralnetworks parallelization recurrent neural networks rnn transformers