[D] Deepminds study into neural networks and the Chomsky hierarchy

Web: https://www.reddit.com/r/MachineLearning/comments/wh3nsr/d_deepminds_study_into_neural_networks_and_the/

The paper “Neural Networks and the Chomsky Hierarchy “ laid out which architectures are best suited for language classes in the Chomsky Hierarchy.

It puts the transformer architecture in type-3, the lowest class. Yet, all anyone is talking about in NLP are transformers.

How come they are so successful if they can only tackle the basic language class?

machinelearning networks neural neural networks study