[R] Exponentially Faster Language Modelling

**TL;DR:** Organize your neurons into a tree to get 78x faster inference (theoretical limit is 341x).

This was demonstrated on BERT-base, where this change preserved 96% of its downstream GLUE performance. For a quick comparison, DistilBERT offers 1.6x acceleration while preserving 97% of GLUE performance.

This is a [HuggingFace Featured Paper from 11/21/2023](https://huggingface.co/papers/2311.10770).

Paper: [https://arxiv.org/abs/2311.10770](https://arxiv.org/abs/2311.10770)

Code: [https://github.com/pbelcak/UltraFastBERT](https://github.com/pbelcak/UltraFastBERT)

Model: [https://huggingface.co/pbelcak/UltraFastBERT-1x11-long](https://huggingface.co/pbelcak/UltraFastBERT-1x11-long)

Abstract:

>Language models only really need to use an exponential fraction of their neurons for individual inferences.
>
>As proof, we …

abstract bert change comparison distilbert faster glue inference language language modelling language models machinelearning modelling neurons organize performance tree