[P] A Simpler @PyTorch Annotated Implementation of EleutherAI's 20B Language Model GPT-NeoX.

**Github:** [**https://github.com/labmlai/neox**](https://github.com/labmlai/neox)

**Annotated implementation:** [**https://lit.labml.ai/github/labmlai/neox/tree/main/src/neox/\_\_init\_\_.py**](https://lit.labml.ai/github/labmlai/neox/tree/main/src/neox/__init__.py)

**Original repo from EleutherAI:** [**https://github.com/EleutherAI/gpt-neox**](https://github.com/EleutherAI/gpt-neox)

We have included samples showing how to generate text and to fine-tune. We haven't included a bunch of optimizations that were present in original GPT-NeoX to keep things simple.

eleutherai gpt implementation language language model machinelearning pytorch