[N] Huggingface/nvidia release open source GPT-2B trained on 1.1T tokens

## [https://huggingface.co/nvidia/GPT-2B-001](https://huggingface.co/nvidia/GPT-2B-001)

## Model Description

GPT-2B-001 is a transformer-based language model. GPT refers to a class of transformer decoder-only models similar to GPT-2 and 3 while 2B refers to the total trainable parameter count (2 Billion) \[1, 2\].

This model was trained on 1.1T tokens with [NeMo](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/nlp/nemo_megatron/intro.html).

Requires Ampere or Hopper devices.

ampere count decoder devices gpt gpt-2 hopper language language model machinelearning transformer