RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens

RedPajama is a project to create a set of leading, fully open-source
models. Today, we are excited to announce the completion of the first step
of this project: the reproduction of the LLaMA training dataset of over 1.2
trillion tokens.

dataset llama project redpajama set tokens training