Which text-gen benchmark to use for 100M parameter (NanoGPT) pretrained-only language model? [D]

I've got model pretraining running on NanoGPT for a GPT2 tokenized dataset and a TokenMonster tokenized dataset, so I can compare the difference. It's only a 100M parameter model, so it doesn't do much.

What benchmark can I use? NanoGPT runs on Pytorch, so I could use something that integrates with PyTorch, or I could use something that sends text prompts and analyzes text responses (or token IDs.)

Is there a standard benchmark that uses the full, non-instruct trained format? …

benchmark dataset difference gen language language model machinelearning nanogpt pytorch running text