[P] Reproducing the "Self-Rewarding Language Models" Paper by MetaAI

Hey all,

After reading the Self-Rewarding Language Models paper by the team at Meta, it felt very approachable and reproducible, so we spent some time implementing it.

​

The scripts provided take any base model and put it in a loop of :

1) Supervised fine-tuning on an initial dataset

2) Generating new prompts using the SFT

3) Generating N responses per prompt

4) Scoring the generated responses 1-5

5) Running DPO on the rewards from the model itself.

​ …

dataset felt fine-tuning hey language language models loop machinelearning meta paper reading scripts supervised fine-tuning team