[D] How to actually do the final PPO with a reward model in RLHF?

Hi,

I want to get hands-on with the RLHF pipeline. I found an online reward model that can be potentially used [https://huggingface.co/OpenAssistant/reward-model-deberta-v3-large-v2](https://huggingface.co/OpenAssistant/reward-model-deberta-v3-large-v2)

One thing that's unclear is how can I use this model for fine-tuning something like GPTNeoX-20B? My end goal is currently just a one-shot answering model (not necessarily a chat)

chat fine-tuning machinelearning ppo rlhf something