How to Code RLHF on LLama2 w/ LoRA, 4-bit, TRL, DPO

Python code to code "Reinforcement Learning from Human Feedback" (RLHF) on a LLama 2 model with 4-bit quantization, LoRA and new DPO method, by Stanford Univ (instead of old PPO). Fine-tune LLama 2 with DPO.

A1. Code for Supervised Fine-tuning LLama2 model with 4-bit quantization.
A2. Code for DPO-Trainer by HuggingFace with PEFT, LoRA, 4-bit bnb, ...

B1. Code for Supervised Fine-tuning LLama1 model with 4-bit quantization, LoRA.
B2. Code for Reward Modelling of LLama1 model with 4-bit quantization.
B3. …

code feedback fine-tuning human human feedback llama llama 2 llama2 llama 2 model lora ppo python quantization reinforcement reinforcement learning rlhf stanford trainer