Zephyr 7B beta - How much does DPO really help?

Colab Full model with DPO: https://drp.li/QYAnu
Colab with SFT Only: https://drp.li/HAvSc

My Links:
Twitter - https://twitter.com/Sam_Witteveen
Linkedin - https://www.linkedin.com/in/samwitteveen/

Github:
https://github.com/samwit/langchain-tutorials (updated)
https://github.com/samwit/llm-tutorials

Timestamps
00:00 Intro
00:15 Zephyr 7B - Model on HF
01:04 Zephyr 7B -Beta Technical Paper
01:49 MT Bench
02:07 AlpacaEval
02:28 UltraChat Dataset
02:48 Zephyr 7B-Beta Flaws
03:21 UltraFeedback Dataset
05:26 Code Time
05:35 Full Model with DPO
08:44 Model with SFT Only
12:11 Alignment Notebook

alignment beta code dataset flaws github intro notebook paper sft technical