Fine-tune a Mistral-7b model with Direct Preference Optimization

Boost the performance of your supervised fine-tuned models

Pre-trained Large Language Models (LLMs) can only perform next-token prediction, making them unable to answer questions. This is why these base models are then fine-tuned on pairs of instructions and answers to act as helpful assistants. However, this process can still be flawed: fine-tuned LLMs can be biased, toxic, harmful, etc. This is where Reinforcement Learning from Human Feedback (RLHF) comes into play.

RLHF provides different answers to the …

act artificial intelligence assistants boost data science direct preference optimization editors pick language language models large language large language models llms making mistral next optimization performance prediction process programming questions them token