Fine-tune an Instruct model over raw text data

Fine-tune a modern chatbot with minimal conversational data for under $10

Purpose

Getting a modern chatbot to uphold it’s capabilities on your own data remains a complex task. Context window sizes are increasing rapidly with leading products like Gemini 1.5 Pro’s and Claude 3’s big leap to a 1 million token capacity. However, a company like The Guardian, where I currently work, has countless code repositories containing hundreds of millions of tokens worth of data.

The recently …

chatbots data science editors pick fine-tuning llm