Reza Shabani - How To Train Your Own LLM (LLM Bootcamp)

In this video, Reza Shabani of replit walks through the process of training your own LLM, from data processing to deployment.

Download the slides and read the talk summary here: https://fullstackdeeplearning.com/llm-bootcamp/spring-2023/shabani-train-your-own

Watch the rest of the LLM Bootcamp videos here: https://www.youtube.com/playlist?list=PL1T8fO7ArWleyIqOy37OVXsP4hFXymdOZ

Outro music made with Riffusion: https://github.com/riffusion/riffusion

00:00 Why train your own LLMs?
04:44 The Modern LLM Stack
07:24 Data Pipelines: Databricks & Hugging Face
13:34 Preprocessing
16:29 Tokenizer Training
19:57 Running Training: MosaicML, Weights & Biases
22:41 Testing & …

bootcamp data databricks data pipelines data processing deployment face hugging face llm llms mosaicml pipelines process processing replit running stack through training video