GPT From Scratch With MLX

Define and train GPT-2 on your MacBook

My goal with this post is to walk you through defining and training GPT-2 from scratch with MLX, Apple’s machine-learning library for Apple silicon. I want to leave no stone unturned from tokenizer to sampling. In the spirit of Karpathy’s excellent GPT from scratch tutorial, we will train a model on the works of Shakespeare [1]. We will start with a blank Python file and …

apple apple silicon deep-dives deep learning gpt gpt-2 library llm machine mlx nlp sampling scratch sergey silicon stone through train training you