llm.c

llm.c

Andrej Karpathy implements LLM training - initially for GPT-2, other architectures to follow - in just over 1,000 lines of C on top of CUDA. Includes a tutorial about implementing LayerNorm by porting an implementation from Python.

Via @karpathy

ai andrej karpathy andrejkarpathy architectures cuda generativeai gpt gpt-2 implementation llm llms python training tutorial via