Language models can explain neurons in language models

Language models can explain neurons in language models

Fascinating interactive paper by OpenAI, describing how they used GPT-4 to analyze the concepts tracked by individual neurons in their much older GPT-2 model. "We generated cluster labels by embedding each neuron explanation using the OpenAI Embeddings API, then clustering them and asking GPT-4 to label each cluster."

Via OpenAI blog

ai analyze api cluster clustering embedding embeddings explorables generated generativeai gpt gpt-2 gpt-4 gpt4 interactive labels language language models llms neuron neurons openai paper