[P] I reproduced Anthropic's recent interpretability research

Not that many people are paying attention to LLM interpretability research when capabilities research is moving as fast as it currently is, but interpretability is really important and in my opinion, really interesting and exciting! Anthropic has made a lot of breakthroughs in recent months, the biggest one being "Towards Monosemanticity". The basic idea is that they found a way to train a sparse autoencoder to generate interpretable features based on transformer activations. This allows us to look at the …

anthropic attention basic capabilities interpretability interpretability research llm machinelearning moving opinion people research