Scale Alone Does not Improve Mechanistic Interpretability in Vision Models

arXiv:2307.05471v2 Announce Type: replace
Abstract: In light of the recent widespread adoption of AI systems, understanding the internal information processing of neural networks has become increasingly critical. Most recently, machine vision has seen remarkable progress by scaling neural networks to unprecedented levels in dataset and model size. We here ask whether this extraordinary increase in scale also positively impacts the field of mechanistic interpretability. In other words, has our understanding of the inner workings of scaled neural networks improved as …

abstract adoption ai systems arxiv become cs.cv dataset information interpretability light machine machine vision networks neural networks processing progress scale scaling systems type understanding vision vision models