The Platonic Representation Hypothesis

arXiv:2405.07987v1 Announce Type: new
Abstract: We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple domains, the ways by which different neural networks represent data are becoming more aligned. Next, we demonstrate convergence across data modalities: as vision models and language models get larger, they measure distance between datapoints in a more and more alike way. We hypothesize that this convergence is driving …

abstract ai models arxiv convergence cs.ai cs.cv cs.lg cs.ne data domains examples hypothesis literature multiple networks neural networks next representation survey type vision vision models