Reliability of CKA as a Similarity Measure in Deep Learning. (arXiv:2210.16156v2 [cs.LG] UPDATED)

Comparing learned neural representations in neural networks is a challenging
but important problem, which has been approached in different ways. The
Centered Kernel Alignment (CKA) similarity metric, particularly its linear
variant, has recently become a popular approach and has been widely used to
compare representations of a network's different layers, of architecturally
similar networks trained differently, or of models with different architectures
trained on the same data. A wide variety of conclusions about similarity and
dissimilarity of these various representations …

arxiv cka deep learning reliability