High Fidelity Visualization of What Your Self-Supervised Representation Knows About. (arXiv:2112.09164v2 [cs.LG] UPDATED)

Discovering what is learned by neural networks remains a challenge. In
self-supervised learning, classification is the most common task used to
evaluate how good a representation is. However, relying only on such downstream
task can limit our understanding of what information is retained in the
representation of a given input. In this work, we showcase the use of a
Representation Conditional Diffusion Model (RCDM) to visualize in data space
the representations learned by self-supervised models. The use of RCDM is …

arxiv fidelity lg representation visualization