[D] Statistics of Learned Representations in Self-Supervised Learning?

I haven’t done an extensive review of the modern literature of SSL, but I was thinking about this earlier. Is there any effort for learned representations to have things like maximum discriminability between distributions of data associated with different pseudo-labels? (Sort of like the LDA Rank Decomposition)

Or, would this actually be a bad idea, if you have very complicated data that can’t be “forced” into separable distributions with respect to their pseudo labels?

In a sense, this is why …

data labels lda literature machinelearning modern review self-supervised learning ssl statistics supervised learning thinking