A Mutual Information Perspective on Federated Contrastive Learning

arXiv:2405.02081v1 Announce Type: new
Abstract: We investigate contrastive learning in the federated setting through the lens of SimCLR and multi-view mutual information maximization. In doing so, we uncover a connection between contrastive representation learning and user verification; by adding a user verification loss to each client's local SimCLR loss we recover a lower bound to the global multi-view mutual information. To accommodate for the case of when some labelled data are available at the clients, we extend our SimCLR variant …

abstract arxiv client cs.lg information lens loss perspective representation representation learning through type verification view