From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication

arXiv:2310.01211v2 Announce Type: replace
Abstract: It has been observed that representations learned by distinct neural networks conceal structural similarities when the models are trained under similar inductive biases. From a geometric perspective, identifying the classes of transformations and the related invariances that connect these representations is fundamental to unlocking applications, such as merging, stitching, and reusing different neural modules. However, estimating task-specific transformations a priori can be challenging and expensive due to several factors (e.g., weights initialization, training hyperparameters, or …

abstract arxiv biases communication cs.lg inductive networks neural networks perspective product space type