[R] Equivariant Architectures for Learning in Deep Weight Spaces - Nvidia 2023 - DWSNets has 60 percentage points more on the MNIST INR dataset in comparison to the transformer!

Paper: [https://arxiv.org/abs/2301.12780](https://arxiv.org/abs/2301.12780)

Github: [https://github.com/AvivNavon/DWSNets](https://github.com/AvivNavon/DWSNets)

Blog: [https://developer.nvidia.com/blog/designing-deep-networks-to-process-other-deep-networks/?=&linkId=100000214235775](https://developer.nvidia.com/blog/designing-deep-networks-to-process-other-deep-networks/?=&linkId=100000214235775)

Abstract:

>Designing machine learning architectures for processing neural networks in their raw weight matrix form is a newly introduced research direction. Unfortunately, the unique symmetry structure of deep weight spaces makes this design very challenging. If successful, such architectures would be capable of performing a wide range of intriguing tasks, from adapting a pre-trained network to a new domain to editing objects represented as functions (INRs or NeRFs). As a first step towards this …

abstract architectures design machine machine learning machinelearning matrix network networks neural networks processing raw research spaces symmetry tasks weight matrix