EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations

arXiv:2306.12059v3 Announce Type: replace
Abstract: Equivariant Transformers such as Equiformer have demonstrated the efficacy of applying Transformers to the domain of 3D atomistic systems. However, they are limited to small degrees of equivariant representations due to their computational complexity. In this paper, we investigate whether these architectures can scale well to higher degrees. Starting from Equiformer, we first replace $SO(3)$ convolutions with eSCN convolutions to efficiently incorporate higher-degree tensors. Then, to better leverage the power of higher degrees, we propose …

abstract architectures arxiv complexity computational cs.ai cs.lg domain however paper physics.comp-ph scale scaling small systems transformer transformers type