Equivariant Pretrained Transformer for Unified Geometric Learning on Multi-Domain 3D Molecules

arXiv:2402.12714v1 Announce Type: new
Abstract: Pretraining on a large number of unlabeled 3D molecules has showcased superiority in various scientific applications. However, prior efforts typically focus on pretraining models on a specific domain, either proteins or small molecules, missing the opportunity to leverage the cross-domain knowledge. To mitigate this gap, we introduce Equivariant Pretrained Transformer (EPT), a novel pretraining framework designed to harmonize the geometric learning of small molecules and proteins. To be specific, EPT unifies the geometric modeling of …

abstract applications arxiv cs.lg domain domain knowledge focus knowledge molecules pretraining prior proteins small transformer type