FoldToken: Learning Protein Language via Vector Quantization and Beyond

arXiv:2403.09673v1 Announce Type: cross
Abstract: Is there a foreign language describing protein sequences and structures simultaneously? Protein structures, represented by continuous 3D points, have long posed a challenge due to the contrasting modeling paradigms of discrete sequences. We introduce \textbf{FoldTokenizer} to represent protein sequence-structure as discrete symbols. This innovative approach involves projecting residue types and structures into a discrete space, guided by a reconstruction loss for information preservation. We refer to the learned discrete symbols as \textbf{FoldToken}, and the sequence …

abstract arxiv beyond challenge continuous cs.ai cs.lg foreign language language modeling protein protein structures q-bio.bm quantization type vector via