Diffusion Language Models Are Versatile Protein Learners

arXiv:2402.18567v1 Announce Type: new
Abstract: This paper introduces diffusion protein language model (DPLM), a versatile protein language model that demonstrates strong generative and predictive capabilities for protein sequences. We first pre-train scalable DPLMs from evolutionary-scale protein sequences within a generative self-supervised discrete diffusion probabilistic framework, which generalizes language modeling for proteins in a principled way. After pre-training, DPLM exhibits the ability to generate structurally plausible, novel, and diverse protein sequences for unconditional generation. We further demonstrate the proposed diffusion generative …

abstract arxiv capabilities cs.lg diffusion framework generative language language model language models modeling paper predictive protein proteins q-bio.bm scalable scale train type