Diffusion on language model embeddings for protein sequence generation

arXiv:2403.03726v1 Announce Type: new
Abstract: Protein design requires a deep understanding of the inherent complexities of the protein universe. While many efforts lean towards conditional generation or focus on specific families of proteins, the foundational task of unconditional generation remains underexplored and undervalued. Here, we explore this pivotal domain, introducing DiMA, a model that leverages continuous diffusion on embeddings derived from the protein language model, ESM-2, to generate amino acid sequences. DiMA surpasses leading solutions, including autoregressive transformer-based and discrete …

abstract arxiv complexities cs.ai cs.lg design diffusion domain embeddings explore families focus language language model lean pivotal protein proteins q-bio.bm type understanding universe