OntoProtein: Protein Pretraining With Gene Ontology Embedding. (arXiv:2201.11147v6 [q-bio.BM] CROSS LISTED)

Self-supervised protein language models have proved their effectiveness in
learning the proteins representations. With the increasing computational power,
current protein language models pre-trained with millions of diverse sequences
can advance the parameter scale from million-level to billion-level and achieve
remarkable improvement. However, those prevailing approaches rarely consider
incorporating knowledge graphs (KGs), which can provide rich structured
knowledge facts for better protein representations. We argue that informative
biology knowledge in KGs can enhance protein representation with external
knowledge. In this work, …

arxiv bio embedding gene ontology protein