Web: http://arxiv.org/abs/2203.15465

Sept. 22, 2022, 1:12 a.m. | Umberto Lupo, Damiano Sgarbossa, Anne-Florence Bitbol

cs.LG updates on arXiv.org arxiv.org

Self-supervised neural language models with attention have recently been
applied to biological sequence data, advancing structure, function and
mutational effect prediction. Some protein language models, including MSA
Transformer and AlphaFold's EvoFormer, take multiple sequence alignments (MSAs)
of evolutionarily related proteins as inputs. Simple combinations of MSA
Transformer's row attentions have led to state-of-the-art unsupervised
structural contact prediction. We demonstrate that similarly simple, and
universal, combinations of MSA Transformer's column attentions strongly
correlate with Hamming distances between sequences in MSAs. Therefore, …

arxiv bio language language models learn protein relationships

More from arxiv.org / cs.LG updates on arXiv.org

Research Scientists

@ ODU Research Foundation | Norfolk, Virginia

Embedded Systems Engineer (Robotics)

@ Neo Cybernetica | Bedford, New Hampshire

2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences

@ Lawrence Berkeley National Lab | San Francisco, CA

Senior Manager Data Scientist

@ NAV | Remote, US

Senior AI Research Scientist

@ Earth Species Project | Remote anywhere

Research Fellow- Center for Security and Emerging Technology (Multiple Opportunities)

@ University of California Davis | Washington, DC

Staff Fellow - Data Scientist

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Staff Fellow - Senior Data Engineer

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Data Scientist (Analytics) - Singapore

@ Momos | Singapore, Central, Singapore

Machine Learning Scientist, Drug Discovery

@ Flagship Pioneering, Inc. | Cambridge, MA

Applied Scientist - Computer Vision

@ Flawless | Los Angeles, California, United States

Sr. Data Engineer, Customer Service

@ Wayfair Inc. | Boston, MA