FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics

arXiv:2402.16901v1 Announce Type: cross
Abstract: Metagenomic data, comprising mixed multi-species genomes, are prevalent in diverse environments like oceans and soils, significantly impacting human health and ecological functions. However, current research relies on K-mer representations, limiting the capture of structurally relevant gene contexts. To address these limitations and further our understanding of complex relationships between metagenomic sequences and their functions, we introduce a protein-based gene representation as a context-aware and structure-relevant tokenizer. Our approach includes Masked Gene Modeling (MGM) for gene …

abstract arxiv cs.ai cs.lg current data diverse environments function functions gene health human language language model limitations metagenomics mixed oceans q-bio.gn research type understanding