DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome

arXiv:2306.15006v2 Announce Type: replace-cross
Abstract: Decoding the linguistic intricacies of the genome is a crucial problem in biology, and pre-trained foundational models such as DNABERT and Nucleotide Transformer have made significant strides in this area. Existing works have largely hinged on k-mer, fixed-length permutations of A, T, C, and G, as the token of the genome language due to its simplicity. However, we argue that the computation and sample inefficiencies introduced by k-mer tokenization are primary obstacles in developing large …

abstract arxiv benchmark biology cs.ai cs.ce cs.cl decoding foundation foundational models foundation model genome permutations q-bio.gn species transformer type