Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

arXiv:2403.03234v1 Announce Type: cross
Abstract: Large-scale sequence modeling has sparked rapid advances that now extend into biology and genomics. However, modeling genomic sequences introduces challenges such as the need to model long-range token interactions, the effects of upstream and downstream regions of the genome, and the reverse complementarity (RC) of DNA. Here, we propose an architecture motivated by these challenges that builds off the long-range Mamba block, and extends it to a BiMamba component that supports bi-directionality, and to a …

abstract advances arxiv biology challenges cs.lg dna effects genome genomic genomics however interactions modeling q-bio.gn scale token type