[D] HyenaDNA and Mamba are not good at sequential labelling ?

Hello guys, I've been working on a sequential labelling using DNA sequences as inputs. Lately there have been 2 foundation models released HyenaDNA (Based on Hyena operator) and Caduceus (based on mamba), I used both pretrained and from scratch models and performances are terrible even with pretrained ones.

Does anyone have experience with this type of models, and what are the potential causes for performance drop ? I am literally getting zero performance for the minority class ? Does mamba …

dna foundation good hello inputs labelling machinelearning mamba ones performances scratch