[P] Jamba: the first production-grade Mamba-based model delivering best-in-class quality and performance.

Post: [https://www.ai21.com/blog/announcing-jamba](https://www.ai21.com/blog/announcing-jamba)



>We are thrilled to announce Jamba, the world’s first production-grade Mamba based model. By enhancing [Mamba](https://arxiv.org/pdf/2312.00752.pdf) Structured State Space model (SSM) technology with elements of the traditional Transformer architecture, Jamba compensates for the inherent limitations of a pure SSM model. Offering a 256K context window, it is already demonstrating remarkable gains in throughput and efficiency—just the beginning of what can be possible with this innovative hybrid architecture. Notably, Jamba outperforms or matches other state-of-the-art models in its size …

machinelearning