The FIRST Production-grade Mamba-based LLM!!!

Introducing Jamba: AI21's Groundbreaking SSM-Transformer Model
Debuting the first production-grade Mamba-based model delivering best-in-class quality and performance.

We are thrilled to announce Jamba, the world’s first production-grade Mamba based model. By enhancing Mamba Structured State Space model (SSM) technology with elements of the traditional Transformer architecture, Jamba compensates for the inherent limitations of a pure SSM model. Offering a 256K context window, it is already demonstrating remarkable gains in throughput and efficiency—just the beginning of what can be possible with …

ai21 architecture class groundbreaking jamba limitations llm mamba performance production quality space ssm state state space model technology transformer transformer architecture transformer model world