AI21 Labs launches Jamba, an AI model that integrates two architectures: transformer and Mamba, which is based on the Structured State Space model (Kyle Wiggers/TechCrunch)

Kyle Wiggers / TechCrunch:

AI21 Labs launches Jamba, an AI model that integrates two architectures: transformer and Mamba, which is based on the Structured State Space model  —  Increasingly, the AI industry is moving toward generative AI models with longer contexts.  But models with large context windows tend to be compute-intensive.

ai21 ai21 labs ai industry ai model architectures industry jamba kyle labs mamba moving space state state space model techcrunch transformer