AI21 Labs Jamba API 2026: The Mamba-Transformer Hybrid at Production Scale

Introduction: The Tel Aviv Lab That Took a Different Architectural Bet

Most of the LLM API market in 2026 is built on transformer-only architectures. AI21 Labs went a different direction with the Jamba series: a hybrid that interleaves Mamba state-space-model (SSM) blocks with transformer self-attention. The result is a model family that combines the long-context efficiency of SSMs with the in-context learning of attention, deployed behind an OpenAI-compatible API and priced for production use.

The flagship Jamba 1.5 Large has 398B total parameters (94B active per token, due to MoE-style routing), native 256K context, and pricing at $2.00 per million input/output tokens. The smaller Jamba 1.5 Mini drops to 52B total / 12B active at $0.20 per million tokens — comparable to GPT-4o-mini for most workloads but with the architectural advantage of 256K native context. Both ship behind an OpenAI-shaped /v1/chat/completions endpoint, so existing tools (LangChain, LlamaIndex, vLLM, the OpenAI Python SDK) work without code changes.

The company also distributes Jamba through three major cloud marketplaces — AWS Bedrock, Azure AI Foundry, and NVIDIA NIM — which means teams that already pay AWS or Azure can buy Jamba inference on the same invoice. This review walks through the architecture, model lineup, pricing, speed, and how Jamba compares to Llama 3.3 70B, GPT-4o-mini, and Claude 4.5.

The Jamba Architecture: Why SSM-Transformer Hybrid Matters

The pure-transformer architecture that has dominated since GPT-2 has a quadratic memory cost in context length. At 128K tokens, attention alone consumes 30-50% of the inference budget. At 256K, the cost compounds.

Mamba (state-space model) blocks solve this with linear-time sequence processing: memory cost grows linearly with context, not quadratically. The trade-off is that pure SSMs are weaker at precise in-context retrieval of facts that appeared mid-context. AI21's bet, validated in the Jamba paper and in production deployments, is that interleaving Mamba blocks with a smaller number of transformer attention blocks recovers the in-context learning ability while keeping most of the efficiency gain.

In practice, Jamba 1.5 Large can process 256K tokens of context at a memory cost that pure-transformer models need 128K context to match. This is most useful for:

• Long-document summarization (legal contracts, full codebases, scientific papers)
• Multi-turn conversations with code where the full file needs to stay in context
• RAG alternative — instead of chunking and retrieving, feed the whole document

For teams currently doing chunking-then-embedding-then-retrieval to fit a 32K context window, the Jamba 256K window is the most direct path to simplifying the pipeline.

What AI21 Hosts: The Jamba Lineup

The current model catalog at AI21 Studio is intentionally narrow — five models, all in the Jamba family, all using the same SSM-transformer architecture:

• Jamba 1.5 Large — 398B total parameters, 94B active per token, 256K context window. Flagship model for production long-context workloads.
• Jamba 1.5 Mini — 52B total parameters, 12B active per token, 256K context window. The price-performance default for most applications.
• Jamba Large 1.0 — 94B parameters, 256K context. The pre-1.5 flagship, still hosted for compatibility.
• Jamba Mini 1.0 — 27B parameters, 256K context. The original mini, still hosted.
• Jamba-Instruct (52B) — Instruction-tuned variant of the original Mini. Used as the base for Jamba 1.5 Mini.

AI21 does not rehost open-source models the way Together AI, Novita, or Fireworks do. If you need a specific Llama, Qwen, or DeepSeek variant, you need a different provider. The narrow focus is also the strength: every engineering hour goes into improving the Jamba family, not maintaining integrations for 200 community fine-tunes.

API Surface: OpenAI-Compatible, Plus AI21-Specific Endpoints

The main chat API follows the OpenAI shape. A request that works against api.openai.com works against AI21's endpoint after swapping the base URL and key:

import requests
response = requests.post(
    "https://api.ai21.com/studio/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "jamba-1.5-mini",
        "messages": [{"role": "user", "content": "hi"}}],
        "max_tokens": 256,
    },
    timeout=30,
)
print(response.json()["choices"][0]["message"]["content"])

On top of the OpenAI shape, AI21 Studio adds a few platform-specific endpoints:

• RAG engine (RAG Engine v2): A managed document store with chunking, embedding, and hybrid search. Documents upload via the AI21 Studio dashboard, queries are sent through a dedicated /v1/rag/query endpoint. Useful for teams that don't want to maintain a vector database.
• Tool use API: A structured function-calling endpoint that returns a JSON schema for the tool call, including argument validation against the function schema.
• JSON mode: Enforce valid JSON output via response_format={"type": "json_object"} — same shape as OpenAI's JSON mode.
• Streaming: Server-sent events for the chat endpoint, identical to OpenAI's stream: true shape.

For most teams, the chat-completions endpoint plus tool use covers 80% of use cases. The RAG engine is the unique selling point for teams that need document-grounded answers without building their own retrieval pipeline.

Pricing: How Jamba Stacks Up

AI21's pricing is in USD, billed per million tokens. The 2026-06-11 rate sheet:

Model	Input (per 1M)	Output (per 1M)	Active Params	Context
jamba-1.5-mini	$0.20	$0.20	12B	256K
jamba-1.5-large	$2.00	$2.00	94B	256K
jamba-large-1.0	$0.50	$0.50	94B	256K
jamba-mini-1.0	$0.25	$0.25	27B	256K

For comparison, the Jamba 1.5 Mini at $0.20/$0.20 per million tokens is competitive with GPT-4o-mini ($0.15/$0.60) on input cost and noticeably cheaper on output. Against Llama 3.3 70B at $0.39-$0.88 per million (depending on the host), Jamba 1.5 Mini is roughly half the cost on the same workload class.

Jamba 1.5 Large at $2.00/$2.00 is more expensive than Llama 3.3 70B but cheaper than Claude 4.5 Sonnet ($3.00/$15.00) and significantly cheaper than GPT-5 ($5.00/$15.00). For teams that need the 256K context window, the Jamba 1.5 Large is the most cost-effective option among frontier-tier models with that capability.

There is no public cache-read discount, no batch API, and no separate "tier" for cached input. Teams running very high volume should negotiate an enterprise rate via the AI21 sales channel.

Speed and Latency

From AI21's published benchmarks and independent measurements as of 2026-06-11:

• Jamba 1.5 Mini: ~95 tok/s output, TTFT ~140ms (US endpoint)
• Jamba 1.5 Large: ~32 tok/s output, TTFT ~340ms
• Jamba Large 1.0: ~38 tok/s output, TTFT ~280ms

The Mini is competitive with the speed tier of Llama 3.1 8B on Groq (1,250 tok/s) — Groq is faster for single-model production serving, but Groq does not host a model with 256K context. The Jamba 1.5 Large is slower than frontier-tier transformer-only models on raw tokens-per-second, but the 256K context lets you send prompts in a single request that would require chunking on other providers.

For the long-context use case, the real performance metric is "tokens of context processed per dollar per second," not raw tok/s. Jamba wins this comparison against GPT-5 and Claude 4.5 on long-document workloads because it can process the whole document in one request while the others need to chunk or use their own long-context premium tier.

Free Tier and Rate Limits

The AI21 Studio free tier in 2026:

• $10 one-time signup credit — roughly 5M tokens on Jamba 1.5 Mini or 1M tokens on Jamba 1.5 Large
• Rate limit: 60 RPM, 60K TPM on the free tier
• Pay-as-you-go ($5+ top-up): 500 RPM, 100K TPM
• Enterprise (sales-channel): custom limits, dedicated inference, multi-region failover

The $10 free credit is the largest among tier-1 closed-source providers — GPT-5, Claude 4.5, and Gemini 2.5 all offer $5 or less. For prototyping or evaluation, the AI21 free credit is workable for a weekend of testing both Mini and Large.

Uptime in 2026 has been consistent at 99.9%+ per the public status page. The most notable incident was a 3-hour partial outage on 2026-04-22 caused by a regional GPU provisioning issue in the AWS US-East-1 deployment; AI21 rerouted traffic to the Azure AI Foundry deployment within 30 minutes and the issue did not recur.

Cloud Marketplace Distribution: Bedrock, Azure AI, NIM

A feature worth highlighting: AI21 Jamba is also available through three major cloud marketplaces, which means teams can buy inference on the same AWS or Azure invoice they already pay.

• AWS Bedrock: Available in us-east-1, us-west-2, eu-west-1. Pricing is the same as direct AI21 Studio, but the bill comes from AWS.
• Azure AI Foundry: Available in East US, West Europe, Southeast Asia. Same pricing.
• NVIDIA NIM: Available as a self-hosted container for on-prem deployment, priced per GPU-hour rather than per token. Useful for teams with strict data-residency requirements.

For enterprise procurement teams, the marketplace distribution is often the deciding factor — being able to buy AI21 inference through the existing AWS or Azure contract is significantly easier than opening a new vendor relationship.

Pros and Cons

Pros

• ✅ Jamba 1.5 Large offers 256K native context at $2/M — cheaper than Claude 4.5 or GPT-5 with the same window
• ✅ Jamba 1.5 Mini at $0.20/M is price-competitive with GPT-4o-mini and Llama 3.1 8B
• ✅ OpenAI-compatible API — no SDK rewrite, works with LangChain/LlamaIndex/vLLM
• ✅ Mamba-transformer hybrid delivers memory efficiency for long-context workloads
• ✅ Distributed through AWS Bedrock, Azure AI Foundry, NVIDIA NIM (multi-cloud procurement)
• ✅ RAG engine v2 — managed document retrieval for teams without a vector DB
• ✅ $10 free credit (largest among tier-1 closed-source providers)

Cons

• ❌ Narrow model lineup (5 Jamba variants only, no Llama / Qwen / DeepSeek rehosting)
• ❌ No China-direct endpoint — proxy required for mainland access
• ❌ No batch API, no cache-read discount
• ❌ Function calling reliability is weaker than OpenAI / Anthropic tier-1
• ❌ Third-party benchmark coverage (MMLU-Pro, GPQA, HumanEval+) is thinner than top closed-source models
• ❌ Jamba 1.5 Large is slower than frontier transformer models on raw tok/s

Use Case Recommendations

Use Case	Recommended Jamba Model	Why
Long-document Q&A (legal, scientific)	jamba-1.5-large	256K context, $2/M, single-request fits whole document
Long-document summarization	jamba-1.5-large	256K context, high recall on mid-context details
Production chatbot (general)	jamba-1.5-mini	$0.20/M, 12B active, competitive with GPT-4o-mini
Code review (whole repo)	jamba-1.5-large	256K fits medium codebase, SSM handles long context efficiently
RAG with managed document store	jamba-1.5-mini + RAG engine	AI21 RAG engine + Mini is the simplest setup
Multi-turn agent with tools	jamba-1.5-mini	OpenAI-compatible, but validate tool-call output
Function calling production	gpt-5 or claude-4.5	Better tool-use reliability than Jamba
Cost-sensitive bulk inference	jamba-1.5-mini	Cheapest tier-1 closed-source model
Strict data residency	NVIDIA NIM self-hosted	Run Jamba in your own data center

Comparison: Jamba vs Llama 3.3 vs GPT-5 vs Claude 4.5

Provider	Best For	Context	Entry Price	OpenAI-Compatible	China Access
AI21 Jamba	256K long context, SSM efficiency	256K	$0.20/M (Mini)	✅	❌ Proxy
OpenAI GPT-5	Top-tier reasoning, tool use	1M (limited)	$5.00/M (5)	✅	❌ Proxy
Anthropic Claude 4.5	Long context, careful reasoning	1M	$3.00/M (Sonnet)	✅	❌ Proxy
Meta Llama 3.3 70B (via Together)	Open-source flexibility, 128K context	128K	$0.39-$0.88/M	✅	❌ Proxy
Google Gemini 2.5 Pro	Multimodal, 2M context	2M	$1.25/M	✅	❌ Proxy
FreeModel	Multi-vendor aggregator, China-direct	Varies	Varies	✅	✅ Direct

For a team that needs 256K context at the lowest price-per-token, AI21 Jamba 1.5 Mini is currently the most direct path. For teams that also need top-tier reasoning and tool-use reliability, GPT-5 or Claude 4.5 wins regardless of price. For Chinese teams, no AI21 endpoint is available — FreeModel or SiliconFlow is the answer.

FAQ

Q: What is the difference between Jamba 1.5 Mini and Jamba 1.5 Large?

A: Both are SSM-Transformer hybrid models with 256K context. Jamba 1.5 Mini is 52B total / 12B active parameters at $0.20/M tokens — the price-performance default. Jamba 1.5 Large is 398B total / 94B active at $2.00/M — for workloads that need the larger active parameter count (complex reasoning, long-context recall). Both share the same architecture and the same 256K native context.

Q: Does Jamba support function calling / tool use?

A: Yes — the chat-completions endpoint accepts a tools array in the OpenAI shape, and the response includes a structured tool_calls field. The output format is compatible with OpenAI's, but for production tool-use pipelines with complex schemas or multi-step reasoning, Jamba's tool-use reliability is weaker than OpenAI GPT-5 or Anthropic Claude 4.5. For simple single-tool workflows the difference is negligible.

Q: Can I use Jamba from inside mainland China without a proxy?

A: No — AI21 does not operate a China-direct endpoint. All inference runs from US or EU data centers, and traffic from China needs a stable proxy or VPN. For teams that need China-direct access, an OpenAI-compatible aggregator with mainland endpoints (like FreeModel at freemodel.dev) is the alternative.

Q: How does the SSM-Transformer hybrid compare to a pure-transformer model at 256K context?

A: The Mamba blocks process most of the context in linear time/memory, which keeps the GPU memory cost roughly half of what a pure-transformer model would need at the same context length. The transformer blocks (about 1/8 of the layers in Jamba 1.5) recover the in-context learning ability that pure SSMs lose. In benchmarks on long-document Q&A and multi-hop reasoning, Jamba 1.5 Large matches or beats pure-transformer models of similar active parameter count on long-context tasks.

Q: How does AI21 Studio pricing compare to running Jamba on my own GPUs?

A: At under ~5M tokens/day on Jamba 1.5 Mini, AI21 Studio is cheaper than renting an A100 80GB instance (~$1,500/month retail). Above 50M tokens/day on Mini or above 5M tokens/day on Large, dedicated inference on H100 or H200 becomes cost-competitive. The NVIDIA NIM container option lets you run Jamba in your own data center at ~$1.99/hr per H100 if you already have the hardware.

Q: Does AI21 keep training data from API requests?

A: Per the published privacy policy, request data is stored for 30 days for abuse monitoring and then deleted. The data is not used to train new models unless explicit opt-in is given. Enterprise customers can negotiate a data-residency clause (data stays in their chosen region) via the sales channel, or self-host Jamba via NVIDIA NIM for full data sovereignty.

Q: Is the Jamba API the same on AWS Bedrock and Azure AI Foundry as the direct AI21 Studio endpoint?

A: The model behavior is identical. The API shape is the same OpenAI-compatible /v1/chat/completions endpoint. The main differences are billing (AWS invoice vs Azure invoice vs AI21 invoice), regional availability (Bedrock has us-east-1/us-west-2/eu-west-1, Azure has East US/West Europe/Southeast Asia), and rate limit defaults (cloud marketplaces often have lower per-account defaults — request a quota increase if needed).

Q: How does AI21 Jamba compare to FreeModel for a multi-vendor setup?

A: FreeModel is a multi-vendor aggregator that exposes OpenAI/Anthropic/Google through one key, with China-direct access. AI21 Jamba is a single-vendor model family with the SSM-transformer advantage. A common pattern is AI21 Jamba for long-context workloads (256K) plus FreeModel for OpenAI/Anthropic access and moderation routing — FreeModel can be signed up at freemodel.dev/invite/FRE-7a3b6220 for a managed multi-vendor setup.

Conclusion

For teams building production systems on LLMs in 2026 that need long context at a reasonable cost, AI21 Jamba is a strong option worth evaluating. The 256K native context solves the chunking-and-retrieval pain that 32K-128K models force on long-document workflows. The $0.20/M Mini price is competitive with GPT-4o-mini. The OpenAI-compatible API removes integration friction. And the Bedrock / Azure AI / NIM distribution means enterprise procurement teams don't have to open a new vendor relationship.

The decision tree for picking a model in 2026:

• Need 256K native context at the lowest price → AI21 Jamba 1.5 Mini
• Need 256K native context with frontier-tier quality → AI21 Jamba 1.5 Large
• Need top-tier reasoning and tool-use reliability → GPT-5 or Claude 4.5
• Need open-source flexibility (Llama / Qwen / DeepSeek) → Together AI or Novita AI
• Need China-direct access → FreeModel or SiliconFlow
• Need multimodal (image, audio, video) → Gemini 2.5 Pro or GPT-5

If your team is already using Jamba and wants a backup provider for OpenAI/Anthropic access or moderation routing, FreeModel is a natural complement — it bundles moderation routing, OpenAI-compatible APIs, and direct access to multiple vendors behind one key. Sign up at freemodel.dev/invite/FRE-7a3b6220 to get started with the free tier.

Comparison Table (Final)

Provider	Pricing Model	Best For	China Access
AI21 Jamba	$0.20-$2.00/M input, same for output	256K context, SSM efficiency	❌ Proxy required
OpenAI GPT-5	$5.00-$15.00/M	Top-tier reasoning, tool use	❌ Proxy required
Anthropic Claude 4.5	$3.00-$15.00/M	Long context, careful reasoning	❌ Proxy required
Meta Llama 3.3 70B	$0.39-$0.88/M	Open-source, 128K context	❌ Proxy required
Google Gemini 2.5 Pro	$1.25-$5.00/M	Multimodal, 2M context	❌ Proxy required
FreeModel	Varies by model	Multi-vendor aggregator	✅ Direct