NVIDIA NIM API 2026: 80+ Models on the World's Most Optimized Inference Stack

Introduction: The Unusual Business Model

NVIDIA enters the AI API market in a unique position: it makes the hardware that virtually every other provider depends on. The NIM (NVIDIA Inference Microservice) platform was originally launched in 2024 as a way to package optimized inference containers for enterprise deployment. The cloud API at build.nvidia.com arrived later as a try-before-you-buy path into the broader NVIDIA ecosystem.

What makes NVIDIA NIM structurally different from Groq, Together AI, or DeepInfra is the business model. NVIDIA does not run inference endpoints itself — it partners with existing providers (Bitdeer AI, Deep Infra, Together AI, DigitalOcean, GMI Cloud) who host the models on NVIDIA DGX infrastructure and set their own prices. Build.nvidia.com is the discovery and routing layer: you select a model, choose a partner endpoint, and get redirected to that partner for billing.

This approach means NVIDIA's model catalog is simultaneously broad (80+ models) and fragmented (each partner has different pricing, rate limits, and SLA commitments). For the AI engineering team evaluating inference options, the question is not "should I use NVIDIA NIM?" but "which of the partner endpoints that host NVIDIA-optimized models should I use, and is the Nemotron architecture worth the extra attention?"

What NVIDIA NIM Actually Hosts

The model catalog on build.nvidia.com falls into three tiers:

1. NVIDIA's Own Nemotron-3 Series

The Nemotron-3 family is NVIDIA's own model line, built on a novel LatentMoE architecture that combines three mechanisms within a single transformer:

• Mamba-2 state-space layers for memory-efficient long-context processing
• Mixture-of-Experts routing for parameter efficiency
• Standard attention heads for quality on precision-critical tasks

The lineup covers every tier:

Model	Total Params	Active Per Token	Best For
Nemotron-3 Ultra 550B	550B	55B	Frontier reasoning, coding, long-context
Nemotron-3 Super 120B	120B	12B	Production chat, enterprise summarization
Nemotron-3 Nano 30B	30B	3B	Lightweight classification, fast tasks
Nemotron-Mini 4B	4B	4B	Edge/embedded, low-cost bulk inference
Llama-3.3-Nemotron-Super-49B v1.5	49B	49B	NVIDIA-optimized Llama 3.3 70B derivative
Llama-3.1-Nemotron-Ultra-253B v1	253B	253B	NVIDIA-tuned Llama 3.1 405B derivative

2. Third-Party Open-Source Models

Beyond NVIDIA's own models, the build.nvidia.com catalog includes 50+ third-party models hosted by partners:

• Llama family: Llama 3.1 8B/70B/405B, Llama 3.2 1B/3B/11B/90B, Llama 3.3 70B, Llama 4 Maverick 17B
• Qwen family: Qwen2.5-Coder-32B, Qwen3-5-122B, Qwen3-Coder-480B, Qwen3-Next-80B, QwQ-32B, Qwen3.5-397B
• Mistral family: Mistral-Nemotron, Mixtral 8x7B/8x22B, Mistral Large 3, Mistral Medium 3.5, Ministral 14B
• DeepSeek: DeepSeek V4 Flash, DeepSeek V4 Pro
• Google Gemma: Gemma 2/3/4, CodeGemma, DiffusionGemma
• Microsoft Phi: Phi-4 Mini, Phi-4 Mini Flash Reasoning, Phi-4 Multimodal
• Chinese models: Kimi K2/K2.5/K2.6, Step 3.5/3.7 Flash, ByteDance Seed OSS, GLM4.7/GLM5.1, MiniMax M2.5/M2.7/M3

3. Embedding, Vision, and Specialized Models

NVIDIA also hosts embedding models (NemoRetriever 300M/1B, NV-Embed, Llama-Nemotron-Embed-1B), reranking models, content safety models (NeMoGuard for jailbreak detection, topic control, PII detection), vision models (FLUX.1, SD3, SDXL), and healthcare models (Evo-2, AlphaFold2, ESMFold).

API Surface: Standard OpenAI Shape, Partner Routing

The cloud API is straightforward for an OpenAI-compatible endpoint. Base URL is https://integrate.api.nvidia.com/v1, and the /v1/chat/completions endpoint accepts the standard OpenAI request body. Streaming, JSON mode, and function calling are all supported.

from openai import OpenAI

client = OpenAI(
    base_url="https://integrate.api.nvidia.com/v1",
    api_key="nvapi-YOUR_API_KEY"
)

completion = client.chat.completions.create(
    model="nvidia/nemotron-3-ultra-550b-a55b",
    messages=[{"role": "user", "content": "Explain LatentMoE in 3 sentences."}],
    max_tokens=512
)

print(completion.choices[0].message.content)

The key difference from a standard OpenAI-compatible provider: model names use the full NVIDIA catalog path (e.g., nvidia/nemotron-3-ultra-550b-a55b).

Pricing: Fragmented by Design

The most unusual aspect of NVIDIA NIM is that pricing varies by partner endpoint for the same model. Taking Nemotron-3 Ultra 550B:

Partner Endpoint	Input (per 1M tokens)	Output (per 1M tokens)	Notes
Deep Infra	$0.50	$2.50	Dedicated GPU from $0.01/min
Together AI	$0.60	$3.60	Dedicated GPU from $0.80/min
Bitdeer AI	$0.80	$2.60	Standard endpoint
DigitalOcean	$0.90	$1.70	H100 inference, all-in-one platform
GMI Cloud	$0.80	$2.60	Standard endpoint

For smaller models, pricing is naturally lower. Llama-3.3-Nemotron-Super-49B is typically $0.35-$0.60/M input across partners. The Nemotron-Mini 4B is available for under $0.05/M.

For comparison against providers that host open-source models with unified billing:

• Together AI Llama 3.3 70B: $0.88/$0.88 per 1M tokens (unified billing)
• Deep Infra Llama 3.3 70B: $0.35/$0.50 per 1M tokens (unified billing)
• Groq Llama 3.3 70B: $0.59/$0.79 per 1M tokens (LPU-accelerated)
• Novita AI Llama 3.3 70B: $0.39/$0.39 per 1M tokens (China-direct)

Speed: Undisclosed but Confident

NVIDIA does not publish per-model token-per-second benchmarks for its cloud API endpoints. What is known: the NIM inference stack runs on TensorRT-LLM, NVIDIA's own inference optimization library, which is widely recognized as one of the most performant GPU inference engines available. In practice, the partner endpoints use different GPU tiers (H100 vs A100 vs DGX systems), so speed will vary by partner.

Pros and Cons

Pros

• ✅ Exclusive Nemotron-3 series with LatentMoE architecture
• ✅ Llama-Nemotron optimized models — NVIDIA-tuned Llama checkpoints
• ✅ OpenAI-compatible API, zero SDK rewrite, free prototyping without credit card
• ✅ 80+ model catalog covering Llama, Qwen, Mistral, DeepSeek, Kimi, GLM, Gemma, Phi
• ✅ Free tier gives real API access for development
• ✅ TensorRT-LLM inference engine is top-tier for GPU utilization efficiency
• ✅ Diverse catalog includes embeddings, rerankers, vision, safety, healthcare

Cons

• ❌ No unified billing — each partner endpoint requires separate account and payment
• ❌ China access requires stable proxy; no ICP-filed infrastructure or RMB billing
• ❌ Speed benchmarks not publicly published; latency depends on partner infra
• ❌ Third-party model catalog overlaps heavily with simpler-billing providers
• ❌ No dedicated SLAs at the free/prototyping tier

Use Case Recommendations

Use Case	Recommended Model	Why
Frontier reasoning / coding	Nemotron-3 Ultra 550B (Deep Infra)	$0.50/M input, exclusive quality
Production chat (English)	Llama-3.3-Nemotron-Super-49B	NVIDIA-tuned Llama, competitive price
Lightweight classification	Nemotron-3 Nano 30B	3B active params, cost-efficient
Bulk embedding	NemoRetriever-300M v2	NVIDIA's retrieval embedding
Content safety filtering	NeMoGuard Jailbreak Detect	NVIDIA safety-optimized model
Self-hosted production	Downloadable NIM container	Deploy on your own GPU stack

Comparison: NVIDIA NIM vs Key Competitors

Provider	Best For	China Access	Billing	Models
NVIDIA NIM	Exclusive Nemotron models + 80 open-source	❌ Proxy	Per-partner	80+
Groq	LPU speed on 7 models	❌ Proxy	Unified	7
Together AI	200+ open-source, US/EU serving	❌ Proxy	Unified	200+
Deep Infra	50+ open-source at lowest price	❌ Proxy	Unified	50+
Novita AI	200+ open-source, China-direct	✅ Direct	Unified	200+
FreeModel	Multi-provider aggregator with routing	✅ Direct	Unified	50+

FAQ

Q: Is NVIDIA NIM self-hosted containers or a cloud API?

A: Both. NVIDIA offers (1) downloadable NIM containers for self-hosted deployment and (2) the cloud API at build.nvidia.com with free prototyping endpoints. This review covers the cloud API.

Q: How does pricing work when models are hosted by partners?

A: Pricing is set per-partner. Nemotron-3 Ultra 550B ranges from $0.50/$2.50 per 1M tokens on Deep Infra to $0.90/$1.70 on DigitalOcean. Each partner requires separate signup and billing.

Q: Can I use NVIDIA NIM from inside China?

A: Unlikely without a proxy. build.nvidia.com is US-based with export control restrictions. For China-direct alternatives: SiliconFlow or Novita AI.

Q: What makes Nemotron-3 different?

A: LatentMoE architecture combining Mamba-2 state-space layers, MoE routing, and standard attention. Ultra 550B has 550B total params but only 55B active per token.

Q: Is the API OpenAI-compatible?

A: Yes. Uses standard /v1/chat/completions. Use OpenAI Python SDK with base_url change. Streaming, JSON mode, and function calling all supported.

Q: How does NVIDIA NIM compare to FreeModel?

A: NVIDIA NIM is a single-provider catalog of 80+ models via partner endpoints. FreeModel at freemodel.dev/invite/FRE-7a3b6220 is a multi-provider aggregator bundling OpenAI, Anthropic, DeepSeek behind one key with moderation routing and China-direct access.

Conclusion

NVIDIA NIM occupies a unique position in the AI API market. It is not trying to beat Groq on speed, Together AI on model breadth, or Novita AI on China access. Instead, it is a gateway into the NVIDIA ecosystem — a way to try NVIDIA's exclusive Nemotron-3 models before committing to self-hosted NIM containers on your own GPU infrastructure.

The decision tree for picking a provider in 2026:

• Need NVIDIA's exclusive Nemotron-3 architecture for evaluation → NVIDIA NIM (free tier)
• Need the Nemotron-3 Ultra 550B at the cheapest rate → NVIDIA NIM via Deep Infra
• Need to evaluate before deploying self-hosted NIM → NVIDIA NIM cloud API
• Need 200+ open-source models with unified billing → Together AI or Novita AI
• Need China-direct multi-provider routing → FreeModel
• Need the fastest inference speed on a specific model → Groq or Cerebras

If your team is experimenting with NVIDIA's Nemotron models and wants a production fallback for multi-provider routing from China, FreeModel complements the setup — it bundles moderation routing, OpenAI/Anthropic access, and direct China connectivity behind one API key. Sign up at freemodel.dev/invite/FRE-7a3b6220.

Comparison Table (Final)

Provider	Pricing Model	Best For	China Access
NVIDIA NIM	Partner-dependent pricing	Exclusive Nemotron models + open-source catalog	❌ Proxy
Groq	$0.59-$0.79/M for Llama 3.3 70B	LPU-optimized speed on 7 models	❌ Proxy
Together AI	$0.18-$0.88/M per model	200+ open-source, US/EU serving	❌ Proxy
Deep Infra	$0.04-$0.50/M for open-source	50+ models at lowest price	❌ Proxy
Novita AI	$0.06-$1.52/M for 200+ models	200+ open-source, China-direct	✅ Direct
FreeModel	Varies by model	Multi-provider aggregator	✅ Direct