NVIDIA NIM

💰 Pricing & Plans

🔧 API & Developer Experience

• •API Style: OpenAI-compatible /v1/chat/completions endpoint at https://integrate.api.nvidia.com/v1 — existing OpenAI SDK clients (openai-python, openai-node, LangChain, LlamaIndex) work by swapping base_url and api_key. Streaming supported via stream=true; tool/function calling on chat-tuned Nemotron checkpoints.
• •Auth & Free Tier: Single NVIDIA_API_KEY Bearer token issued from build.nvidia.com; no credit card for the 40 free endpoint models. Free tier carries per-model RPM caps (no fixed monthly credit pool); partner endpoints require a separate API key from each partner's console.
• •Model Catalog (100+ models, 23 Nemotron-branded): Across LLM, vision, embedding, ASR, OCR, guardrails: Nemotron-3 Ultra 550B (a55B), Nemotron-3 Super 120B (a12B), Nemotron-3 Nano 30B (a3B), Nemotron-3.5-Lightning-30B-A3B, Nemotron-3 Nano Omni 30B (reasoning), Llama-3.1-Nemotron-Ultra-253B v1, Llama-3.3-Nemotron-Super-49B v1.5, Llama-3.1-Nemotron-Nano-8B v1, plus Nemotron-Embed-1B, Nemotron-OCR-v1/v2, Nemotron-Parse, Nemotron-ASR-Streaming, Nemotron-VoiceChat, llama-nemotron-rerank-vl-1b-v2, mistral-nemotron, and Nemoguard safety / topic-control / jailbreak-detect.
• •Reasoning Mode: Nemotron-3 Ultra / Super / Nano-Omni checkpoints expose an explicit thinking trace via extra_body={"chat_template_kwargs":{"enable_thinking":True}} and stream chunk.choices[0].delta.reasoning_content — the same chat-completion surface returns both reasoning and final answer, so apps get chain-of-thought for free.
• •Context & Limits: Up to 16384 completion tokens on flagship Nemotron; context windows vary by checkpoint (Nemotron-3 Ultra typically 128K class). Standard OpenAI sampling knobs (temperature, top_p, top_k, frequency/presence penalties) all honored. No fine-grained tool-use schema beyond OpenAI function-calling shape.
• •Self-Hosted Path: Every public NIM ships as a Docker container with NVIDIA AI Enterprise license — one helm/docker run brings up Triton Inference Server + TensorRT-LLM + vLLM-optimized engine on H100 / H200 / L40S / A100 / RTX PRO 6000. Identical API surface to integrate.api.nvidia.com/v1, so the same client code moves between hosted and self-hosted.
• •Inference Engines: Under the hood: TensorRT-LLM for NVIDIA-tuned models, vLLM for community open-source checkpoints, Triton Inference Server as the serving layer. Container GPU matrix documented per model (A100 / H100 / L40S / A10G / A100-PG509-200 visible on the catalog filter).

🧬 Nemotron-3 LatentMoE: Mamba-2 + MoE + Attention Hybrid

What makes Nemotron-3 (Ultra 550B / Super 120B / Nano 30B) genuinely interesting in a sea of plain Llama fine-tunes is the LatentMoE hybrid: each layer mixes a Mamba-2 selective state-space model with sparse Mixture-of-Experts and a smaller share of vanilla attention, instead of stacking attention everywhere. The practical effect is linear-time sequence handling for the bulk of the compute (Mamba-2 scales O(n) in tokens vs attention's O(n²)), with attention kept only where it earns its keep — long-range dependencies, in-context retrieval, and tool-use planning. At inference this translates to noticeably steadier tok/s as prompt length grows past 32K, less KV-cache pressure on H100 / H200 GPUs, and room for the same hardware to serve a 550B-active-parameter model that would otherwise need much larger deployments. NVIDIA complements the architecture with explicit reasoning checkpoints (Nemotron-3 Nano Omni 30B reasoning, plus the enable_thinking flag on Ultra / Super) so the same weights can be steered into either chat or chain-of-thought mode by the caller, without re-hosting.

🌐 China Access & Latency

build.nvidia.com and integrate.api.nvidia.com are both US-hosted, and US export-control rules restrict NVIDIA's top-end accelerators (H100 / H200 / Blackwell-class) from being shipped or fronted from mainland China — the public NIM endpoint is reachable from China only via a stable overseas proxy / VPN / SD-WAN, with typical round-trip latency of 250-450 ms depending on the egress (HK / Tokyo / Singapore). The free endpoint accepts overseas-email registration without credit card, but the service itself logs every prompt and output for product-improvement per the Trial Terms of Service, which is a hard blocker for any confidential code or proprietary data. Practical China-friendly paths: (1) hit Nemotron through an OpenRouter or Together AI partner key (both work behind HK / Tokyo edge nodes and accept overseas cards, with partner-set rate cards often matching NVIDIA's listed tier); (2) self-host the same NIM container inside an Asia-Pacific cloud that is not subject to the export rule (AWS Tokyo, Azure Singapore, GCP Taiwan, or a HK / SG GPU host) — you bring your own license from NVIDIA AI Enterprise and the API surface is identical; (3) use the free build.nvidia.com endpoint purely for evaluation and route all production traffic through one of the above. There is no Alipay / WeChat Pay top-up, no China-domestic pricing tier, and no official NVIDIA mainland channel — every workable route involves an overseas component.