Baseten
Listed at https://www.baseten.co
💰 Token Pricing
| Type | Price | Note |
|---|---|---|
| Input | Per-second GPU billing (H100: $1.89/hr, A100: $0.83/hr, L40S: $0.54/hr) | per million tokens |
| Output | Model-dependent (inference time × GPU rate) | per million tokens |
| Cache Read | N/A (no caching layer on hosted inference) | Discounted |
🤖 Supported Models (35)
✨ Pros
- ✓Per-second GPU billing, the most transparent pay-as-you-go among Replicate-style competitors
- ✓Custom model hosting (Truss framework) — deploy any Hugging Face model in one command
- ✓Cold start < 1 second (proprietary warm pool + autoscaling)
- ✓Dedicated Deployments with reserved GPU capacity and SLA commitment
- ✓Supports PyTorch / TensorRT / vLLM inference engines, auto-selects the optimal backend
- ✓MCP server support for Claude/Cursor integration
- ✓No GPU idle fees on self-deployed models since 2025 (advantage over RunPod / Lambda Labs)
⚠️ Cons
- ×No pretrained model marketplace (vs Replicate 200+ ready-to-use models; Baseten is more self-deploy focused)
- ×China access requires stable proxy
- ×Per-second billing advantage diminishes for sub-5s inference calls
- ×Truss framework learning curve (vs direct HuggingFace Inference API)
- ×Dedicated Deployment starting price is higher than bare AWS/GPU instances
🎯 Best For
Self-deploy open-source models (Llama / Qwen / DeepSeek) on production GPUs; mid-to-large teams needing Dedicated SLAs; MCP integration into Claude / Cursor
💰 Pricing & Plans
| Plan / Model | Billing Unit | Input | Cache Input | Output / Rate |
|---|---|---|---|---|
| Basic | $0/month, pay as you go | — | — | Model APIs + Dedicated Deployments |
| GLM-5.3 | per 1M tokens | $1.40 | $0.14 | $4.40 |
| DeepSeek V4 Pro 0813 | per 1M tokens | $1.32 | $0.132 | $3.96 |
| DeepSeek-V4-Flash-0731 | per 1M tokens | $0.13 | $0.028 | $0.26 |
| Kimi K3 | per 1M tokens | $3.00 | $0.30 | $15.00 |
| NVIDIA Nemotron 3 Ultra | per 1M tokens | $0.60 | $0.12 | $2.40 |
| GPT OSS 120B | per 1M tokens | $0.10 | — | $0.50 |
| A100 80GB (Dedicated) | per minute of GPU time | — | — | $0.06667/min (~$4.00/hr) |
| H100 80GB (Dedicated) | per minute of GPU time | — | — | $0.10833/min (~$6.50/hr) |
| B200 180GB (Dedicated) | per minute of GPU time | — | — | $0.16633/min (~$9.98/hr) |
| Pro / Enterprise | volume discounts, custom SLA | — | — | Get a quote |
🔧 API & Developer Experience
- •API compatibility: OpenAI- and Anthropic-compatible endpoints at inference.baseten.co/v1
- •Coding agents: Native support for Claude Code, Codex CLI, Pi, Droid, OpenCode via BASETEN_API_KEY or Baseten Switch
- •MCP servers: Official workspace MCP (api.baseten.co/mcp) + docs MCP for agent-driven deployment
- •Prompt caching: Automatic KV-cache discount on reused prefixes; x-session-affinity pins a conversation to one replica
- •Custom models: Package and serve any model in any framework with Truss, Baseten's open-source standard
- •Rate limits by tier: Basic 15 RPM / 100K TPM → Basic verified 120/500K → Pro 120/1M → Enterprise custom
- •Trial & billing: Free credits on new accounts; monthly workspace budget with email alerts and optional enforcement
⚡ Model APIs & the Baseten Inference Stack
Baseten began as a self-deployment platform, but its Model APIs now ship a pre-optimized catalog of frontier open-weight models — GLM-5.3, DeepSeek V4 Pro, Kimi K3, NVIDIA Nemotron 3 Ultra, GPT OSS 120B and Inkling — all running on the Baseten Inference Stack with OpenAI- and Anthropic-compatible endpoints. Pricing is per million tokens with three columns: input, cache input and output, so a workload that reuses a long system prompt pays $0.14 instead of $1.40 per million tokens on GLM-5.3. Caching is automatic, and session affinity pins a conversation to a single replica to keep its prefix warm. Vision, audio and extended reasoning are exposed through a documented feature-support matrix rather than hidden flags. Teams that outgrow token pricing can move the same model onto Dedicated Deployments, billed per minute of GPU time down to a T4 or up to a B200.
🌐 China Access & Latency
Baseten has no mainland China points of presence. Inference endpoints resolve through US regions, so users inside China typically see 200-400 ms added round-trip latency and should expect to route through a stable proxy or a Hong Kong/Singapore relay. The platform itself is not blocked, and the OpenAI- and Anthropic-compatible surface means existing China-side client code works unchanged once the network path is sorted. For latency-sensitive production traffic, the practical pattern is to run Baseten's open-weight Model APIs as a quality fallback behind a domestic provider such as GLM or Qwen, or to pair a China-adjacent relay with Baseten Switch so coding agents keep a single credential. Enterprise customers can deploy into their own cloud or VPC, which is the only way to pin data residency outside US regions — Baseten advertises custom global regions on that tier.