Lepton AI
Listed at https://lepton.ai
💰 Token Pricing
| Type | Price | Note |
|---|---|---|
| Input | Llama 3.3 70B: $0.80/M; Llama 3.1 405B: $3.50/M; Qwen 2.5 72B: $0.80/M; DeepSeek-R1: $2.00/M | per million tokens |
| Output | Same rate as input for most models (symmetric pricing) | per million tokens |
| Cache Read | N/A (serverless GPU does not cache input) | Discounted |
🤖 Supported Models (50)
✨ Pros
- ✓Multi-region data residency: pin inference to us-west-2, us-east-1, eu-west-1, or ap-northeast-1 (data never leaves the chosen AWS region)
- ✓OpenAI-compatible API: switch by changing base_url + api_key, no application-layer refactor
- ✓SOC 2 Type II + GDPR + HIPAA compliant with signed DPA for EU customers
- ✓Serverless per-token billing + Dedicated endpoints from $4.50/hour (cheaper above ~116M tokens/day)
- ✓50+ open-source models (Llama 3.1 405B, Qwen 2.5 72B, DeepSeek-R1, FLUX.1, Whisper V3)
- ✓Dedicated Enterprise tier for single-tenant private deployment (covers FedRAMP/ITAR)
⚠️ Cons
- ×Region is fixed at workspace creation and cannot be migrated later (use multiple workspaces for multi-region)
- ×Several flagship models (Llama 3.1 405B / DeepSeek-R1) not yet available in EU/JP regions
- ×$5 free credit smaller than Groq's daily free quota or GitHub Models' daily limits
- ×Closed models (GPT-5.5, Claude Opus, etc.) not hosted — use OpenRouter for aggregator access
- ×Serverless per-token billing advantage diminishes for sub-5s calls; dedicated only pays off above ~116M tokens/day
- ×China access requires proxy with cross-Pacific latency (no China-direct edge)
🎯 Best For
Teams needing OpenAI-compatible API with DPA / SOC 2 / GDPR compliance for customer security reviews; EU/JP/US cross-region inference workloads; production environments requiring Dedicated GPU SLA
💰 Pricing & Plans
| Plan / Model | Billing Unit | Input | Output | Notes |
|---|---|---|---|---|
| Trial credit | $5 / 30 days | — | — | One-time grant on sign-up, enough to fully evaluate any flagship model |
| Serverless GPU | per 1M tokens | Symmetric | Symmetric | Most open-weight models at $0.80-$3.50/M (see model list below) |
| Llama 3.3 70B | per 1M tokens | $0.80 | $0.80 | Symmetric input/output; Meta flagship open-weight |
| Llama 3.1 405B | per 1M tokens | $3.50 | $3.50 | Largest open-weight Llama; cached input ~$0.35/M |
| Qwen 2.5 72B | per 1M tokens | $0.80 | $0.80 | Alibaba Qwen flagship; strong multilingual |
| DeepSeek-R1 | per 1M tokens | $2.00 | $2.00 | Reasoning model; chains chain-of-thought tokens |
| FLUX.1 [schnell] / [dev] | per image | $0.05 / $0.10 | — | Image generation; [schnell] is 4-step fast, [dev] is higher quality |
| Dedicated H100 endpoint | per GPU-hour | — | — | From $4.50/hour; cheaper above ~116M tokens/day |
| Pro / Enterprise | Custom | — | — | Signed DPA, SOC 2 Type II, HIPAA, FedRAMP/ITAR support |
🔧 API & Developer Experience
- •API compatibility: OpenAI-compatible base_url swap (api.lepton.ai/v1); one-line migration from OpenAI/Anthropic clients
- •leptonai Python SDK: Official `leptonai` client reads endpoint OpenAPI schema and exposes each path as a native Python method (`Client.run()`)
- •`lep` CLI tool: Manage endpoints, batch jobs, dev pods, Ray/Slurm clusters, fine-tuning jobs, storage, secrets from the terminal
- •Workspace regions: Pin inference to us-west-2, us-east-1, eu-west-1, or ap-northeast-1 (Tokyo) at workspace creation; data never leaves region
- •Endpoint types: Serverless per-token + Dedicated GPU ($4.50/hr H100); auto-scale from 0 to N replicas
- •BYOC (Bring Your Own Compute): Connect existing H100/A100/L40S clusters into a Lepton workspace as a node group and run the same `lep` CLI on top
- •Compliance: SOC 2 Type II + GDPR + HIPAA; signed DPA for EU customers; Enterprise tier covers FedRAMP/ITAR single-tenant deployments
- •Trial & billing: $5 free credit valid 30 days; prepaid credits + per-token serverless; Enterprise invoiced monthly
🛠️ Multi-Region GPU Cloud for Production AI
Lepton AI is a Pythonic GPU cloud platform purpose-built for deploying AI workloads with strict data-residency requirements. The platform lets you pin inference to one of four regions — US West/East, EU West, or AP-Northeast (Tokyo) — at workspace creation, so data never crosses borders mid-request. The OpenAI-compatible base_url means existing OpenAI/Anthropic client code drops in with a one-line base_url swap; no application-layer refactor. The leptonai Python SDK goes further by reading each deployed endpoint's OpenAPI schema and exposing its paths as native methods, so the IDE can autocomplete endpoints the same way it would autocomplete a local function. Beyond serverless per-token billing, Dedicated H100 endpoints start at $4.50/hour and become more economical than serverless above ~116M tokens/day, and the BYOC (Bring Your Own Compute) model lets you attach existing H100/A100/L40S clusters to a Lepton workspace and manage them with the same `lep` CLI as native endpoints. After NVIDIA's 2025 acquisition the platform is being rebranded to NVIDIA DGX Cloud Lepton, but the leptonai SDK and the 50+ open-weight model catalog remain accessible at api.lepton.ai.
🌐 China Access & Latency
Lepton AI does not operate any mainland China points of presence, so all inference endpoints resolve through the four overseas regions (us-west-2, us-east-1, eu-west-1, ap-northeast-1 Tokyo). China-based users should expect 200-350 ms of added round-trip latency on the Tokyo edge and 300-500 ms on the US edges; a stable proxy or a Hong Kong/Singapore relay is required in either case. The platform itself is not blocked, and because the API is OpenAI-compatible, existing domestic client code continues to work once the proxy is configured. The `$5 free credit` is enough to evaluate any model end-to-end but smaller than Groq's daily free quota or GitHub Models' daily limits, so deep testing still requires a paid plan. For latency-sensitive production traffic, the practical pattern is to keep Lepton as a quality backstop (Llama 3.1 405B, DeepSeek-R1 reasoning, FLUX.1 image generation) behind a domestic primary provider like GLM or Qwen, and use the Tokyo region as the closest edge. Enterprise customers can deploy Lepton endpoints into their own cloud or VPC to put data residency outside US territory.