Bifrost (by Maxim)
Listed at https://www.getmaxim.ai/bifrost
💰 Token Pricing
| Type | Price | Note |
|---|---|---|
| Input | OSS: $0 (Apache-2.0 self-hosted); upstream model token rates pass through with no markup | per million tokens |
| Output | OSS: $0 (Apache-2.0 self-hosted); upstream model token rates pass through with no markup | per million tokens |
🤖 Supported Models (1000)
✨ Pros
- ✓Open-source Apache-2.0 AI gateway in Go — 7,869 ⭐ and 1,176 forks on GitHub (as of 2026-09-08)
- ✓Published benchmarks: 50× faster than LiteLLM at sustained 500 RPS, 68% lower memory footprint
- ✓<100 µs added latency at 5,000 RPS; single OpenAI-compatible /v1 endpoint; drop-in base_url swap
- ✓23+ providers and 1,000+ models: OpenAI / Anthropic / Bedrock / Vertex / Azure / Cohere / Mistral / Groq / Cerebras / Ollama / HuggingFace / Replicate + self-hosted
- ✓Built-in automatic failover, load balancing, semantic caching, virtual-key budget control, MCP gateway, OTel/Prometheus observability
- ✓14-day Enterprise free trial plus VPC / on-prem / air-gapped deployment, SAML/OIDC SSO, Vault integration, guardrails
⚠️ Cons
- ×OSS tier does not include SAML SSO, Vault integration, or guardrails — Enterprise-only features
- ×No public hosted SaaS endpoint — every customer must self-host in their own VPC, on-prem, or air-gapped environment
- ×Upstream model prices pass through with no markup, but the OSS tier has no unified billing/reconciliation dashboard (Enterprise tier adds this)
- ×Newer ecosystem than Portkey or Helicone (repo created 2025-03); third-party integrations (Vercel AI SDK / LangChain) need manual configuration
- ×No official mainland-China mirror; for data-residency you must self-host on a domestic cloud and route to a self-hosted upstream (Ollama / vLLM)
🎯 Best For
Teams that need an OSS-first, self-hosted, enterprise-grade AI gateway; cost-sensitive workloads that want zero-markup transparent pass-through of upstream token prices; ops teams that need MCP gateway + OTel observability piped straight into Datadog / Honeycomb; and regulated finance, healthcare, or government customers who must keep every prompt byte inside their own VPC / on-prem / air-gapped environment.
💰 Pricing & Plans
| Tier | Cost | Self-Hosting | Key Features |
|---|---|---|---|
| OSS (open source) | $0 — Apache-2.0 | Docker / K8s / Go binary / npx | Routing across 23+ providers, 1,000+ models, automatic failover, load balancing, semantic caching, virtual keys, MCP gateway, OTel/Prometheus observability, budget & rate limits, built-in web UI at :8080 |
| Enterprise Free Trial | $0 for 14 days | VPC / on-prem / air-gapped deployment | Everything in OSS plus guardrails, adaptive load balancing, cluster mode, enterprise SSO (SAML/OIDC), Vault support, MCP with federated auth, log exports, audit logs |
| Enterprise Production | Custom — book a demo | VPC / on-prem / air-gapped | SLA-backed support, custom security controls, governance, in-VPC deployments, cluster mode at scale |
🔧 API & Developer Experience
- •OpenAI-Compatible Endpoint: Single OpenAI-compatible /v1/chat/completions endpoint at http://localhost:8080 (self-hosted) — drop-in for any OpenAI SDK by swapping base_url. Anthropic, Vercel AI SDK, and LangChain also work with a one-line base-url change.
- •23+ Providers Out of the Box: OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure OpenAI, Cohere, Mistral, Groq, Cerebras, Ollama, Hugging Face, Replicate — and custom self-deployed models — all reachable through one consistent interface.
- •1,000+ Models Without Per-Provider SDKs: Bifrost normalises model naming (e.g. openai/gpt-4o-mini) and the request/response format across every upstream provider, so the same client code routes to any of the 1,000+ models without provider-specific SDK juggling.
- •Automatic Failover & Load Balancing: Configurable per-route failover across providers and weights for weighted load balancing. If OpenAI is degraded for you, Anthropic takes over without client retry glue. Verified in the 5,000 RPS benchmark.
- •Semantic Caching: Built-in semantic cache reduces token spend on repeated or near-duplicate prompts — pairs with virtual keys for per-team/per-customer budget isolation and rate limits.
- •MCP Gateway + OTel Observability: Bifrost itself is an MCP gateway (mcp-gateway, mcp-client, mcp-server tags) so you can route MCP tool calls through the same governance layer. OTel-compatible metrics/traces plug straight into Datadog, Honeycomb, Grafana, Prometheus.
⚡ Why Bifrost: 50× LiteLLM at 5k RPS, <100 µs Overhead
Bifrost is the first OSS AI gateway that treats the gateway itself as a hot-path performance problem. Maxim's published benchmarks put it at 50× the throughput of LiteLLM in sustained 500 RPS, with 68% lower memory, and under 100 microseconds of added latency at 5,000 RPS. The data plane is Go; ships as a single binary, Docker image, npx package, or Kubernetes manifest — from zero to a local gateway with web UI in under a minute (`npx -y @maximhq/bifrost`, then http://localhost:8080). Trade-offs are typical for an OSS-first gateway: no SAML SSO, Vault, or guardrails (Enterprise-only), and no public hosted SaaS endpoint — you run Bifrost yourself in your own VPC, on-prem, or air-gapped, which is why regulated teams pick it over OpenRouter or Helicone Cloud.
🌐 Regional Availability & Latency
Because Bifrost is self-hosted software, not a hosted API, 'regional availability' is really about where you choose to deploy it and which upstream providers you route to. Run it on AWS US-East-1 and OpenAI / Anthropic / Vertex / Bedrock respond at native latencies; deploy on AWS eu-central-1 and EU-resident providers (Mistral, Azure EU) get the lowest TTFT. Bifrost's own overhead is sub-100 µs at 5,000 RPS regardless of region, so the latency budget is dominated by the upstream model API, not the gateway. Mainland China teams can deploy Bifrost on a domestic cloud (Alibaba, Huawei Cloud) with a self-hosted Ollama or vLLM endpoint as the only upstream — the cleanest way to keep every prompt byte on-shore while using the same gateway interface as global developers.