Anyscale
Listed at https://endpoints.anyscale.com
💰 Token Pricing
| Type | Price | Note |
|---|---|---|
| Input | Llama 3.3 70B: $0.39/M, DeepSeek-R1: $1/M, Mixtral 8x22B: $0.90/M | per million tokens |
| Output | Llama 3.3 70B: $0.39/M, DeepSeek-R1: $1/M, Mixtral 8x22B: $0.90/M | per million tokens |
🤖 Supported Models (20)
✨ Pros
- ✓Llama 3.3 70B at only $0.39/M tokens, competitive
- ✓Built on Ray distributed framework, high availability
- ✓Unified API interface
- ✓Function Calling support
- ✓Enterprise SLA and support
⚠️ Cons
- ×China requires proxy
- ×Limited model selection (mostly open-source)
- ×UI/UX leans technical
- ×Brand recognition lower than Together AI
- ×Ray ecosystem complexity may impact user experience
🎯 Best For
AI developers needing stable open-source model endpoints; Ray ecosystem users; value-conscious enterprise users
💰 Pricing & Plans
| Model / Resource | Price | Metering | Notes |
|---|---|---|---|
| Llama 3.3 70B | $0.39 / 1M tokens | Per token (in+out) | Competitive open-weights inference via Anyscale Endpoints |
| DeepSeek-R1 | $1.00 / 1M tokens | Per token (in+out) | Reasoning model at a budget price point |
| Mixtral 8x22B | $0.90 / 1M tokens | Per token (in+out) | MoE model for general chat workloads |
| CPU-only instance | $0.0135 / hr | Per accelerator | Cheapest Ray compute tier |
| NVIDIA T4 | $0.5682 / hr | Per accelerator | Entry GPU for small inference workloads |
| NVIDIA A10G | $1.3635 / hr | Per accelerator | Mid-tier GPU, balance of cost and throughput |
| NVIDIA A100 | $4.9591 / hr | Per accelerator | High-memory GPU for larger models and training |
| New user credit | $100 free | One-time | One-time grant on signup |
🔧 API & Developer Experience
- •API Surface: Anyscale Endpoints exposes an OpenAI-compatible HTTP API for serverless inference (chat completions + embeddings). The same surface serves Ray jobs on the broader Anyscale Platform, letting you move between hosted inference and Ray distributed compute without changing stacks.
- •Models: The endpoint catalog is open-weights focused: Llama 3.3 70B, Llama 3.1 405B, Mixtral 8x22B, DeepSeek-R1, CodeLlama 70B and Qwen 2.5 72B. Newer flagship third-party models are served through Nscale's Inference Endpoints ('Run models via API') after the acquisition.
- •Ray Integration: Anyscale's differentiator is Ray — you can scale a single Python script from laptop to a multi-node GPU cluster with @ray.remote. The platform adds managed scheduling, autoscaling and observability on top of Ray for distributed training and batch inference.
- •SDKs & Tooling: First-class Python SDK built on Ray; OpenAI Python SDK works for the endpoints interface by pointing base_url at the Anyscale endpoint. Notebooks and code templates are provided for LLM fine-tuning, batch inference and multimodal workloads.
- •Deployment Options: Hosted (fully managed, no setup) or Bring-Your-Own-Cloud (BYOC) inside your own VPC, on AWS/GCP/Azure or on-prem. Hosted invoices monthly; BYOC can be billed through your cloud marketplace.
- •Authentication: API key + token from the Anyscale console, set as an environment variable for the Python SDK. Per-project keys scope access, and the console supports team members and usage dashboards.
- •Rate Limits & SLA: Enterprise SLA with 24x7 support and unlimited case submissions; lower tiers get business-hours support with a case limit. Usage-based billing with no monthly minimum in the hosted tier.
🧠 Ray & the Nscale Acquisition
Anyscale's defining asset is Ray, the de-facto open-source distributed compute engine for AI, deployed across nearly every major cloud. On July 30, 2026 the NVIDIA-backed UK neocloud Nscale agreed to acquire Anyscale for a reported $1.65 billion, folding Endpoints and the Ray platform into Nscale's full-stack AI cloud (data centers in Norway, the UK, Texas and Portugal). The stated goal is multi-cloud neutrality: Ray workloads can now run on Nscale's own NVIDIA infrastructure and on AWS/GCP/Azure simultaneously, giving teams one Ray-consistent control plane across providers. For developers the practical near-term impact is continuity — the Anyscale API, SDK and pricing remain unchanged — plus a deeper pathway into Nscale's managed Slurm, Kubernetes Service and Inference Endpoints for when a Ray job outgrows the hosted tier.
🌐 Regional Availability & Latency
Anyscale is a US-based company serving the API from hosted regions plus your own cloud (BYOC) VPCs, with latency closest to whichever region you deploy in. There is no mainland China regional endpoint or official China access program, so production access from China requires a stable overseas proxy or an aggregator that fronts the Anyscale API. Under Nscale ownership the footprint expands to Nscale data centers in the Nordics, UK, Texas and Portugal, which shortens latency for EU and US-East workloads and adds a sovereign-AI option for enterprises that need data residency. Teams serving Chinese end users should budget for proxy infrastructure and evaluate a domestic open-weights endpoint (e.g. Alibaba Bailian or Tencent Hunyuan) as a latency-safe fallback for inference.