Case Study · 14 min read

GPT-6 Astra in Production: Perplexity + Devin

Two production case studies of GPT-6 Astra, one week apart: Perplexity running it on end-to-end systems and Cognition Devin running it as the lead in Fusion. Benchmarks, prompts, and pricing math.

Two Production Case Studies, One Week, One Model

The week of September 11, 2026 produced two unusually candid third-party accounts of how GPT-6 Astra behaves once it is wired into a real engineering workflow. On September 11, the team behind Devin at Cognition published a long post on the architecture of Fusion, a lead-plus-sidekick harness in which Astra does the cheap half of every coding task. Three days later, on September 14, OpenAI's news feed published a vendor-written post on Perplexity's use of Astra, anchored on a quote from Perplexity co-founder Johnny Ho.

Both posts have a striking common feature: neither team is selling Astra. They are explaining what they are doing with it. That is a different kind of evidence than a benchmark table, and it is the kind most API buyers cannot get from the vendor's own documentation. This article pulls the two accounts side by side, extracts the patterns, and turns them into the four or five things a developer actually has to think about before routing a real workload to Astra.

Perplexity: Astra Owns the Loop

Perplexity's account is the easier of the two to summarize, and the more aggressive of the two about what Astra is allowed to do. Ho describes Astra as a system that handles tasks once reserved for human staff, and the categories he names are concrete: drafting internal and external communications, editing real-world systems, monitoring production software, and writing test programs that simulate realistic service responses.

The line that matters for an API team is the quote at the end. We're actually able to trust it with full end-to-end systems and check in on it much less frequently than previous generations of models. That is not a benchmark number. It is a change in operating posture: less frequent human checkpoints, more of the loop owned by the model, and a higher bar for what counts as supervision-worthy work.

The second-order claim Ho makes is that this loops back into the product. Better code from GPT-6 Astra translates almost directly into a sharper search product for Perplexity. The framing matters because it is the same shape as the September 12 prompting guidance from OpenAI engineer Eric Provencher: improvements in Astra's coding ability do not stay contained in developer tools, they ripple outward into whatever product the model touches. Perplexity is the first named case study of that ripple being visible in a production consumer product.

The four production categories

  1. Communications — drafting internal and external messages. The implicit shift is from "Astra proposes copy, a human edits" to "Astra ships copy, a human reviews on a schedule."
  2. Software edits — real changes to real code in the production codebase, not draft branches or sandboxed clones.
  3. Production monitoring — observing running services and responding to anomalies. The cost of a bad action here is bounded, but the cost of slow action is not.
  4. Test programs that simulate realistic service responses — the most novel of the four. Perplexity uses Astra to build mock services (a language model API, a connector) so that real application code can be tested against simulated realistic responses. This is the loop Ho specifically calls out as one of the model's most useful applications, because manual testing does not scale with the pace of software changes.

The through-line across all four is the same: each is a task where the model has more patience than a human, and where the cost of occasional mistakes is bounded by the surrounding code or the surrounding human review schedule. Perplexity is not running Astra unsupervised — it is running Astra with a different, looser cadence of supervision, and the result is a different ratio of human-time to model-time per task.

Cognition: A Harness Around the Model

The Cognition post is a different kind of artifact. It is not a testimonial, it is a release note for a piece of infrastructure. The team has been running Fusion on Devin Cloud for months; the September 11 post announces the same harness in Devin Desktop and the CLI, and it is the first place Cognition has published a real benchmark table comparing frontier models as lead and sidekick.

The Fusion architecture in one paragraph

Fusion runs two parallel agents on the same task, each with its own persistent context and its own tool surface. The lead agent runs on a frontier model — in the published numbers, Fable 5.1 or Astra — and is in charge of the session: it owns the plan, the interpretation of ambiguity, and the review of the sidekick's work. The sidekick is a cheaper model — Cognition's own SWE-2 in the published numbers — and it does the mechanical work: exploring code, implementing changes, running tests, and reporting back.

The two agents do not pass full transcripts between each other. They exchange briefs, results, and feedback. The sidekick never needs the lead's full conversation history to implement a change, and the lead never needs every intermediate tool result to review the work. Each side builds up its own persistent context, and that is what makes the cost math work — every agent gets to take full advantage of prompt caching on the context that does not change.

This is the design choice that matters for an API team. Fusion is not a routing layer in the conventional sense, where the dispatcher looks at the prompt and decides which model to send it to. Fusion is two agents that run together, with the lead retaining final authority and the right to take control back. Cognition's framing is that frontier intelligence is always in charge, and the sidekick exists to keep token spend on mechanical work low without giving up review.

The published benchmark table

The numbers come from a partnership with Artificial Analysis and Vals AI across several coding agent benchmarks, and they are worth reproducing because they are the cleanest public measurement of "frontier model + cheaper sidekick" versus "frontier model alone."

Harness + lead Score Cost per task (USD) Notes
Codex (baseline, no Fusion) 100 $7.47 Reference run on the same benchmark suite.
Devin Fusion, Fable 5.1 + SWE-2 61.7 $4.54 39% lower cost than the Codex baseline.
Devin Fusion, Astra + SWE-2 61.6 Not stated Cost-equivalent pairing with Fable 5.1 as the lead.
Codex + Astra (max) 61.6 Not stated Single-agent run, no sidekick.
Claude Code, Fable 5.1 (max) 62.2 36%+ lower than baseline Single-agent Fable 5.1 run, no sidekick.
Claude Code, Opus 5 (max) 60 Highest in this set Reference run for the more expensive lead.

Three patterns are visible. First, Fusion with Fable 5.1 + SWE-2 is the cost story: 39% cheaper than the Codex baseline while scoring within 0.5 of the best single-agent frontier run. Second, Fusion with Astra + SWE-2 lands at 61.6, essentially tied with Fusion's Fable 5.1 lead, suggesting the harness does the heavy lifting and the lead's specific identity matters less than the architecture. Third, the single-agent frontier runs cluster in a tight band (60 to 62.2), which is the part the post implicitly warns against: the gap between models is small, and the gap between harnesses is large.

The 9% Fable finding

The single most quotable finding in the post is from a model swap Cognition ran internally. They replaced the lead from Opus 4.8 to Fable 5; Fable nominally costs twice as much per token. But with the same sidekick, Fable-led sessions cost 9% less on average, while scoring higher on FrontierCode. The mechanism is in the post: Fable delegated earlier and gave better briefs, while Opus micromanaged the sidekick and redid much of its work.

That is the same shape as the Perplexity story. The work that matters is not the model's per-token cost. It is how much unnecessary back-and-forth the lead creates for the sidekick, and how much redundant work the lead does because it does not trust the sidekick's output. A model that delegates cleanly is cheaper to run, even when its per-token price is higher.

The Shared Pattern: Where Astra Adds Value

Two independent teams, both running Astra in production, both making the same observation about how it changes the operating model. The pattern is not "Astra is cheaper" — its per-token price is $10 per million input, $50 per million output, $1.00 per million cached input, $12.50 per million cache writes. The pattern is "Astra's higher per-token cost is offset by the work it removes from the lead's loop."

For Perplexity, the work that is removed is human attention. Engineers check in less frequently because the model's edits are correct enough that the review loop can run on a longer cadence. The economics for an API team are direct: if you are paying a developer to review model output and the review is the bottleneck, a model that needs less frequent review is cheaper even at a higher per-token rate.

For Cognition, the work that is removed is the lead's redundant verification. Fable-led sessions cost 9% less than Opus-led sessions at twice the per-token price because the lead stops redoing the sidekick's work. The economics for an API team are again direct: a model that delegates cleanly to a cheaper model is cheaper in total, and the per-token price is a misleading input to the cost model.

The 39% Fusion saving and the 9% Fable-over-Opus saving are the same finding in different units. They are both saying: model selection is a system property, not a per-token property, and the system has a lead, a sidekick, and a review loop.

The Context Math Is Different at $1.00 Cached Input

Both case studies lean on prompt caching, and that is the part of Astra's pricing that the existing documentation under-emphasizes. At $1.00 per million cached input tokens and $12.50 per million cache writes, the cost ratio between cached and uncached input is 10:1, and the cost ratio between a cache write and a cache read is 12.5:1 in the other direction. That asymmetry is what makes Fusion's architecture economically coherent.

For a single-agent run, the math is straightforward. Every system prompt, every AGENTS.md, every skill description is either sent at the full $10/M rate or written to cache once and re-read at $1/M. The crossover point for a given prompt's lifetime in a session is around 10 reads; below that, the cache write fee eats the savings. Above that, the cache dominates.

For a two-agent Fusion run, the math is much friendlier to caching. The lead's context is the user-facing session, which is small and changes constantly. The sidekick's context is its own working set, which is also small but is reset every time the lead hands off a new brief. The Fusion post's key claim — that each agent builds up its own persistent context — is what makes the cache hit reliably: each agent's context stays stable across many of its own turns, even as it changes between agents.

For an API team building anything like Fusion, the practical implications are:

  • Stable system prompts are the highest-leverage thing to cache. A 4,000-token system prompt that is re-sent 50 times costs $2.00 uncached or about $0.45 with cache. That is $1.55 saved per 50 requests, before any sidekick is even in the picture.
  • Cache writes are expensive enough that a write-once-per-session prompt is not a free win. At $12.50/M, a 4,000-token cache write costs $0.05 per session. If the session is short enough that the prompt only gets re-read twice, the cache is a net loss. The 10:1 ratio sets a floor of about 10 reads per write for caching to break even.
  • Lead-and-sidekick architectures get more out of caching than single-agent runs because each agent's working context is more stable than a merged single-agent context would be. The Fusion numbers (39% cost reduction) are partly a function of the model choice and partly a function of the cache architecture.

Decision Matrix: Astra as Lead, Sidekick, or Neither

For an API team, the three roles Astra can play in a multi-model workflow are distinct enough to be worth separating.

Role What you are paying for When it pays When it does not
Astra as the only model $10/M input, $50/M output, $1/M cached input Single-call workloads where you want the highest answer quality per call and the call volume is low enough that the per-token price is dominated by other costs. High-volume pipelines or long-context agent runs, where the per-token math is the binding constraint.
Astra as the lead in Fusion Same per-token prices, but a much smaller fraction of the token spend (most of the work is the sidekick's). Multi-step engineering work where review quality matters and a cheap sidekick can carry the mechanical load. Cognition's 39% cost reduction is the headline number. Tasks where the lead is the bottleneck anyway (e.g. planning-heavy work where the sidekick has nothing to do).
Astra as the sidekick The opposite of the lead role. You are buying reliable per-token execution under a more capable lead's direction. Rare. Cognition did not publish a benchmark with Astra as the sidekick, and the implicit finding from the post is that the lead's quality dominates the architecture's quality. Most coding-agent workloads today. A cheaper, more focused model is the better sidekick choice.

The practical reading: the default for an API team today is to start with Astra as the lead in a Fusion-style harness with a sidekick model that is appropriate to the workload. That is the configuration Cognition has been running in production for months, and it is the configuration that Perplexity's reduced-supervision finding points to as well, with the lead in their case being the only agent rather than a pair.

What This Changes About Prompting Guidance

The September 12 OpenAI prompting guidance — the one that says cut broad skill scopes, drop blanket read requirements, revisit approval rules — fits cleanly into the two case studies. Perplexity's reduced check-in cadence is only possible because the model does not stop mid-task on the kind of false-positive approval requests that earlier models needed hard-coded rules to prevent. Cognition's lead-and-sidekick architecture is only viable because the lead's briefs are precise enough that the sidekick can execute them without re-asking.

Both teams are, in different language, doing what the OpenAI prompting guidance describes. Both are running the model in an environment where the surrounding scaffolding is lean. Neither team publishes a copy of the prompts they use, but the existence of two independent production deployments in the same week, on the same model, in roughly the same shape, is the strongest external evidence yet that lean prompting is the default rather than an optimization.

For a developer integrating Astra today, the through-line is short. Use it as a lead, not as an only-agent. Cache the parts of the context that are stable. Audit your AGENTS.md and skill files for rules that were written for an earlier model. Expect the review cadence to drop, and design your logs and your rollback plan for that lower cadence rather than for the per-turn supervision that the previous generation required.

Sources

  • Cognition blog, Introducing Fusion in Devin Desktop & CLI — September 11, 2026 (Fusion lead-and-sidekick architecture, benchmark table with Artificial Analysis and Vals AI, 39% cost reduction with Fable 5.1 + SWE-2 pairing, 9% Fable-over-Opus finding).
  • OpenAI news feed, Perplexity trusts GPT-6 Astra with end-to-end systems — September 14, 2026 (Johnny Ho on communications, software edits, production monitoring, mock-service test programs; reduced check-in cadence; better Astra code → better Perplexity search product).
  • OpenAI news feed, Cognition helps Devin test its own work with GPT-6 Astra — September 11, 2026 (companion vendor-written post that names Astra as a default in Devin's testing flow).
  • OpenAI prompting guidance, GPT-6 Astra needs leaner prompts and fewer guardrails — Eric Provencher, September 12, 2026 (the model-side argument for the environment in which the two case studies are running).
  • apirank, GPT-6 Astra API Pricing 2026 (September verification): $10/M input, $50/M output, $1.00/M cached input, $12.50/M cache writes; 10:1 cached-to-uncached input ratio.

Routing real workloads to GPT-6 Astra?

Astra runs $10/$50 per million input/output with cached input at $1.00 and cache writes at $12.50. Put those numbers next to every other frontier endpoint before you wire it into a Fusion-style harness.

Compare API Pricing →