AI Shqipe
valon-hajredini.aishqipe.com/posts/gpt-6-astra-claude-fable-5-1-benchmarks
Valon Hajredini
Subscribe

GPT-6 Astra and Claude Fable 5.1: The Benchmarks Side by Side

OpenAI and Anthropic shipped their frontier models two days apart in September 2026, at exactly the same token price. The benchmark tables say one thing, the independent indexes say another, and the bill depends on how you use them. Here are the numbers, chart by chart.

Anthropic released Claude Fable 5.1 on 1 September 2026. OpenAI answered with GPT-6 Astra on 3 September. Both cost $10 per million input tokens and $50 per million output tokens, both take around a million tokens of context and return up to 128K, and both expose the same five effort levels from low to max. That is where the similarity ends. On the vendor benchmark tables Astra wins most rows. On the independent indexes Fable leads. And on price, the headline number hides two line items that can move a real bill by a factor of four.

The OpenAI symbol, an interlocking hexagonal knot
The Claude symbol, an orange starburst

The scoreboard, vendor numbers first

Each company publishes its own results and picks its own comparisons, so the cleanest way to read them is side by side on the benchmarks both report. The agentic set is where the two labs have spent the year, and it is where the gap is widest. All scores below are at maximum effort.

Terminal-Bench 4.0 — GPT-6 Astra: 57.9%Terminal-Bench 4.0 — Claude Fable 5.1: 55.8%Terminal-Bench-Science — GPT-6 Astra: 64.6%Terminal-Bench-Science — Claude Fable 5.1: 52.6%DeepSWE v1.1 — GPT-6 Astra: 74.1%DeepSWE v1.1 — Claude Fable 5.1: 67.4%FrontierCode 1.1 — GPT-6 Astra: 53.3%FrontierCode 1.1 — Claude Fable 5.1: 50.9%AutomationBench — GPT-6 Astra: 41.4%AutomationBench — Claude Fable 5.1: 31.4%HLE with tools — GPT-6 Astra: 57.2%HLE with tools — Claude Fable 5.1: 65%0%20%40%60%80%Terminal-Benc…Terminal-Benc…DeepSWE v1.1FrontierCode…AutomationBen…HLE with toolsScoreTerminal-Bench 4.0 — GPT-6 Astra: 57.9%Terminal-Bench 4.0 — Claude Fable 5.1: 55.8%Terminal-Bench-Science — GPT-6 Astra: 64.6%Terminal-Bench-Science — Claude Fable 5.1: 52.6%DeepSWE v1.1 — GPT-6 Astra: 74.1%DeepSWE v1.1 — Claude Fable 5.1: 67.4%FrontierCode 1.1 — GPT-6 Astra: 53.3%FrontierCode 1.1 — Claude Fable 5.1: 50.9%AutomationBench — GPT-6 Astra: 41.4%AutomationBench — Claude Fable 5.1: 31.4%HLE with tools — GPT-6 Astra: 57.2%HLE with tools — Claude Fable 5.1: 65%Terminal-Bench 4.0Terminal-Bench-ScienceDeepSWE v1.1FrontierCode 1.1AutomationBenchHLE with tools0%50%100%Score
  • GPT-6 Astra
  • Claude Fable 5.1
Agentic and coding benchmarks at maximum effort, higher is better. Vendor-reported scores compiled by DataCamp from the two launch posts. Anthropic quotes a standard error of roughly 3.5 to 4.5 points on Terminal-Bench, so the two-point gap there is inside the noise.
Show the 12 numbers behind this chart
Agentic and coding benchmarks at maximum effort, higher is better. Vendor-reported scores compiled by DataCamp from the two launch posts. Anthropic quotes a standard error of roughly 3.5 to 4.5 points on Terminal-Bench, so the two-point gap there is inside the noise. — the numbers this chart is drawn from
CategoryGPT-6 AstraClaude Fable 5.1
Terminal-Bench 4.057.9%55.8%
Terminal-Bench-Science64.6%52.6%
DeepSWE v1.174.1%67.4%
FrontierCode 1.153.3%50.9%
AutomationBench41.4%31.4%
HLE with tools57.2%65%

Astra takes five of the six. The margins that matter are Terminal-Bench-Science, twelve points, and AutomationBench, ten points, both long-horizon tool-use tasks. Terminal-Bench 4.0 is effectively a tie. The one row Fable wins outright is Humanity's Last Exam with tools, by nearly eight points, which is the broadest general knowledge test in the set.

On the knowledge and perception benchmarks the pattern repeats, with Astra ahead everywhere and the biggest gap on mathematics.

GPQA Diamond — GPT-6 Astra: 96%GPQA Diamond — Claude Fable 5.1: 93.7%FrontierMath Tier 4 — GPT-6 Astra: 97.6%FrontierMath Tier 4 — Claude Fable 5.1: 87.8%ScreenSpot-Pro — GPT-6 Astra: 92.7%ScreenSpot-Pro — Claude Fable 5.1: 87.3%BenchCAD — GPT-6 Astra: 95.9%BenchCAD — Claude Fable 5.1: 84.3%0%20%40%60%80%100%GPQA DiamondFrontierMath Tier 4ScreenSpot-ProBenchCADScoreGPQA Diamond — GPT-6 Astra: 96%GPQA Diamond — Claude Fable 5.1: 93.7%FrontierMath Tier 4 — GPT-6 Astra: 97.6%FrontierMath Tier 4 — Claude Fable 5.1: 87.8%ScreenSpot-Pro — GPT-6 Astra: 92.7%ScreenSpot-Pro — Claude Fable 5.1: 87.3%BenchCAD — GPT-6 Astra: 95.9%BenchCAD — Claude Fable 5.1: 84.3%GPQA DiamondFrontierMath Tier 4ScreenSpot-ProBenchCAD0%50%100%Score
  • GPT-6 Astra
  • Claude Fable 5.1
Knowledge, mathematics and screen understanding at maximum effort, higher is better. Vendor-reported scores compiled by DataCamp. FrontierMath Tier 4 is the research-level set where most models scored under 30% a year ago.
Show the 8 numbers behind this chart
Knowledge, mathematics and screen understanding at maximum effort, higher is better. Vendor-reported scores compiled by DataCamp. FrontierMath Tier 4 is the research-level set where most models scored under 30% a year ago. — the numbers this chart is drawn from
CategoryGPT-6 AstraClaude Fable 5.1
GPQA Diamond96%93.7%
FrontierMath Tier 497.6%87.8%
ScreenSpot-Pro92.7%87.3%
BenchCAD95.9%84.3%

Three Astra numbers are missing from the charts because there is no Fable figure on the same setup. OpenAI reports 99.9% on ARC-AGI-3, with the caveat that it used a custom harness. It reports 72.6% on the OSWorld 2.0 offline set, where Anthropic publishes 41.7% on the strict set and 77.9% on the partial set, which are different scoring rules. And it reports 100% on ExploitBench against Fable's 70%, which is the reason OpenAI says Astra crosses the Critical cybersecurity threshold in its Preparedness Framework and gates those capabilities behind a trusted access programme. Anthropic does the same the other way round: Mythos 5.1, the same weights as Fable with lighter safeguards, scores 60.9% on Terminal-Bench 4.0 against Fable's 55.8%, and only trusted organisations get it.

What the independent testers say

Artificial Analysis runs both models through the same harness at maximum effort, and its numbers reverse the vendor tables. Fable 5.1 scores 65.7 on the Intelligence Index, Astra 61.2, which is a statistical tie with the older GPT-5.6 Sol at 60.9. On the Coding Agent Index, Fable running inside Claude Code scores 70, the highest the index has recorded, and Astra inside Codex scores 67.

Intelligence Index — GPT-6 Astra: 61.2Intelligence Index — Claude Fable 5.1: 65.7Coding Agent Index — GPT-6 Astra: 67Coding Agent Index — Claude Fable 5.1: 70020406080Intelligence IndexCoding Agent IndexIndex scoreIntelligence Index — GPT-6 Astra: 61.2Intelligence Index — Claude Fable 5.1: 65.7Coding Agent Index — GPT-6 Astra: 67Coding Agent Index — Claude Fable 5.1: 70Intelligence IndexCoding Agent Index050100Index score
  • GPT-6 Astra
  • Claude Fable 5.1
Artificial Analysis composite indexes at maximum effort, higher is better. The Coding Agent Index measures each model inside its vendor's own agent, Claude Code and Codex. Index points are not percentages.
Show the 4 numbers behind this chart
Artificial Analysis composite indexes at maximum effort, higher is better. The Coding Agent Index measures each model inside its vendor's own agent, Claude Code and Codex. Index points are not percentages. — the numbers this chart is drawn from
CategoryGPT-6 AstraClaude Fable 5.1
Intelligence Index61.265.7
Coding Agent Index6770

Two other independent measurements cut the other way. Artificial Analysis puts the mean cost of an Intelligence Index task at $1.67 on Astra and $3.76 on Fable, because Astra used far fewer tokens to get its answer at the same per-token price. And its hallucination test has Astra at 51% at maximum effort, down from 92% on the previous generation, which is a large improvement and still a number to plan around. Fable's effort curve tells the same story from the other side: at low effort it scores 26.3% on Terminal-Bench-Science for $11.10 a task, and at max it scores 52.6% for $37.90.

Raw speed is close. OpenRouter measures median throughput at 49 tokens per second for Astra and 52 for Fable, with median time to first token of 3.6 seconds and 4.4 seconds respectively. Astra also sells a Fast mode at twice the price for up to twice the speed. Fable has no fast mode, and its equivalent lever is the effort dial. The number OpenAI leads with is task time rather than token speed: 40 minutes per OSWorld task against 75 minutes for GPT-5.6 Sol.

A 300 metre resolution elevation map of a patch of Venus, with a shield volcano 15 kilometres across shown as a bright yellow dome with a summit pit
Anthropic's launch result that is hardest to argue with: a 300 metre elevation model of a third of Venus, produced by Fable 5.1 from 30-year-old Magellan radar data. A shield volcano 15 kilometres across resolves as a bright dome. Image from Anthropic's announcement.

Same sticker price, different bill

Both models list at $10 in and $50 out, and both offer batch processing at half price. The bill diverges on two lines. Cache reads cost $1.00 per million on Astra and $0.25 on Fable, and on a long agent session, where every turn re-reads the same system prompt, tool list and file context, cache reads are most of the tokens. And once a request passes 272K tokens of input, Astra charges $20 per million in and $75 per million out, with cache reads doubling too. Fable charges the same price at any context length. Hover the bars to read each figure.

GPT-6 Astra, input
$10 per million tokens
Claude Fable 5.1, input
$10 per million tokens
GPT-6 Astra, output
$50 per million tokens
Claude Fable 5.1, output
$50 per million tokens
GPT-6 Astra, cache read
$1.00 per million tokensFour times Fable 5.1's cache read price. On a long agent session cache reads are most of the bill.
Claude Fable 5.1, cache read
$0.25 per million tokensCut from $1.00 on Fable 5. This one line is the whole Fable 5.1 price change.
GPT-6 Astra, input above 272K
$20 per million tokensOpenAI charges 2x on input and cache reads once a request passes 272K tokens of input.
Claude Fable 5.1, input above 272K
$10 per million tokensAnthropic applies no long context surcharge on Fable 5.1.
GPT-6 Astra, output above 272K
$75 per million tokens1.5x on output above 272K tokens of input.
Claude Fable 5.1, output above 272K
$50 per million tokensSame $50 at any context length.
List API prices per million tokens, September 2026, from the two vendors' pricing pages. Batch processing is half price on both.

Whether those two lines matter depends entirely on the shape of your traffic. Drag the sliders below to model a workload and watch the daily bill on each model. The defaults describe a coding agent: a large cached prefix, a modest amount of fresh input, and a few thousand output tokens per call.

Interactive · runs in a sandboxed frameisolated

The calculator makes the trade-off explicit. On a short chat turn with no cached prefix the two models cost the same to the cent. On a coding agent with a 150K cached context the Fable bill is roughly 30% lower. Push the context past 272K and Astra's surcharge doubles the gap again. Against that, Artificial Analysis found Astra spends fewer tokens per task, so on work where the model is thinking rather than reading, its per-task cost can come out lower even at the same list price.

Which one to pick

  • Long agent sessions over a large codebase or document set: Fable 5.1. The cache read price and the absence of a long context surcharge dominate the bill, and the Coding Agent Index puts it ahead inside its own tool.
  • Mathematics, CAD, screen understanding and computer use: Astra. The vendor gaps on FrontierMath, BenchCAD and ScreenSpot-Pro are too large to be noise, and OpenAI's 40 minutes per OSWorld task is a real productivity claim.
  • Short, high-volume calls with no shared prefix: either. The prices are identical and the throughput is within a few tokens per second. Run your own eval on fifty real requests before you commit.
  • Security work: read the access rules before you read the benchmarks. Astra's exploit capabilities sit behind trusted access, and Fable routes penetration testing and exploit development to the Opus models. The 100% ExploitBench score is not something a normal API key can reach.
  • Whichever you choose, check that your prompt cache is actually hitting. A timestamp in the system prompt or an unsorted tool list silently invalidates the prefix, and then you pay full input price on both models.

Sources: OpenAI's GPT-6 Astra announcement, Anthropic's Fable 5.1 announcement, DataCamp's side-by-side compilation, Emergent's Astra benchmark breakdown for the Artificial Analysis figures, and OpenRouter's model pages for Astra and Fable 5.1 for throughput and latency. This blog's earlier post on the Fable 5.1 launch covers its effort curve, safeguards and API changes in detail.

Share

Related

Claude Sonnet 5.5: Opus-Level Scores at Half the Price, and Where It Still Falls Short
Gemini 4 Argon: Google's New Frontier Model, the Benchmarks, the Price and What It Can Do
Claude Opus 5.5: Every Benchmark, From Anthropic, Cursor, Artificial Analysis, Sonar and ARC Prize