This is not a leaderboard; it is a map for choosing. It gathers the strongest and most reasonably priced models for writing, research, coding, and everyday tasks. Models and prices keep moving; what you see here is the most recently checked snapshot.
Updated 2026-10-12 Auto-refreshes every Monday and Thursday snapshot 2026-W42
One new face this cycle: Claude Haiku 5.5, released by Anthropic on Oct 7, costs $0.10/$0.50 per million tokens ($0.50/$2.50 when the prompt exceeds 100K tokens), below the $0.20/$1.20 of gpt-5.6-luna; on the AA intelligence index it scores 43 at its highest reasoning setting and 34 at the default medium setting. It produced about 440 million output tokens to run the index, which places it among the verbose models, and its average cost per task is $0.21, so part of the low unit price is eaten by output volume. It has only the AA board among third-party results so far; long-horizon agentic, SWE-bench Verified and ARC-AGI-2 have no result for it, so this cycle it goes on no card and counts as a reference, not a verdict.
The two new faces recorded last cycle keep their readings. Claude Sonnet 5.5, released by Anthropic on Sep 28, scores 56 on the AA intelligence index, second only to Opus 5.5 at 58, and costs $2/$10 per million tokens, the same as Sonnet 5; yet its average cost per index task is $7.60, about 50% above Sonnet 5, so what is cheap is the unit price, not the spend per task. GPT-6.1 Sol, released by OpenAI on Sep 29, replaces GPT-6 Sol: AA 52, one point below GPT-6 Astra, also $2/$10, with an average cost per task of $0.72, roughly a tenth of Sonnet 5.5.
Both have only the AA board among third-party results so far; long-horizon agentic, SWE-bench Verified and ARC-AGI-2 have no result for either. This cycle they go only on the intelligence card and the value card, none of the five scenario picks changes, and the coding slot stays with Sonnet 5.
Separately, Google's Gemini 4 Argon, released Sep 30, reads 53 on AA, level with GPT-6 Astra and one point above GPT-6.1 Sol, at an introductory $2/$10 per million tokens ($4/$20 afterwards). It is currently open only to the security defenders in Google's Fairwind Program, so most developers cannot use it yet; this cycle records the reading and places it on no card.
Nothing else has a new revision: the DeepSWE primary data file is still the Sep 22 revision, SWE-bench Verified has no new revision, and ARC-AGI-2 carries over the Sep 28 readings.
Start here: this cycle in five lines
Everyday workClaude Opus 5At $5/$25 per million tokens it costs half of Fable 5.1 and GPT-6 Astra (both $10/$50); it reads 51 on the AA intelligence index (it is not listed on the v4.3.2 table this cycle, so last cycle's reading carries over), and it still leads SWE-bench Verified at 96%. Its stablemate Opus 5.5, released Sep 22, is cheaper ($4/$20) and seven points higher on AA, and this cycle it added ARC-AGI-2 (High 93.3%, $0.41 per task, above Opus 5's 90.4%), but the two coding boards this slot weighs most (SWE-bench Verified and long-horizon agentic) still have no result for it, so this slot is unchanged this cycle; it remains the name most likely to take this slot next cycle.
Complex reasoningClaude Fable 5.1 / GPT-6 AstraThe AA index has a new leader this cycle: Claude Opus 5.5, released Sep 22, reads 58, five points above Fable 5.1 and Astra (tied at 53), and this cycle it was also submitted to ARC-AGI-2: 93.3% at the High setting, $0.41 per task, below Astra's 95.0% and above Fable 5.1's 90.0%. It leads one of the two reasoning-related boards and is second on the other, and neither coding board has it yet, so this slot does not change. The tie between Fable 5.1 and Astra holds, and the two names are just as hard to separate. Astra's submitted ARC-AGI-2 result of 95.0% (max setting, $1.12 per task) is still the high mark on that board, ahead of Sol at 92.5%, Opus 5 at 90.4% and Fable 5.1 at 90.0%; on DeepSWE its 74.1% ties with Gemini 3.8 Flash and Opus 5. Both are priced at $10/$50 per million tokens, Astra has a 1.05M-token context and a 128K output cap, and running the full AA index costs $3.26 for Astra against $7.63 for Fable 5.1. The order of the names does not move this cycle: on the AA board Fable 5.1 is still ahead at 53.4 against 52.7, but those 0.7 points sit inside the noise band and either choice is defensible; when budget matters, look at Opus 5 at half the price, or at Opus 5.5, cheaper per token still and for now only a reference point ($4/$20).
CodingClaude Sonnet 5SWE-bench Verified 85.2% at $2 per million tokens: near-top accuracy for pocket change. That board published no new revision this cycle and has no Astra or Fable 5.1 row yet. On long-horizon agent work the top three this cycle (Astra 74.1%, Gemini 3.8 Flash 73.8%, Opus 5 73.6%) tie within their confidence intervals, so the choice comes down to per-task cost; see the scatter plot section below.
High volume / batchGLM-5.3-Flash / gpt-5.6-lunaGLM-5.3-Flash shipped on Aug 26 at $0.15/$0.50 per million tokens (the launch half-off promotion expired Sep 9, so that list price is what you actually pay from this cycle on), and it takes 63.4% on long-horizon agentic work at $0.48 per task: inside the error bars of luna's 67.2% at one sixth the cost, with weights released the same day under MIT, so you can self-host once volume makes the API uneconomic. luna's $0.20/$1.20 carries no time-of-day and no expiry conditions, which keeps it the second pick in this slot on pricing stability alone. DeepSeek changed generations this cycle: V4.1-Flash shipped Sep 10 at $0.30/$1.20 per million tokens peak and $0.15/$0.60 off-peak, MIT licensed, 552B total parameters with 16B active, intelligence 39 this cycle, but its off-peak output price of $0.60 is still above GLM-5.3-Flash's $0.50, its peak rate is more than double, and it has no long-horizon agentic result yet. More to the point, from Sep 14 both old names, deepseek-v4-pro and deepseek-v4-flash, route to V4.1-Flash, so any pipeline still calling an old name should re-confirm which model it is actually running. GPT-6 Astra's lowest low setting costs $1.60 per task (67.0%, recalculated Sep 22) and Gemini 3.8 Flash $2.36, both a price tier up; Meta's Muse Spark 1.3 looks comparable under its Contributor plan at $0.10/$0.20, but that plan trains on your prompts, which rules it out for volume work whose data has to stay with you. Step 5 Preview (StepFun, Sep 20) arrives at $1.00/$2.70 per million tokens with intelligence 44: more expensive per token than luna, several times GLM-5.3-Flash, and without a long-horizon agentic result, so it does not place in this slot; what matters about it is where its score sits against its unit price, which is covered in the observations below. GPT-6 Luna, released Sep 22, joined ARC-AGI-2 this cycle (Max 59.3%, $0.06 per task), but it has no long-horizon agentic result yet, so it is not considered for this slot for now. High-volume work is decided in the band under fifty cents per task, so this slot does not change hands.
Budget no objectClaude Opus 5Still leads SWE-bench Verified at 96%, and it is the only top-tier model with results on all three boards: DeepSWE 73.6% ties with Astra's 74.1% and Gemini 3.8 Flash's 73.8% within the confidence intervals, ARC-AGI-2 90.4%. GPT-6 Astra is ahead on ARC-AGI-2 (95.0%) and on AA intelligence (53 against 51 on v4.3.2), but SWE-bench Verified still has no Astra row, so the pure coding board offers no comparison and this slot stays put; if Astra lands on that board next cycle, it gets another look. Opus 5.5, released Sep 22, is cheaper per token ($4/$20) and leads AA at 58, and this cycle it added ARC-AGI-2 at 93.3% (above Opus 5's 90.4%), but the two coding boards, SWE-bench Verified and long-horizon agentic, still have no result for it, so it gets a look alongside Astra once they do.
Use this to narrow your options, not to replace testing on your own tasks.
Cost · Capability One chart: the real difference is cost
Each model's accuracy on DeepSWE (long-horizon agentic coding) against its average cost per task. Note that the X axis runs opposite to convention: cheaper to the right, more accurate toward the top, so the top-right corner means accurate and cheap. Full chart notes are below the figure.
Claude (Anthropic)GPT (OpenAI)Gemini (Google)GLM (Zhipu)Kimi (Moonshot)efficiency frontier (all settings)one model's effort curve (low to max)
How to read this chart: six main models (sol, terra, luna, fable-5, opus 4.8, sonnet 5) each get an "effort curve" connecting the cost and score of their five settings from low to max; the model name sits at its highest-scoring point. The flatter the curve, the less accuracy a model loses when you dial effort down to save money. The other seven models are plotted only at their best point, as background reference. The gold line is the efficiency frontier: the settings for which no point on the chart is both more accurate and cheaper. This cycle the entire frontier sits on GPT family settings (gpt-5.6-luna, gpt-5.6-terra, gpt-5.6-sol). Hover over any point for its score, ±95% confidence interval, and cost per task.
Source · DeepSWE v1.1 (Datacurve), 113 tasks, updated 2026-09-22; figures come straight from the leaderboard's public data file, and cost is the per-task average in USD. This is one benchmark's view (long-horizon agentic coding): rankings for pure coding or everyday chat differ. The pure coding board (SWE-bench Verified) published no new revision this cycle and is still led by Claude Opus 5 at 96%, with the access-restricted Mythos 5 (95.5%) and Fable 5 (95%) close behind; neither Fable 5.1 nor the GPT-6 Astra previewed on Sep 3 has a row there yet. The data file behind this chart is still the Sep 22 revision this cycle; against the Sep 3 file, no score or confidence interval moved, and the only change is the per-task cost of Astra's five tiers (recalculated at current pricing, down about 30% and nearly 40% at max); none of the new faces that joined the AA board from Sep 21 to Oct 7 (Opus 5.5, Sonnet 5.5, GPT-6.1 Sol, Grok 4.7, MiMo-V2.6-Pro, Haiku 5.5), GPT-6 Luna, or Step 5 Preview has a result on this board yet; OpenAI reports GPT-6.1 Sol on par with Astra on DeepSWE, but the Sep 22 data file does not include it, so this page does not count it. Astra's xhigh setting at 74.1% (±2.9, $4.43 per task in the Sep 22 revision) takes first place on long-horizon agentic work but cannot be separated, by this page's tie criterion, from Gemini 3.8 Flash (73.8%, $2.36 per task, ±1.4) or Claude Opus 5 (73.6%, $11.84 per task, ±3.9), with cost the only real difference. The points on this chart lag the data file: fifteen models in the data file are not yet plotted, among them GPT-6 Astra (the long-horizon leader), Gemini 3.8 Flash (73.8%), Claude Opus 5 (73.6%), GLM-5.3 (69.0%), Kimi K3 (68.5%), Grok 4.6 (67.5%), Gemini 3.7 Flash (65.5%), GLM-5.3-Flash (63.4%, $0.48 per task), DeepSeek V4-Pro (62.8%, $0.24 per task), Qwen3.8-Max (57.5%), Muse Spark 1.2 (54.9%), Grok 4.5 (53.8%) and DeepSeek V4-Flash-0731 (53.3%, whose $0.10 per task is the lowest on the board); those points await manual entry, so the efficiency curve drawn here contains neither the top three nor the cheapest row. Prices lag too, and in both directions: OpenAI's Jul 30 cut was never recalculated, so Luna and Terra look more expensive here than they are; DeepSeek moved to peak and off-peak pricing on Aug 16, so the few cents shown for those rows is clearly low during peak hours, and from Sep 14 both deepseek-v4-pro and deepseek-v4-flash route to the V4.1-Flash released on Sep 10, which makes those two points historical positions for those two models. The Arena human blind-test WebDev overall scores cited on this page carry over from last cycle: Claude Opus 5 first (1691), Kimi K3 second (1674), while on the same site's Frontend sub-board Kimi K3 is first (1679). For other ways to draw this: the official interactive DeepSWE board (every setting of every model plotted, with cost, output tokens, or agent steps on the X axis), or Artificial Analysis (a different scatter: overall intelligence versus price).
View the data table (13 models)
Model
Vendor
Effort
DeepSWE score
Cost per task
Frontier
gpt-5.6-sol
GPT
max
72.7% ±2.8
$8.39
◆
claude-fable-5
Claude
xhigh
69.9% ±3.2
$13.41
gpt-5.6-terra
GPT
max
69.6% ±2.6
$4.95
gpt-5.6-luna
GPT
max
67.2% ±4.0
$3.03
◆
gpt-5.5
GPT
xhigh
67.0% ±6.5
$7.23
claude-opus-4.8
Claude
max
59.0% ±1.8
$13.22
claude-sonnet-5
Claude
max
53.8% ±4.2
$26.40
gpt-5.4
GPT
xhigh
51.8% ±1.5
$5.65
glm-5.2
GLM
max
43.8% ±1.7
$3.92
gemini-3.5-flash
Gemini
high
36.1% ±4.0
$3.45
kimi-k2.7-code
Kimi
default
30.5% ±0.5
$2.82
claude-sonnet-4.6
Claude
high
29.9% ±4.1
$5.52
gemini-3.1-pro
Gemini
high
11.7% ±1.5
$2.14
Generalization One chart: generalization ability
ARC-AGI-2 tests a batch of abstract visual reasoning puzzles no model has seen, to measure "generalization ability": whether a model can reason on the fly about rules it has never encountered, rather than pattern-matching to similar problems from its training data. A higher score means it handles genuinely novel problems better; this is a different capability from the coding benchmarks above.
Up front: Claude Haiku 5.5 has no official ARC-AGI-2 submission. Submissions are voluntary and vendor-initiated, not something we curate; the 14 models in the table below are the officially published results we could find.
Claude (Anthropic)GPT (OpenAI)Gemini (Google)GLM (Zhipu)Grok (xAI)
Source · ARC Prize (arcprize.org/leaderboard), verified 2026-09-28; cost is the per-task figure vendors reported at submission time, on a different basis from the DeepSWE chart above, comparable only within this chart, not additive across charts.
This leaderboard's model roster isn't identical to the one above: ARC-AGI-2 submissions are voluntary and vendor-initiated, not something we selected. As of verification, Claude Haiku 5.5, Gemini 4 Argon, Claude Sonnet 5.5, MiMo-V2.6-Pro, Grok 4.7, GPT-6.1 Sol, Step 5 Preview, GLM-5.3, GLM-5.3-Flash, DeepSeek V4.1-Flash, Qwen3.8-Max, Muse Spark 1.3, Muse Spark 1.2, Muse Glimmer have no official results, so they don't appear in this table; that doesn't mean weaker generalization, just that no comparable data exists yet on this leaderboard. Model names follow ARC Prize's official submission naming, which may refer to a different generation than the coding-benchmark names above.
View data table (14 models)
Model
Vendor
Effort
ARC-AGI-1
ARC-AGI-2
Cost Per Task
GPT-5.6 Sol
GPT
Max
96.5%
92.5%
$1.44
Claude Opus 5
Claude
Max
97.5%
90.4%
$2.06
Claude Fable 5
Claude
Max
98.5%
89.2%
$5.45
GPT-5.5
GPT
xHigh
95.0%
85.0%
$1.87
Gemini 3 Deep Think
Gemini
2/26
96.0%
84.6%
$13.62
GPT-5.6 Terra
GPT
Max
96.5%
83.9%
$1.09
Gemini 3.1 Pro
Gemini
Preview
98.0%
77.1%
$0.96
Claude 4.7
Claude
Max
92.0%
75.8%
$7.43
Claude Opus 4.8
Claude
High
92.0%
72.1%
$2.74
Gemini 3.5 Flash
Gemini
High
92.5%
72.1%
$0.85
GPT-5.6 Luna
GPT
Max
88.0%
59.5%
$0.67
Claude Sonnet 4.6
Claude
Max
86.0%
58.3%
$2.72
Grok 4.5
Grok
High
85.7%
52.6%
$0.78
GLM-5.2
GLM
N/A
77.0%
22.8%
$0.25
Safety Resistance to prompt injection
Indirect prompt injection is an agent's biggest security weakness: an attacker hides malicious instructions inside data the model reads (web pages, tool output, code), steering the agent into unauthorized actions. This chart shows the probability an attack succeeds within k attempts, lower is better. This is a safety axis, separate from the capability and cost axes above.
Note first: these numbers are Anthropic-reported, not a third-party leaderboard result. The attack set is the independent security firm Gray Swan's public indirect-injection corpus (28 scenarios, 1,130 highly transferable attacks); Anthropic ran it against each model at Opus 5's launch, under uniform settings (no extra defenses on Claude, extended thinking on for all models). Other vendors have not published self-run results on the same set, so cross-vendor comparison is indicative only, not a verdict.
Claude (Anthropic)GPT (OpenAI)Gemini (Google)Muse (Meta)Grok (xAI)bands dark to light: k=1 / k=10 / k=15
Source · Gray Swan indirect-prompt-injection corpus (grayswan.ai), numbers Anthropic-reported, reported on 2026-07-24; darker to lighter bands are k=1 / k=10 / k=15 (fewer tries to break means less robust). A safety axis, not additive with the capability or cost boards above.
See data table (14 models)
Model
Vendor
k=1
k=10
k=15
Opus 5
Claude
0.2%
1.6%
2.0%
Mythos 5
Claude
0.3%
2.1%
2.6%
Fable 5
Claude
0.4%
2.3%
2.8%
Opus 4.8
Claude
0.5%
4.1%
5.5%
Sonnet 5
Claude
0.6%
4.7%
5.9%
Muse Spark
Muse
2.9%
14.3%
16.5%
GPT 5.6 Sol
GPT
3.1%
16.3%
20.0%
GPT 5.5
GPT
3.0%
17.4%
20.8%
GPT 5.6 Terra
GPT
5.4%
26.0%
30.4%
Gemini 3.6 Flash
Gemini
7.3%
32.2%
37.3%
GPT 5.6 Luna
GPT
8.3%
38.6%
43.9%
Gemini 3.1 Pro
Gemini
14.2%
45.7%
49.2%
Gemini 3.5 Flash
Gemini
14.1%
54.2%
60.5%
Grok 4.5
Grok
13.4%
54.2%
60.8%
How these picks are ranked
First check whether the gap is within noise: the top few usually tie, and one or two points of score do not mean "stronger".
When tied, compare cost: within the same capability band, pick whichever is cheaper per task or per million tokens.
Then match the task type: pure coding, long-horizon agentic work, and everyday chat measure different abilities; confirm the board measures your use case.
Count data sovereignty as cost: when data must stay on your own machines, self-hostable open models are worth more than their price.
This is the order used to rank this page, not a hard rule. For your own situation, use the prompts at the bottom and let an AI re-compare against your task.
"Cost" here means API usage cost, not a chat subscription fee. DeepSWE figures are cost per task; SWE-bench figures are price per million tokens. The two baselines differ and should only be compared within their own kind.
Coding boards come in two kinds: those run with each vendor's own harness (such as Artificial Analysis's Coding Agent Index, where each lab uses its own Claude Code, Codex, or Grok Build) tend to be optimistic and hard to compare across vendors; those run with a standardized agent (such as DeepSWE, one yardstick, contamination-resistant) are fairer but cover fewer models. Check which kind a score belongs to before reading it.
By Use-Case Picking a model by task
General intelligence
1Claude Opus 5.5AA index 58 · $4/$20
2Claude Sonnet 5.5AA index 56 · $2/$10
3Claude Fable 5.1AA index 53
4GPT-6 Astra (OpenAI)AA index 53
5GPT-6.1 Sol (OpenAI)valueAA index 52 · $2/$10
Two new names joined the AA intelligence index this cycle. Claude Sonnet 5.5, released by Anthropic on Sep 28, ranks second at 56, 18 points above Sonnet 5's 38 and only two behind Opus 5.5 (58); but it averages about 193k output tokens per index task, roughly seven times GPT-6 Astra, for an average cost of $7.60 per task. GPT-6.1 Sol, released by OpenAI on Sep 29, replaces GPT-6 Sol (48) and reads 52, one point below Fable 5.1 and Astra, tied at 53, with an average cost of $0.72 per task. Both have only this one third-party board so far; their placement follows this card's ordering rule and is a reference point, not a settled result. The fifth slot weighs score against price, and this cycle GPT-6.1 Sol takes it over from Muse Spark 1.3 (48, $1.25/$4.25): Sonnet 5.5 at the same $2/$10 per million tokens scores four points higher, but costs about ten times as much per task, so on value it is Sol. Other readings carry over the Sep 24 v4.3.2 table: Opus 5.5 58, Fable 5.1 and Astra tied at 53, Muse Spark 1.3 48, Grok 4.7 and MiMo-V2.6-Pro 46, Qwen3.8-Max (0902) and GLM-5.3 45, Grok 4.6, Step 5 Preview and Kimi K3 44, GPT-5.6 Terra and GLM-5.3-Flash 42, Gemini 3.8 Flash 41, DeepSeek V4.1-Flash 39, Sonnet 5 38, Muse Glimmer 17; Opus 5 (51), Fable 5 (50), GPT-5.6 Sol (47) and GPT-5.6 Luna (37) carry over earlier readings. Gaps of a point or two at the top should still be read as ties: Sonnet 5.5 against Opus 5.5, and Sol against Fable 5.1 and Astra. AA has issued several version numbers within a month, and reposting sites often attach a new version number to old-scale figures, so confirm the version before citing. Google's Gemini 4 Argon (Sep 30) reads 53, level with Fable 5.1 and Astra, and AA measures its hallucination rate at 15%, the lowest of any model scoring 45 or above; it is currently open only to invited security defenders, so most developers cannot use it yet and it is not placed on this card this cycle.
Source · Artificial Analysis Intelligence Index v4.3.2 (read Sep 24; Sonnet 5.5, GPT-6.1 Sol and Gemini 4 Argon read Oct 1)
Pure coding (SWE-bench Verified)
1Claude Opus 596%
2Claude Fable 595%
3Claude Sonnet 5value85.2% · $2/M
This board published no new revision this cycle and the figures are unchanged; it also has no row yet for GPT-6 Astra, Fable 5.1 or Gemini 3.8 Flash. OpenAI did not self-report SWE-bench Verified for Astra, only Terminal-Bench 4.0 at 57.9% and OSWorld 2.0 at 72.6%, both on the vendor's own harness. Opus 5 still leads at 96%. Mythos 5 (95.5%) is access-restricted, and Fable 5 (95%) follows, so the practical ceiling for most users is Opus 5. The top few are packed inside one percentage point, and this board is close to saturated for frontier models. One more case on this card is worth flagging: DeepSeek V4-Pro scores 80.6% and ranks 11th on the source this card cites, while a separate independent evaluation site running the same SWE-bench Verified set through its own harness gives it 96.4% and second place. Same benchmark name, two rulers, 16 percentage points apart. One more to note: the V4-Pro API name has been routed to DeepSeek V4.1-Flash since Sep 14, so that row can only be read as history from here on. Check that scores share a harness before comparing across sites; this card uses only the source it names.
The primary data file is still the Sep 22 revision this cycle (113 tasks); comparing the Sep 22 revision with the Sep 3 file entry by entry, every model's score and confidence interval matches the Sep 3 file; the only change is the per-task cost of GPT-6 Astra's five tiers: xhigh $6.52→$4.43, high $5.72→$3.92, max $12.37→$7.50, medium $4.38→$3.08, low $2.19→$1.60, down about 30%, and nearly 40% at max. The file states the pricing basis (per million tokens: uncached input $10, cache write $12.50, cache read $1, output $50) but not the reason for the recalculation. Astra still leads on its xhigh tier at 74.1% (±2.9, roughly 29 steps on average), and by this page's tie criterion it cannot be separated from Gemini 3.8 Flash at 73.8% (±1.4) or Claude Opus 5 at 73.6% (±3.9); what separates the three is still cost: $4.43 against $2.36 against $11.84 per task. Gemini 3.8 Flash is still the cheapest thing in the tie band, but its cost gap to Astra narrows from a third last cycle to about half. Astra's max tier at ±0.8 is still the narrowest confidence interval on the board, it scores no higher than xhigh, and after the recalculation it costs about 70% more than xhigh. Fable 5.1 is still absent, and none of the four new faces that joined the AA board on Sep 21 to 22 (Opus 5.5, GPT-6 Sol, Grok 4.7, MiMo-V2.6-Pro), GPT-6 Luna from the same batch, or Step 5 Preview has a result on this board yet. Below the tie band: GPT-5.6 Sol (72.7%, $8.4), Fable 5 (69.9%) and GLM-5.3 (69.0%, $3.99) cannot be separated from the top three either; in the middle, Kimi K3 (68.5%), Grok 4.6 (67.5%, $3.45), gpt-5.6-luna (67.2%), Gemini 3.7 Flash (65.5%), GLM-5.3-Flash (63.4%, $0.48) and DeepSeek V4-Pro (62.8%, $0.24) cannot be separated pairwise either. Apart from Astra's five rows, per-task costs are still the prices that applied when each setting was run: OpenAI's Jul 30 price cut was never recalculated, so Terra and Luna are cheaper in practice; DeepSeek moved to peak and off-peak pricing on Aug 16, so V4-Pro's $0.24 and V4-Flash's $0.10 run higher than these figures during peak hours; and from Sep 14 both of those API names route to the V4.1-Flash released on Sep 10, so those two rows should be read as historical results for those two models rather than as what the same name would give you today. GLM-5.3's weights are out, so the 69.0% row is a self-hostable result; the vendor README's self-reported 66.9% comes from a different harness and should not be compared against it directly.
The workhorse for volume, batch and parallel agent work: good enough and cheap. The roster on this card does not change this cycle. Claude Haiku 5.5, released Oct 7, costs $0.10/$0.50 per million tokens with AA 43 at its highest reasoning setting, a lower unit price than gpt-5.6-luna, but it has no long-horizon agentic result yet, so like Step 5 Preview it gets no slot for now. Gemini 3.8 Flash still leads: 73.8% on long-horizon agentic work at $2.36 per task, tied inside the confidence intervals with the long-horizon leader GPT-6 Astra at 74.1% ($4.43 per task after the Sep 22 recalculation) for about half the cost; its $0.75/$3.75 per million tokens is written to hold until 2026-12-31 and then doubles to $1.50/$7.50; on the v4.3.2 ruler its intelligence reads 41, two points above its stablemate 3.7 Flash at 39. GLM-5.3-Flash holds the cost end: 63.4% long-horizon at $0.48 per task, the only entry here under $0.5 per task that stays inside the tie band; 320B total parameters with 18B active, weights released under MIT; the launch half-off promotion ($0.075/$0.25) expired Sep 9, so the $0.15/$0.50 list price is what you actually pay from this cycle on. Grok 4.6 takes intelligence 44 at $2/$6; Grok 4.7 (Sep 21) takes 46 at the same price but has no long-horizon agentic result yet, so it does not replace it this cycle. Last cycle this card said it stood at the same unit price as Qwen3.8-Max (40) and four points ahead; that no longer holds: the Qwen3.8-Max on the AA board is now the 0902 snapshot from Sep 1, taking 45 at the same $2/$6, which puts the two inside the tie band, and the old 40 belongs to a different, now deprecated model. gpt-5.6-luna's $0.20/$1.20 is the only price here with no expiry and no time-of-day condition, and it reaches 67.2% at its top long-horizon setting. Two entries stay off the list: Astra's low tier at 67.0% and $1.60 per task after the Sep 22 recalculation, now below Gemini 3.8 Flash's $2.36, but its $10/$50 per million tokens has no cheaper variant, so at real throughput the output price swamps the per-task figure; Meta's Muse Spark 1.3 lists at $1.25/$4.25 with intelligence 48, and its Contributor plan at $0.10/$0.20 comes with prompts being used for training, so this card does not count it among the cheap options. DeepSeek V4-Flash at $0.10 per task is still the lowest figure on the board, but that number rests on pre-Aug 16 pricing, and the model left the API on Sep 14 with its name routed to the V4.1-Flash released on Sep 10 (peak $0.30/$1.20, off-peak $0.15/$0.60, intelligence 39 this cycle); the new model has no long-horizon agentic result yet and its off-peak output price is above GLM-5.3-Flash's $0.50, so this slot still withholds a place from it. Step 5 Preview (StepFun, Sep 20) arrives at $1.00/$2.70 per million tokens with intelligence 44, a unit price below half of Grok 4.6 at a tied score, which fits this card very well, but it has no long-horizon agentic result at all and still carries a preview label, so it gets no place this cycle and only a line in the observations. This card once carried three prices with expiry dates; the GLM-5.3-Flash one expired Sep 9, and the remaining two mean you should price 2027 into any selection you make now.
Source · Artificial Analysis / DeepSWE / published vendor pricing
Best Value The strongest picks at each price point
These are the picks most worth considering at each budget, what analysts call the value frontier.
Best value (closed)
1Gemini 3.8 Flash (Google)value73.8% · $0.75/$3.75 per M
2GPT-6.1 Sol (OpenAI)valueAA 52 · $2/$10 per M
3Muse Spark 1.3 (Meta)valueAA 48 · $1.25/$4.25 per M
4Grok 4.6 (xAI)valueAA 44 · $2/$6 per M
5Claude Sonnet 5value85.2% · $2/M
Near-top capability at a fraction of the price. The default first choice for most applications. Top of the list is still Gemini 3.8 Flash, released by Google on Sep 2: 73.8% on long-horizon agentic work ties the long-horizon leader GPT-6 Astra at about half its per-task cost after the Sep 22 recalculation, and at $0.75/$3.75 per million tokens it is the cheapest closed model in that tie band. That price is conditional, stated to run until 2026-12-31 and then revert to $1.50/$7.50; its intelligence index is 41 and SWE-bench Verified has no result for it yet, so the value is in the long-horizon agent slot, not across the board. New in second place is GPT-6.1 Sol, released by OpenAI on Sep 29: intelligence 52, one point below Astra, at $2/$10 per million tokens, a fifth of Astra's price, with an average AA index cost of $0.72 per task; it replaces GPT-6 Sol (48) from Sep 22, and neither the long-horizon agentic board nor the coding boards have it yet. OpenAI reports it on par with Astra on DeepSWE, but the public data file has no such entry, so this page does not count it. Meta's Muse Spark 1.3 moves to third: intelligence 48 at $1.25/$4.25 per million tokens, still cheaper than Sol, with a 1M-token context and no released weights; it also has a Contributor plan at $0.10/$0.20 on condition that prompts are used for training, and this card uses the standard price. Grok 4.6 is fourth: $2/$6, intelligence 44, long-horizon 67.5%, $3.45 per task (mind xAI's long-context tier: once a prompt exceeds 200k tokens, everything is billed at double rate); the Qwen3.8-Max 0902 snapshot at the same price scores 45, a tie by this page's standard, and a tie does not change the pick. Sonnet 5 still gets a near-top coding score at a fraction of the price. Claude Sonnet 5.5 (Sep 28, AA 56), also at $2/$10, is not on this card: the unit price is the same, but its index cost per task is $7.60, about ten times Sol, and value is about actual spend. GPT-6 Astra is priced at $10/$50, outside this card's price band. Grok 4.7 from xAI (Sep 21, $2/$6, AA 46) has no long-horizon agentic result yet and stays off the list this cycle. Qwen3.8-Max weights have still not been released; Hugging Face has Qwen3.8-2.4T-A95B (intelligence 40) and Qwen3.8-27B. Step 5 Preview (StepFun, Sep 20) undercuts everything on this card on price ($1.00/$2.70, intelligence 44) and promises weights on Oct 15, but for now it has only the AA board, so it stays off the list until the promise is kept.
Best open, self-hostable
1GLM-5.3 (Z.ai)openAA 45 · DeepSWE 69.0% · 753B
2MiMo-V2.6-Pro (Xiaomi)openAA 46 · 1.02T/42B active · MIT
This column is for keeping data inside your own environment, or for holding cost down over the long run. The list does not move this cycle; last cycle added one slot, with MiMo-V2.6-Pro, released by Xiaomi under MIT on Sep 21, entering in second place. The top layer is still Zhipu's GLM-5.3 (not Flash): weights released on Hugging Face on Aug 31 (zai-org/GLM-5.3), 753B total parameters, the same base as GLM-5.2 with the difference in post-training. The licence is not MIT but an MIT-style custom licence with one extra clause: model-as-a-service providers with more than $10B in annual revenue need to clear a Z.AI safety review first, while everyone else self-hosting, individual or company, is unaffected. Set against Kimi K3: intelligence 45 against 44 (the order flipped on the new ruler, but in both cycles the gap is a single point, which is noise) and long-horizon 69.0% against 68.5%, also inside the error bars, but 753B against 2.8T means that at the same 4-bit quantisation the memory requirement is about a quarter of Kimi K3's, which is why it sits at the top of this column. The weights have been public for a week, independent replication has not caught up, and it has not been submitted to ARC-AGI-2, so treat it as a reference point rather than a settled result. Kimi K3 is third: AA 44 and ARC-AGI-2 60.4%, still the highest among submitted open-weight models, but 2.8T parameters at 4-bit quantisation is roughly 1.4TB of high-bandwidth memory, which in practice means data-centre hardware. GLM-5.3-Flash (320B with 18B active, MIT, long-horizon 63.4%, $0.48 per task, intelligence 42) and last cycle's replacement DeepSeek V4.1-Flash (552B with 16B active, MIT, intelligence 39 this cycle) each hold a hardware tier; the lowest barrier is still Meta's Muse Glimmer at 30B under Apache 2.0, which runs on a single 24GB consumer GPU after 4-bit quantisation, at the price of an intelligence score of just 17 on v4.3.2 (this cycle's reading; last cycle said 18), the only entry in this column below 20. Last cycle this slot read 35 and the card caption read 24, both readings from older versions, and both were corrected in the Sep 10 cycle. The DeepSeek slot moved to V4.1-Flash last cycle: it shipped Sep 10 and took over deepseek-v4-pro's API traffic from Sep 14, reads 39 on the v4.3.2 ruler (it read 40 on the earlier v4.3 ruler, a slight drop with the change of ruler) and still sits above the V4-Pro it replaces, with total parameters down from 1.6T to 552B and active parameters down from 49B to 16B, which lowers the self-hosting hardware barrier by an order of magnitude, under the same MIT licence, with a 1M-token context and native image understanding. It has no long-horizon agentic, pure coding or ARC-AGI-2 result yet, so this column lists it on parameter count and intelligence with benchmarks pending. The new second entry, MiMo-V2.6-Pro, is a MoE with 1.02T total parameters and 42B active, with weights released on Hugging Face (XiaomiMiMo/MiMo-V2.6-Pro-RL) under MIT; its AA 46 is currently the highest among open-weight models, and its API costs $0.435/$0.87 per million tokens. It sits in the tie band with GLM-5.3 at 45, and it ranks second because 1.02T needs more memory than GLM-5.3's 753B, and because long-horizon agentic work, SWE-bench Verified and ARC-AGI-2 have no result for it yet, so treat it as a reference point rather than a settled result. Neither GPT-6 Astra nor Muse Spark 1.3 releases weights, so neither appears here. The case for self-hosting is unusually clear this cycle: vendors retired old API names, pulled old models, and an evaluation board even repointed a name at a different model, and none of that touches a set of weights you run yourself. Two candidates sit outside the list pending delivery: the Step 5 Preview StepFun released on Sep 20 (600B total parameters, 27B active, intelligence 44) has weights promised for Oct 15, and Qwen3.8-Max's weights repository, promised since Sep 1, still has not appeared. This column admits only released weights, so both wait until the promise is kept.
What the benchmarks leave out
The same model can look very different on another board or another task. These are gaps worth knowing beyond the scores.
Claude Opus 5.5 (Anthropic, Sep 22)Released by Anthropic on Sep 22 at $4/$20 per million tokens (batch $2/$10) with a 1M-token context, a fifth below Opus 5's $5/$25. AA intelligence reads 58 at the max setting and 56 at xhigh, and running the full index costs $5.98, between Astra's $3.26 and Fable 5.1's $7.63; it is the first model since v4.3.2 to clear the tie band at the top, by five points. The vendor also self-reports Terminal-Bench 4.0 at 66.4%, but that runs on its own harness, so this page does not rank on it and it should not be compared directly with Astra's 57.9%. On Sep 28 it was found to have been submitted to ARC-AGI-2: 93.3% at the High setting, $0.41 per task, second among the models this page tracks behind GPT-6 Astra's 95.0%; XHigh reads 92.5% ($0.67) and Max drops to 91.7% ($1.85), so the top setting did not buy a higher score; ARC-AGI-1 High reads 98.5%. The two coding boards, long-horizon agentic and SWE-bench Verified, still have no result for it, so this page still places it on the intelligence card only; if those two fill in next cycle, the everyday-work and no-budget-limit slots are the most likely to need another look. A reference point, not a settled result.
Step 5 Preview (StepFun, Sep 20)A sparse MoE with 600B total parameters and 27B active, a 1M-token context and native image understanding, with the API opening the day it was announced, priced at $1.00/$2.70 per million tokens. AA intelligence 44 (43.7) puts it inside the tie band with Grok 4.6 at 44 and Kimi K3 at 44, but its unit price is below half of Grok 4.6's ($2/$6) and between a third and a fifth of Kimi K3's ($3/$15), which makes it the cheapest position in that score band right now. StepFun has also promised weights for Oct 15, and if that lands, the top of the open-weight column will need another look: GLM-5.3, which holds it today, is 753B total parameters at intelligence 45, while Step 5 Preview takes 44 on 600B. It gets no place on any list this cycle, because the only board that has it is AA: long-horizon agentic, pure coding and ARC-AGI-2 all lack a row for it, the weights have not landed, and a promised weight release already has an unfulfilled precedent on this page (Qwen3.8-Max, promised since Sep 1). Treat it as a reference point, not a settled result.
AA intelligence index re-versioned a third time in September (this time the scoring method)Artificial Analysis shipped v4.3, v4.3.1 and v4.3.2 within September; this page used v4.3 last cycle and moves to v4.3.2 now. Unlike last cycle, what changed is the scoring method rather than the questions: v4.3.1 replaced the pairwise judging panel for AA-Briefcase and GDPval-AA v2 with current models (Claude Opus 5, GPT-5.6 Sol, Gemini 3.8 Flash), leaving the rubric-grading panel untouched; v4.3.2 then lifted GDPval-AA to v2.1, moved the Elo anchor from human experts to DeepSeek V4.1-Flash (max tier) equals 1600 and switched scoring to a Crowd-BT model fit, with AA-Briefcase moving to v1.1 in step, still anchored at GPT-5.5 (medium) equals 1000 and refitted the same way. The composition and weighting of the ten evaluations were untouched, so scores barely moved: the order of the top five and their rounded integers are identical to last cycle. Each of those two changes is worth recording. A judging panel made of current models means the models being scored and the models doing the scoring are the same cohort, so relative positions among the leaders now carry a circular component. Moving the Elo anchor to a commercial model that can be retired, repriced, or even have its API name taken over by another model ties the ruler to a moving reference point, which makes cross-version differences even less usable than before; only ranks travel. Carry the version number whenever you cite an AA score, and especially this cycle: four version numbers in one month, two of which produce scores that look almost identical.
Claude Sonnet 5.5: same unit price, different spend per taskSonnet 5.5 costs $2/$10 per million tokens, the same as Sonnet 5, and its AA intelligence score jumps from 38 to 56. But AA also records that it averages about 193k output tokens per index task, roughly seven times GPT-6 Astra and about 60% more than Opus 5.5, for an average cost of $7.60 per task, about 50% above Sonnet 5. GPT-6.1 Sol, released the same week at the same $2/$10, costs $0.72 per task. The unit price only tells you what each token costs; how long the model thinks decides what you actually pay, so per-task spend is a better basis for comparing cost than the price list. Two Terminal-Bench figures should also not be mixed: AA's standardised run measures 64%, while the vendor's self-reported 70.6% runs on its own harness; the OSWorld 2.1 figure of 80.1% is also self-reported and has no independent rerun yet.
Per-task cost figures have a shelf lifeThe top three on the long-horizon board tie, but at three different costs: Astra $4.43 per task, Gemini 3.8 Flash $2.36, Opus 5 $11.84; Astra's own five settings run from $1.60 (low, 67.0%) to $7.50 (max, 73.2%), with max about 70% more than xhigh and no higher score. Astra's five rows were recalculated at current pricing in the Sep 22 data file, down about 30% and nearly 40% at max, with scores unchanged; the file states only the pricing basis, not the reason, and the other models' costs on the same board were not recalculated. Within the same score bracket Sol ($8.4), Terra ($5.0), Grok 4.6 ($3.45) and Luna ($3.0) all cost far less; at the bottom, DeepSeek V4-Flash scores 53.3% at $0.10 per task, tied within error with Sonnet 5 (53.8%) at $26.4. But all of these were computed at the prices in force when each setting was run: OpenAI cut Luna by 80% and Terra by 20% on Jul 30 (cheaper in practice), while DeepSeek moved to peak and off-peak pricing on Aug 16 with peak output at $3.96 per million tokens (dearer in practice). Since Sep 14 it goes further: the names deepseek-v4-pro and deepseek-v4-flash are routed to V4.1-Flash, shipped Sep 10, so those two rows keep their scores and their costs while the same API names now return a different model. The cost column of an evaluation board ages faster than the score column, and sometimes what has aged out is not the price but the model itself; check the date before quoting it.
Claude Sonnet 5Cheapest unit price in the Claude line ($2 per million tokens), yet at its top setting on the long-horizon board it is the most expensive per task at $26.4: the board's data shows it averaging 268 steps and burning over seventy million input tokens to solve 53.8% (Fable 5 needs about 68 steps on the same tasks). Cheap per token does not mean cheap per task, least of all on long-horizon work.
Recommendation changelog
When a top pick changes, the date and reason are recorded here so you can trace how the judgment evolved. Newest first.
2026-10-01No scenario pick changes hands; this same-day entry is added because (c) a key fact is corrected: the earlier entry today said this cycle had two new arrivals and missed Google's Gemini 4 Argon, released Sep 30. It reads 53 on AA, level with GPT-6 Astra, at an introductory $2/$10 per million tokens ($4/$20 afterwards) and $1.99 per AA index task. It is currently open only to the security defenders in Google's Fairwind Program, so most developers cannot use it yet; it goes on no card and is added only to the headline, the intelligence card note, the caveats and the ARC-AGI-2 not-yet-submitted list. Google's self-reported DeepSWE v1.1 77.9% is not in the Sep 22 public data file and is not counted here.
2026-10-01No scenario pick changes hands this cycle; this entry is added because (b) the card lists changed. Claude Sonnet 5.5, released by Anthropic on Sep 28 (AA 56, $2/$10, average index cost $7.60 per task), goes into second place on the intelligence card; GPT-6.1 Sol, released by OpenAI on Sep 29 (AA 52, $2/$10, $0.72 per task), replaces GPT-6 Sol and goes into fifth place on the intelligence card (taking over from Muse Spark 1.3) and second place on the closed value card, where Muse Spark 1.3 moves to third. Both have only the AA board among third-party results; long-horizon agentic, SWE-bench Verified and ARC-AGI-2 have no result for either, and the coding slot stays with Sonnet 5. Sonnet 5.5 is added to the ARC-AGI-2 not-yet-submitted list, and GPT-6 Sol is replaced there by GPT-6.1 Sol. DeepSWE is still the Sep 22 revision, SWE-bench Verified has no new revision, and ARC-AGI-2 carries over the Sep 28 readings.
2026-09-28No scenario pick changes hands this cycle; this entry is added because (a) primary data has a new revision: the primary ARC-AGI-2 JSON endpoint still returns 404, but ARC Prize's per-model result pages are readable this cycle and show three new submissions. Claude Opus 5.5 (released Sep 22) reads High 93.3%, $0.41 per task (XHigh 92.5%, Max 91.7%, Medium 87.5%, ARC-AGI-1 High 98.5%), second among the models this page tracks behind GPT-6 Astra's 95.0%; Gemini 3.8 Flash reads High 89.2%, $0.40 per task; GPT-6 Luna (released Sep 22) reads Max 59.3%, $0.06 per task. Opus 5.5 and Gemini 3.8 Flash are removed from the not-yet-submitted list, which now starts with MiMo-V2.6-Pro; none of the three new results is plotted yet, pending manual addition. The description of Opus 5.5 changes from having only the AA board among third-party results to having two, but the two coding boards still have no result for it, so everyday work, complex reasoning (93.3% is below Astra's 95.0%) and no budget limit all stay unchanged. DeepSWE is still the Sep 22 revision, SWE-bench Verified has no new revision, and AA intelligence carries over the Sep 24 v4.3.2 readings; no new major model release was found after Sep 24.
2026-09-24No scenario pick changes hands this cycle; this entry is added because all three of (a), (b) and (c) apply. (a) The DeepSWE data file shipped a Sep 22 revision: entry by entry, no model's score or confidence interval moved, and only the per-task cost of GPT-6 Astra's five tiers was recalculated at current pricing (per million tokens: uncached input $10, cache write $12.50, cache read $1, output $50), xhigh $6.52→$4.43, high $5.72→$3.92, max $12.37→$7.50, medium $4.38→$3.08, low $2.19→$1.60, with no reason given in the file; the Astra costs cited on the long-horizon agentic, volume and closed-model value cards and in the observations were replaced in one batch, and the cost gap between Gemini 3.8 Flash and Astra was rewritten from a third to about half. (b) Two cards gained entries: the intelligence card adds Claude Opus 5.5 (released Sep 22, AA 58) in its first slot, with Opus 5 giving up its place; the open-weight card adds MiMo-V2.6-Pro, released by Xiaomi under MIT on Sep 21 (1.02T total, 42B active, AA 46), in second place, with the rest moving down. Both have only the AA board among third-party results and are listed as reference points rather than settled results; Opus 5.5's self-reported Terminal-Bench 4.0 66.4% runs on the vendor's own harness and is not on the same scale as the standardised runs. (c) A corrected reading: Muse Glimmer from 18 to 17. Two new readings are recorded without a place: Grok 4.7 (Sep 21, AA 46, $2/$6) and GPT-6 Sol (Sep 22, AA 48, $2/$10), while GPT-6 Luna, released the same day, has no AA reading yet. Opus 5, Fable 5, GPT-5.6 Sol and GPT-5.6 Luna are not listed on the v4.3.2 table this cycle, so last cycle's readings carry over. SWE-bench Verified has no new revision, and the primary ARC-AGI-2 JSON still returns 404, so the Sep 4 reading carries over.
2026-09-21No card changes hands this cycle; this entry is added under (a) a primary data file shipping a new version and (c) a verifiable correction to key numbers. (a) The Artificial Analysis intelligence index shipped three versions within September, and the v4.3 this page used last cycle has been superseded by v4.3.1 and v4.3.2, so every AA reading moves to v4.3.2. Unlike the Sep 10 change, what moved is the scoring method rather than the questions: v4.3.1 replaced the pairwise judging panel for AA-Briefcase and GDPval-AA v2 with current models (Claude Opus 5, GPT-5.6 Sol, Gemini 3.8 Flash); v4.3.2 lifted GDPval-AA to v2.1, moved the Elo anchor to DeepSeek V4.1-Flash (max) equals 1600 and switched to a Crowd-BT fit, with AA-Briefcase moving to v1.1 (still anchored at GPT-5.5 medium equals 1000). The composition and weighting of the ten evaluations were untouched, scores barely moved, and the order and rounded integers at the top match last cycle, so no slot changes hands. (c) Two number corrections: Qwen3.8-Max went from 40 to 45, but that is the same name on the board pointing at a different model (it now means the 0902 snapshot from Sep 1 at 45.4, and the original row is marked deprecated at 40.2) rather than one model improving, so the line about Grok 4.6 standing four points above Qwen3.8-Max at the same price has been rewritten as a tie on both the high-efficiency / low-latency card and the closed-model value card, with Grok 4.6 keeping its place because it also has a long-horizon agentic result; and DeepSeek V4.1-Flash is corrected from 40 to 39 (39.456), which does not change the open-weight column. One new object of observation: the Step 5 Preview StepFun released on Sep 20 (600B total parameters, 27B active, intelligence 44, $1.00/$2.70, weights promised for Oct 15), which only AA has, so it goes into the observations and caveats without a place on any list. All three primary data files (DeepSWE, ARC-AGI-2, injection attacks) published no new version for a third cycle running, so the chart and the numbers around it carry over.
2026-09-16The substantive move this cycle comes from DeepSeek changing generations, and it trips two of the three gates: (b) a card list changes, and (c) key figures are corrected against a checkable source. (b) The 'best open-weight, self-hostable' column swaps DeepSeek V4-Pro for V4.1-Flash, shipped Sep 10: 40 on the v4.3 intelligence scale against V4-Pro's 36, total parameters down from 1.6T to 552B and active parameters from 49B to 16B, still MIT licensed with a 1M-token context, which lowers the self-hosting hardware bar by an order of magnitude. (c) Pricing and routing changed: V4.1-Flash runs $0.30/$1.20 per million tokens at peak and $0.15/$0.60 off-peak; from 04:00 UTC on Sep 14 requests to deepseek-v4-pro are served by V4.1-Flash at Flash prices, and deepseek-v4-flash and deepseek-v4-flash-vision-exp are routed to it as well, so the two long-horizon rows ($0.24 and $0.10 per task) now carry API names the board no longer serves; a line was added to the card notes, the observations and the chart note. None of the four boards has a new revision: AA is still v4.3, the DeepSWE primary file is still the Sep 3 revision (generated_at identical character for character), SWE-bench Verified has none, and the ARC-AGI-2 primary JSON endpoint still returns 404, so the Sep 4 reading stands. Chart data and all five scene picks are unchanged. V4.1-Flash has no long-horizon agentic, pure-coding or ARC-AGI-2 result yet, and has been added to the ARC not-submitted list and to the new-arrivals caveat.
How to read this data
This list is not exhaustive: it covers a set of major models we track continuously; newly released or lesser-known ones may not be in yet (Grok 4.5 was missing for a while). Absence does not mean a model is not worth choosing; for anything else, pull a fresh comparison yourself.
Boards measure different things: SWE-bench measures verifiable bug-fixing accuracy, Arena's blind boards measure whether people like what they see. Arena itself splits into WebDev overall, frontend, and fullstack sub-boards whose rankings do not always agree; the 1691 and 1674 figures quoted here are the WebDev overall board, where Kimi K3 is second while it leads the frontend sub-board. Confirm the board measures your use case, confirm which sub-board you are reading, and do not treat one ranking as settled.
Score gaps are often noise: the top few usually sit within each other's error bars, which is a tie. SWE-bench Verified now has its leaders within one percentage point, close to saturation. Do not read one or two points as strength; the real difference is usually cost.
Data on the new arrivals is incomplete: Claude Haiku 5.5 (Oct 7, AA 43 at its highest reasoning setting) has only the intelligence board among third-party results; the vendor-reported agentic results run on its own harness and are not used here. Mistral Large 4 Preview (Oct 6, AA 38, $1.36/$4.18 per million tokens, listed by AA as proprietary) likewise has only that one board and gets no slot. Claude Sonnet 5.5 (Sep 28, AA 56), GPT-6.1 Sol (Sep 29, AA 52) and Gemini 4 Argon (Sep 30, AA 53) have only the intelligence index among third-party boards so far. Sonnet 5.5's self-reported Terminal-Bench 4.0 70.6% and OSWorld 2.1 80.1% run on the vendor's own harness, while AA's standardised run measures Terminal-Bench 4.0 at 64%; GPT-6.1 Sol's self-reported DeepSWE result is not in the public data file. Gemini 4 Argon self-reports 77.9% on DeepSWE v1.1, above the long-horizon leader Astra's 74.1%, but it is not in the Sep 22 public data file, so this page does not count it; it is currently open only to the security defenders in Google's Fairwind Program. Of the Sep 21 to 22 batch, Claude Opus 5.5 (AA 58) has ARC-AGI-2 (High 93.3%) while the two coding boards still have no result for it; Grok 4.7 (Sep 21, 46) and MiMo-V2.6-Pro (Sep 21, 46) still have only the intelligence index, and GPT-6 Luna has only ARC-AGI-2 (Max 59.3%); Opus 5.5's self-reported Terminal-Bench 4.0 66.4% likewise runs on the vendor's own harness. This page puts Opus 5.5, Sonnet 5.5 and GPT-6.1 Sol on the intelligence card, GPT-6.1 Sol on the value card and MiMo-V2.6-Pro on the open-weight card, all as reference points rather than settled results. Step 5 Preview (StepFun, Sep 20) likewise has only the intelligence index (44) and no weights released, so it gets no place. GPT-6 Astra (Sep 3) has long-horizon agentic (74.1%, ±2.9), ARC-AGI-2 (95.0%) and intelligence (53), but SWE-bench Verified still has no row for it, so the pure coding slot has nothing to compare against; its ARC-AGI-3 results differ by 37 percentage points between the standard harness and the vendor adapter, so check which one a figure comes from before using it. Meta's Muse Spark 1.3 (Sep 2) has only the intelligence index (48) so far, with nothing on DeepSWE or ARC-AGI-2. Gemini 3.8 Flash (Sep 2) added ARC-AGI-2 this cycle (High 89.2%, $0.40 per task), alongside long-horizon agentic (73.8%, ±1.4) and intelligence (41); SWE-bench Verified still has no row for it. Claude Fable 5.1 (Sep 1) has intelligence (53) and ARC-AGI-2 (90.0%) but no row on either coding board. DeepSeek V4.1-Flash (Sep 10) has only the intelligence index so far (39 this cycle), with nothing on long-horizon agentic work, SWE-bench Verified or ARC-AGI-2, so the open-weight column lists it on parameter count and intelligence. The re-generation Qwen3.8-Max 0902 (Sep 1) likewise has only the intelligence index (45), and the coding numbers the vendor posts itself run on its own harness and are not comparable with the standardised runs cited here. Zhipu's GLM-5.3 weights were released only on Aug 31, independent replication has not caught up, and it has not been submitted to ARC-AGI-2; GLM-5.3-Flash still has just the one long-horizon result (63.4%, ±4.4). The ARC-AGI-2 results of Astra, Fable 5.1, Opus 5.5, Gemini 3.8 Flash and GPT-6 Luna, along with the long-horizon points of Gemini 3.7 Flash and DeepSeek V4-Pro, are also not yet plotted on the chart below. For all of these, run your own small batch before committing to large volumes or betting on the licence terms.
Not sure which model fits you?
This page is a general comparison. Your actual choice also depends on your task, budget, usage volume, and error tolerance. Copy a question below into the AI you already use and let it compare for your situation.