Skip to content
, ,

Coding Agent Index v2.1 Explained: Full Leaderboard, Costs and the Claude Tie

10–15 minutes
Woman in an orange satin shirt in a server room holding a card with the Claude logo, hook: Claude ties for top spot

Artificial Analysis released Coding Agent Index v2.1 on 13 November 2026. It swaps in harder tests, and Claude Fable 5.2 and Claude Opus 5.5 now tie at the top with a score of 58.

This guide explains what the index measures and shows the full v2.1 ranking with cost per task. It also covers how to use the numbers when you pick between Claude Code, Codex, Antigravity CLI and the rest.

TLDR

  • What it is: an equal-weight average of three agent benchmarks, 322 tasks in total, each run 3 times.
  • New in v2.1: FrontierCode-AA 2.0 (132 tasks) replaces FrontierCode v1.1, and Terminal-Bench 4.1 replaces 4.0.
  • Top score: four setups tie at 58. Three run in Claude Code (Fable 5.2 High, Fable 5.2 Xhigh, Opus 5.5 Max) and one is Devin Fusion CLI.
  • Best value at the top: Claude Code with Fable 5.2 High scores 58 for $7.51 per task. The Xhigh setting scores the same for $15.47.
  • Cheap and strong: Codex with GPT-6.1 Sol (Max) scores 52 for just $1.51 per task.

What the Coding Agent Index Measures

Most leaderboards score a model on its own. The Coding Agent Index scores an agent variant: a harness such as Claude Code or Codex, plus a model, plus its settings. Change the reasoning effort and you get a separate row.

That matters because the same model can score very differently in two harnesses. The harness decides how files are read, how commands run and when the agent stops.

How the score is built

  • Three runs per task: every task is attempted 3 times and the attempts are averaged.
  • Equal task weight: task scores are averaged inside each benchmark.
  • Equal benchmark weight: the index is a simple average of the three benchmark scores, so each one counts for a third.
  • Failures score zero: errors, time-outs and safety refusals all count as 0 for that attempt.

What else the page reports

Next to the score, Artificial Analysis lists cost per task, time per task and tokens per task. Cost is the pay-per-token API price, not your subscription price.

It also reports safety refusal rates and reward hacking rates. Reward hacking here means an attempt that passed but was then zeroed by integrity checks, for example for tampering with tests.

The Three Benchmarks Inside v2.1

The index now has 322 tasks. Here is what each part tests and how it is graded.

Benchmark Tasks What it tests How it is scored
FrontierCode-AA 2.0 132 (private) Real changes to open-source repositories, judged the way a maintainer would 0 if any blocking check fails, otherwise 70% to 100%
Terminal-Bench 4.1 66 Agentic work in a terminal: installing, configuring, debugging, scripting Test suite pass or fail (pass@1)
SWE-Atlas-QnA-AA 124 Tracing code and explaining how a repository behaves Pass only if every rubric item passes; editing tracked files fails the task

FrontierCode-AA 2.0

FrontierCode comes from Cognition, the team behind Devin. Version 2.0 uses Cognition’s rule-based integrity scanner plus an AI judge. The rubric judge is GPT-6 Luna (medium).

AlphaSignal’s summary says 2.0 uses 55% more repositories and 43% more blocking criteria per task than before. It is much harder. Across 23 setups tested on both versions, the mean score fell from 45% to 29%.

Terminal-Bench 4.1

Terminal-Bench 4.1 keeps the same 66 tasks as 4.0 but updates many of their verifiers. Artificial Analysis says every one of the 23 setups scored on both versions moved by less than 3 points.

SWE-Atlas-QnA-AA

This part asks the agent questions about a codebase. The judge is GPT-6 Luna (xhigh) with five votes per criterion. An answer passes only if all rubric items pass, so partial answers get no credit.

Note: Two of the three benchmarks are graded partly by an OpenAI model (GPT-6 Luna), and the reward-hacking reviews use Claude models. That is normal for benchmark labs, but keep it in mind when vendors quote these scores.

What Changed From v2.0

  • FrontierCode: v1.1 (100 tasks) is replaced by FrontierCode-AA 2.0 (132 tasks).
  • Terminal-Bench: 4.0 is upgraded to 4.1, with the same tasks and stricter verifiers.
  • SWE-Atlas-QnA-AA: unchanged at 124 tasks.
  • Cost method: now matches the Intelligence Index, using live cache-hit measurements.
  • Total tasks: 322, up from 290.

Scores dropped across the board. AlphaSignal reports the average index score fell by about 5 points. Claude Code with Opus 5.5 Max scored 61.7 on v2.0 and now scores 58, which is a test change rather than a weaker model.

Full v2.1 Leaderboard With Cost per Task

These rows come from the v2.1 tables on Artificial Analysis comparison pages, read on 15 November 2026. Scores are rounded as AA shows them. “FC” is FrontierCode-AA 2.0, “TB” is Terminal-Bench 4.1 and “QnA” is SWE-Atlas-QnA-AA.

Agent Model (effort) Index FC TB QnA Cost / task Time / task
Claude Code Claude Fable 5.2 (Xhigh) 58 49 64 62 $15.47 53.3m
Claude Code Claude Fable 5.2 (High) 58 49 64 62 $7.51 31.3m
Claude Code Claude Opus 5.5 (Max) 58 44 65 66 $12.61 1.1h
Devin Fusion CLI Opus 5.5 Max + GPT-6 Luna High 58 44 63 68 $8.75 54.9m
Claude Code Claude Sonnet 5.5 (Max) 56 39 66 64 $11.40 1.4h
Claude Code Claude Fable 5.2 (Max) 56 44 63 60 $44.22 2.2h
Antigravity CLI Gemini 4 Carbon (High) 55 38 65 64 $7.13 33.6m
Muse Code Muse Wonder Full (Max) 54 35 60 69 not shown 33.7m
Claude Code Claude Fable 5.2 (Medium) 53 36 60 64 $2.89 15.1m
Codex GPT-6 Astra (Max) 53 43 58 59 $7.63 26.0m
Claude Code Claude Fable 5.1 (Max) 52 33 58 66 $11.70 36.3m
Codex GPT-6.1 Sol (Max) 52 42 54 60 $1.51 21.9m
Codex GPT-6.1 Terra (Max) 49 41 51 54 $1.53 28.1m
Kimi Code CLI Kimi K3.1 Pro (Max) 49 22 67 57 $2.94 41.8m
Claude Code Claude Fable 5.2 (Low) 48 31 53 60 $2.49 13.4m
Antigravity CLI Gemini 4 Argon (High) 45 25 57 55 $6.84 37.7m
Muse Code Muse Wonder (Max) 44 24 47 62 $7.59 27.2m
Grok Build Grok 4.7 (Xhigh) 42 30 32 63 $15.35 36.6m
Mistral Vibe Mistral Large 4 (High) 41 20 42 59 $1.34 49.0m
Opencode GLM-5.3 (Max) 40 21 40 59 $5.41 52.8m
Muse Code Muse Ember (Max) 40 21 43 55 $0.22 19.9m
DeepSeek Harness DeepSeek V4.1 Flash 1017 (Max) 37 21 28 62 $0.32 31.4m
Muse Code Muse Spark 1.3 (Max) 37 22 30 59 $4.34 24.4m
Claude Code Step 5 Preview 36 18 30 59 $3.55 59.5m
Claude Code Qwen3.8 Max 33 25 17 56 $4.03 1.4h
Claude Code Claude Haiku 5.5 (Max) 30 23 32 34 $2.25 1.3h
Mistral Vibe Mistral Large 4 (Medium) 30 18 26 45 $1.02 43.1m
Codex GPT-6 Luna (Max) 26 24 15 38 $0.19 23.5m

Warning: This table only lists rows we could confirm on v2.1. GitHub Copilot (Copilot Fusion scored 60 on v2.0) and a few smaller setups did not show a v2.1 score at the time of writing, so they are left out rather than guessed.

Category leaders

  • FrontierCode-AA 2.0: Claude Fable 5.2 (High and Xhigh) lead at 49%, five points above Opus 5.5 Max.
  • Terminal-Bench 4.1: Kimi Code CLI with Kimi K3.1 Pro is top at 67 in the table (AA’s post on X gives 68%), with Sonnet 5.5 Max at 66.
  • SWE-Atlas-QnA-AA: Muse Wonder Full leads at 69, then Devin Fusion CLI at 68.

Cost vs Score: Where the Value Is

The cost column is where v2.1 gets interesting. Among setups scoring 48 or more, AlphaSignal notes costs vary about 30-fold.

Setup Index Cost / task Points per dollar
Codex + GPT-6.1 Sol (Max) 52 $1.51 34.4
Codex + GPT-6.1 Terra (Max) 49 $1.53 32.0
Claude Code + Fable 5.2 (Low) 48 $2.49 19.3
Claude Code + Fable 5.2 (Medium) 53 $2.89 18.3
Antigravity CLI + Gemini 4 Carbon (High) 55 $7.13 7.7
Claude Code + Fable 5.2 (High) 58 $7.51 7.7
Devin Fusion CLI 58 $8.75 6.6
Claude Code + Opus 5.5 (Max) 58 $12.61 4.6
Claude Code + Fable 5.2 (Xhigh) 58 $15.47 3.7
Claude Code + Fable 5.2 (Max) 56 $44.22 1.3

Points per dollar is our own simple maths (index divided by cost). It is a rough guide, not an AA metric.

Three things stand out

  • More effort is not always better: Fable 5.2 at Max scores 56, lower than High and Xhigh at 58, while costing almost six times more than High.
  • Time-outs explain part of it: AlphaSignal reports Fable 5.2 Max hit time limits on 16% of FrontierCode attempts and 13% of SWE-Atlas attempts, against 1% and 0% at Xhigh.
  • Codex is the budget pick: GPT-6.1 Sol gets within 6 points of the top for about a fifth of the cost of Fable 5.2 High.
Quick answer: If you pay per token, Claude Code with Fable 5.2 on High gives the top score for the lowest price among the 58s. If cost matters most, Codex with GPT-6.1 Sol is the value leader.

What It Means for Picking a Coding Agent

The index is a useful starting point, but your choice also depends on how you pay and where you work. Here is how the main tools look through the v2.1 lens.

Claude Code

Claude Code holds three of the four top spots. The real decision is which model and effort to use. Fable 5.2 High is the sweet spot on this index.

Opus 5.5 Max is stronger on SWE-Atlas (66 vs 62) if your work is more “explain this codebase” than “write this patch”. For the model itself, see our Claude Fable 5.2 guide.

Codex

Codex sits a few points lower but is far cheaper per task. GPT-6 Astra (Max) scores 53 at $7.63, while GPT-6.1 Sol (Max) scores 52 at $1.51. For a deeper look at both tools, read Codex vs Claude Code.

Antigravity CLI (formerly Gemini CLI)

Google has been moving Gemini CLI users to Antigravity CLI, so this is the Google row on the index. Gemini 4 Carbon (High) scores 55 at $7.13, and its Terminal-Bench score of 65 is only 2 points off the top.

Cursor, Windsurf and Cline

These editor-based tools do not appear in the v2.1 rows we could confirm. So the index cannot rank them directly. Our Cursor vs Windsurf vs Claude Code vs Cline comparison covers them on features and price instead.

Tip: Many editor tools let you pick the model underneath, such as a Claude or GPT model. The model rows above give a rough hint, but harness differences can move scores by several points.

How to Read the Leaderboard Step by Step

  1. Open the Coding Agent Index page. Go to artificialanalysis.ai/agents/coding-agents and check the version label says v2.1.
  2. Find your harness. Look for the tool you actually use, such as Claude Code, Codex or Antigravity CLI.
  3. Match the effort setting. Check the effort in brackets (Low, Medium, High, Xhigh, Max). Your own default may differ from the tested one.
  4. Read the sub-scores. Pick the benchmark closest to your work: FrontierCode for patches, Terminal-Bench for DevOps and shell work, SWE-Atlas for code understanding.
  5. Check cost and time. Compare cost per task and time per task, not just the index. A 2-point gain may cost 5 times more.
  6. Treat small gaps as ties. Agent runs vary from attempt to attempt, so a 1 to 3 point gap is not a clear win.
  7. Try your top two on real tasks. Run both on a few tickets from your own repo before you commit to a plan.

Limits of the Index

  • API cost, not plan cost: cost per task uses pay-per-token prices. It excludes subscriptions, infrastructure, engineering and supervision costs.
  • Only three task types: there is no front-end design, no long multi-day project and no human-in-the-loop work.
  • Private tasks: FrontierCode-AA 2.0 tasks are not public, so you cannot inspect them yourself.
  • AI judges: two benchmarks are graded partly by GPT-6 Luna, which can make mistakes.
  • Fallbacks: some attempts are finished by a fallback model. AlphaSignal reports 9% of attempts for Devin Fusion CLI and for Claude Code with Opus 5.5 Max.
  • Snapshots age fast: new models and harness updates land every few weeks. The index has moved from v1.0 in May-June 2026 to v2.1 in November.

For a wider view across chat, reasoning and coding, see our best AI model leaderboard for November 2026.

My Take

The headline tie is real, but the bigger story is cost. Four setups share the top score, yet their price per task ranges from $7.51 to $15.47. Fable 5.2 on High looks like the smart default for Claude Code users.

I also like that v2.1 made the hard part harder. FrontierCode 2.0 cut the average by 16 points, which gives more room to separate agents over the next few months.

Still, no single number should pick your tool. Use the index to make a shortlist, then test on your own code.

Common Problems and Fixes (Misreading the Index)

Problem: “My agent got worse since last month”

Fix: Check the version. v2.1 uses harder tests, so most scores dropped. Opus 5.5 Max went from 61.7 to 58 without any model change.

Problem: “Max effort must be the best setting”

Fix: Not on this index. Fable 5.2 Max scores 56 while High scores 58, partly because Max runs out of time more often.

Problem: “The cost column does not match my bill”

Fix: The column is pay-per-token API cost. If you use a Claude Max or ChatGPT plan, your real cost depends on plan limits, not on per-token prices.

Problem: “A model scores well, so any tool using it will too”

Fix: The score belongs to the harness plus model pair. Opus 5.5 Max scores 58 in both Claude Code and Devin Fusion CLI, but the sub-scores and costs differ.

Problem: “My tool is not on the list, so it must be bad”

Fix: Missing means not tested in this version, not failed. Cursor and several editor tools simply have no v2.1 row.

Problem: “One point ahead means clearly better”

Fix: Treat gaps of 1 to 3 points as a tie. Look at the sub-score that matches your work and at cost per task instead.

Frequently Asked Questions

When was Coding Agent Index v2.1 released?

Artificial Analysis released v2.1 on 13 November 2026, a few weeks after v2.0 launched on 23 October 2026.

Which coding agent is number one on v2.1?

Four setups tie at 58: Claude Code with Fable 5.2 High, Claude Code with Fable 5.2 Xhigh, Claude Code with Opus 5.5 Max, and Devin Fusion CLI with Opus 5.5 Max plus GPT-6 Luna High.

How many tasks are in the index?

There are 322 tasks: 132 in FrontierCode-AA 2.0, 66 in Terminal-Bench 4.1 and 124 in SWE-Atlas-QnA-AA. Each task is attempted three times.

Why did all the scores go down?

FrontierCode-AA 2.0 is much harder than v1.1. The mean FrontierCode score fell from 45% to 29%, which pulled the average index down by about 5 points.

What is the cheapest strong setup?

Codex with GPT-6.1 Sol (Max) scores 52 at $1.51 per task. In Claude Code, Fable 5.2 on Medium scores 53 at $2.89 per task.

Is Cursor or GitHub Copilot on the v2.1 leaderboard?

We could not confirm a v2.1 row for either. GitHub Copilot’s Copilot Fusion scored 60 on v2.0, but its v2.1 score was not shown when we checked.

Does the cost include my Claude or ChatGPT subscription?

No. Cost per task is based on pay-per-token API prices. It does not cover subscriptions, infrastructure or the time people spend reviewing the work.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Woman in an orange satin shirt in a server room holding a card with the Claude logo, hook: Claude ties for top spot