Cheapest coding agent per solved issue: ten harnesses compared
On DeepSWE with every harness on DeepSeek V4 Flash, CodeAF cost 39.9 cents per solved task, the lowest of ten; the next was 1.9x that. Method and limits.
The cheapest coding agent per solved issue in AgentField's DeepSWE comparison was CodeAF in senior-dev mode: $24.73 of model spend for 62 solved tasks, 39.9 cents per solve. The next cheapest, mini-swe-agent, paid 76.9 cents, 1.9x as much, and Codex CLI 82.8 cents. Every harness ran the same model, deepseek/deepseek-v4-flash-0731 through OpenRouter, on the same 113 tasks.
Cost per task and cost per solve are different measures. An agent that gives up early is cheap per task and expensive per solve. The number that tells you what working code costs is spend divided by the tasks it actually solved.
At a glance
| Harness | Solved (of 113) | Model spend | Per solved task | Against CodeAF |
|---|---|---|---|---|
| CodeAF (senior-dev) | 62 | $24.73 | $0.399 | 1x |
| mini-swe-agent | 56 | $43.09 | $0.769 | 1.9x |
| Codex CLI | 51 | $42.23 | $0.828 | 2.1x |
| Pi | 42 | $39.61 | $0.943 | 2.4x |
| Claude Code | 16 | $21.72 | $1.357 | 3.4x |
| Oh My Pi | 31 | $55.98 | $1.806 | 4.5x |
| Kilo Code | 30 | $54.65 | $1.822 | 4.6x |
| OpenCode | 30 | $56.96 | $1.899 | 4.8x |
| Muse Code | 3 | $13.54 | $4.513 | 11.3x |
| DeepSeek Harness | 16 | $169.47 | $10.592 | 26.6x |
Spend is each harness's billed OpenRouter spend for the whole 113-task pass, from arms.csv. Each harness had its own OpenRouter key, so the bills do not mix.
Method
The comparison is on the main DeepSWE results page: the full DeepSWE set, the official verifiers at 0b9fabb, 3 hours per task, one pass per harness. "Solved" counts tasks the verifier passed. "Per solved task" is total spend divided by tasks solved, so the cost of failed attempts is included in the price of each success, which is what you pay in practice.
Results
CodeAF solved the most tasks and had the third-lowest total bill. It spent less than mini-swe-agent, Codex CLI and Pi while solving more than any of them. Two harnesses spent less in total, Muse Code ($13.54) and Claude Code ($21.72), but solved 3 and 16 tasks, so each of their solves cost more.
The model price was the same for everyone. What differs is how many tokens an agent spends before it reaches a correct patch, and how often it reaches one.
Where the other tools did better
- Lowest cost per task. Muse Code (12 cents) and Claude Code (19 cents) cost less per task than CodeAF (22 cents), because they finished tasks sooner and solved far fewer.
- Lowest total bill. Muse Code and Claude Code each spent less in total than CodeAF.
Limits of this result
- It is AgentField's own comparison, one pass each, not a public leaderboard.
- Spend depends on the model. On a more expensive model every row rises; the multiples are what this run shows.
- CodeAF used the provider's default sampling; the other nine sent temperature 1.0 and top-p 0.95. CodeAF ran a day later, on 2026-09-12 UTC.
- The Claude Code row is Claude Code driving DeepSeek V4 Flash through an Anthropic-compatible endpoint, not Claude Code with Claude models.
Questions people ask
What is the most cost-efficient AI coding agent?
In AgentField's DeepSWE comparison, with ten harnesses on DeepSeek V4 Flash, CodeAF in senior-dev mode had the lowest cost per solved task: 39.9 cents. mini-swe-agent paid 1.9x that, Codex CLI 2.1x and Claude Code 3.4x.
Why measure cost per solved issue instead of cost per task?
Because a failed attempt still costs money. An agent that stops early looks cheap per task while producing little working code. Dividing total spend by tasks solved prices each success with the failures it took to get there.
How much does CodeAF cost to run?
CodeAF itself is free and open source under Apache 2.0; you pay only your model provider, or nothing with a local model. In this comparison the model bill came to 22 cents a task and 39.9 cents per solved task on DeepSeek V4 Flash.
Is Claude Code cheaper than CodeAF?
Per task, yes, in this run: 19 cents against 22. Per solved task, no: Claude Code solved 16 tasks for $21.72, $1.36 each, against CodeAF's 39.9 cents. Both ran DeepSeek V4 Flash, so this says nothing about Claude Code's cost with Claude models.
Sources
arms.csvand the campaign README, checked 2026-11-21.- CodeAF FAQ, for licence and pricing, checked 2026-11-21.
Last updated 2026-11-21. Every figure computed from arms.csv.