Last updated · Santosh Kumar Radha

CodeAF vs mini-swe-agent: full coding agent or minimal baseline?

mini-swe-agent is a 100-line research baseline; CodeAF is a full terminal coding agent. On DeepSWE, same model, CodeAF solved 62 tasks to 56.

mini-swe-agent and CodeAF both drive any model through a terminal, but they are built for different jobs. mini-swe-agent, from the Princeton and Stanford team behind SWE-bench, is "the 100 line AI agent" with no tools other than bash: a hackable baseline for research and evaluation, and the harness DeepSWE's own leaderboard uses for every model. CodeAF by AgentField is a full terminal coding agent that runs many tasks from one window, keeps each on its own branch, and has a long-task mode, senior-dev.

On AgentField's DeepSWE comparison, with both on deepseek/deepseek-v4-flash-0731, CodeAF in senior-dev mode solved 62 of 113 tasks and mini-swe-agent 2.4.6 solved 56, the closest result among the ten harnesses tested. CodeAF's cost per solved task was 39.9 cents against mini-swe-agent's 76.9.

At a glance

CodeAFmini-swe-agent
What it isTerminal coding agent for daily work and unattended tasksMinimal agent, "just some 100 lines of python" for the agent class (README)
Made byAgentFieldThe Princeton and Stanford team behind SWE-bench and SWE-agent
LicenceApache 2.0 (repo)MIT (repo)
ModelsOpenRouter, DeepSeek, GLM, Kimi, MiniMax, Qwen, Codex plan, Ollama, any OpenAI-compatible endpoint (FAQ)"All models via litellm, openrouter, portkey, and more"
ToolsFile editing, search, shell, tasks and subagentsBash only, by design
Many tasks at onceYes, each in an isolated copy on its own branchOne linear run per instance; scale by running more
Long unattended worksenior-dev mode, codeaf senior-dev --max-cost 2 --json "<brief>"One run until the step or cost limit
DeepSWE, same model62 of 113 (54.9%)56 of 113 (49.6%)
Cost per solved task39.9 cents76.9 cents (1.9x)

Where CodeAF won on DeepSWE

The rows come from the per-task records of the comparison on the DeepSWE results page, every harness on DeepSeek V4 Flash, one pass each.

MeasureCodeAFmini-swe-agent
Tasks solved, all 1136256
Medium and hard tasks, 763630
Go tasks, 342117
Python tasks, 342018
Mean fail-to-pass tests passing88.64%88.17%
Model spend for the pass$24.73$43.09
Solved by one but not the other2923

Where mini-swe-agent did better

  • TypeScript: 16 of 35 against CodeAF's 15.
  • Kept tests: 99.66% of existing tests still passing, against CodeAF's 99.42%.
  • Unique solves: 7 tasks no other harness solved, against CodeAF's 5.
  • Speed: 44 minutes of agent time per task on average, against CodeAF's 54.
  • Simplicity: the whole agent can be read in minutes, which is why researchers use it as a baseline. Its own README points to DeepSWE's blog for a result where it "beats Claude Code and Codex on DeepSWE".

Which one to choose

Pick mini-swe-agent to study a model, to run a clean evaluation baseline, or to build your own agent on a small, readable core. Pick CodeAF to work in a repository day to day, run several tasks at once on separate branches, or hand a long change to an agent that keeps going for hours on the model you choose.

Limits of the benchmark rows

The DeepSWE rows are AgentField's own comparison, not a public leaderboard, with one pass per harness. CodeAF used the provider's default sampling while mini-swe-agent sent temperature 1.0 and top-p 0.95, and CodeAF ran a day later, on 2026-09-12 UTC. mini-swe-agent 2.4.6 was about seven weeks old at the time but still its latest release. A six-task gap on one pass is small.

Questions people ask

Is CodeAF better than mini-swe-agent?

For day-to-day coding work, CodeAF does more: parallel tasks on separate branches, file tools and a long-task mode. On AgentField's DeepSWE comparison, both on DeepSeek V4 Flash, CodeAF solved 62 of 113 tasks and mini-swe-agent 56, at about half the cost per solved task. For research baselines, mini-swe-agent's simplicity is the point.

Why does DeepSWE's leaderboard use mini-swe-agent?

DeepSWE's leaderboard ranks models, and runs them all inside one minimal harness so the model is the only thing that changes. AgentField's comparison does the opposite: it holds the model fixed and changes the harness.

Can mini-swe-agent and CodeAF use the same models?

Mostly. mini-swe-agent supports models through litellm, OpenRouter and Portkey. CodeAF supports OpenRouter, DeepSeek, GLM, Kimi, MiniMax, Qwen, a Codex plan, Ollama and any OpenAI-compatible endpoint.

Sources

Last updated 2026-11-21. Benchmark rows are CodeAF senior-dev mode and mini-swe-agent 2.4.6.