CodeAF vs mini-swe-agent: full coding agent or minimal baseline?
mini-swe-agent is a 100-line research baseline; CodeAF is a full terminal coding agent. On DeepSWE, same model, CodeAF solved 62 tasks to 56.
mini-swe-agent and CodeAF both drive any model through a terminal, but they are built for different jobs. mini-swe-agent, from the Princeton and Stanford team behind SWE-bench, is "the 100 line AI agent" with no tools other than bash: a hackable baseline for research and evaluation, and the harness DeepSWE's own leaderboard uses for every model. CodeAF by AgentField is a full terminal coding agent that runs many tasks from one window, keeps each on its own branch, and has a long-task mode, senior-dev.
On AgentField's DeepSWE comparison, with both on deepseek/deepseek-v4-flash-0731, CodeAF in senior-dev mode solved 62 of 113 tasks and mini-swe-agent 2.4.6 solved 56, the closest result among the ten harnesses tested. CodeAF's cost per solved task was 39.9 cents against mini-swe-agent's 76.9.
At a glance
| CodeAF | mini-swe-agent | |
|---|---|---|
| What it is | Terminal coding agent for daily work and unattended tasks | Minimal agent, "just some 100 lines of python" for the agent class (README) |
| Made by | AgentField | The Princeton and Stanford team behind SWE-bench and SWE-agent |
| Licence | Apache 2.0 (repo) | MIT (repo) |
| Models | OpenRouter, DeepSeek, GLM, Kimi, MiniMax, Qwen, Codex plan, Ollama, any OpenAI-compatible endpoint (FAQ) | "All models via litellm, openrouter, portkey, and more" |
| Tools | File editing, search, shell, tasks and subagents | Bash only, by design |
| Many tasks at once | Yes, each in an isolated copy on its own branch | One linear run per instance; scale by running more |
| Long unattended work | senior-dev mode, codeaf senior-dev --max-cost 2 --json "<brief>" | One run until the step or cost limit |
| DeepSWE, same model | 62 of 113 (54.9%) | 56 of 113 (49.6%) |
| Cost per solved task | 39.9 cents | 76.9 cents (1.9x) |
Where CodeAF won on DeepSWE
The rows come from the per-task records of the comparison on the DeepSWE results page, every harness on DeepSeek V4 Flash, one pass each.
| Measure | CodeAF | mini-swe-agent |
|---|---|---|
| Tasks solved, all 113 | 62 | 56 |
| Medium and hard tasks, 76 | 36 | 30 |
| Go tasks, 34 | 21 | 17 |
| Python tasks, 34 | 20 | 18 |
| Mean fail-to-pass tests passing | 88.64% | 88.17% |
| Model spend for the pass | $24.73 | $43.09 |
| Solved by one but not the other | 29 | 23 |
Where mini-swe-agent did better
- TypeScript: 16 of 35 against CodeAF's 15.
- Kept tests: 99.66% of existing tests still passing, against CodeAF's 99.42%.
- Unique solves: 7 tasks no other harness solved, against CodeAF's 5.
- Speed: 44 minutes of agent time per task on average, against CodeAF's 54.
- Simplicity: the whole agent can be read in minutes, which is why researchers use it as a baseline. Its own README points to DeepSWE's blog for a result where it "beats Claude Code and Codex on DeepSWE".
Which one to choose
Pick mini-swe-agent to study a model, to run a clean evaluation baseline, or to build your own agent on a small, readable core. Pick CodeAF to work in a repository day to day, run several tasks at once on separate branches, or hand a long change to an agent that keeps going for hours on the model you choose.
Limits of the benchmark rows
The DeepSWE rows are AgentField's own comparison, not a public leaderboard, with one pass per harness. CodeAF used the provider's default sampling while mini-swe-agent sent temperature 1.0 and top-p 0.95, and CodeAF ran a day later, on 2026-09-12 UTC. mini-swe-agent 2.4.6 was about seven weeks old at the time but still its latest release. A six-task gap on one pass is small.
Questions people ask
Is CodeAF better than mini-swe-agent?
For day-to-day coding work, CodeAF does more: parallel tasks on separate branches, file tools and a long-task mode. On AgentField's DeepSWE comparison, both on DeepSeek V4 Flash, CodeAF solved 62 of 113 tasks and mini-swe-agent 56, at about half the cost per solved task. For research baselines, mini-swe-agent's simplicity is the point.
Why does DeepSWE's leaderboard use mini-swe-agent?
DeepSWE's leaderboard ranks models, and runs them all inside one minimal harness so the model is the only thing that changes. AgentField's comparison does the opposite: it holds the model fixed and changes the harness.
Can mini-swe-agent and CodeAF use the same models?
Mostly. mini-swe-agent supports models through litellm, OpenRouter and Portkey. CodeAF supports OpenRouter, DeepSeek, GLM, Kimi, MiniMax, Qwen, a Codex plan, Ollama and any OpenAI-compatible endpoint.
Sources
- mini-swe-agent: README and 2.4.6 release, checked 2026-11-21.
- CodeAF: README, FAQ, senior-dev docs, checked 2026-11-21.
- DeepSWE comparison: per-task records and
arms.csv, checked 2026-11-21.
Last updated 2026-11-21. Benchmark rows are CodeAF senior-dev mode and mini-swe-agent 2.4.6.