CodeAFRun a hundred jobs at once

Run a hundred jobs at once

One brief becomes a hundred parts, one per file, and all hundred run side by side under a cap you set.

If a job splits into a hundred small ones, it should take as long as one small one.
  • a hundred product pages
  • a doc comment for every exported function
  • one translation per locale file

Some work is a hundred copies of one small job: a summary for every product page, a docstring for every function, a check for every config file. In this playbook one /task brief asks for one part per file, the run splits into a hundred parts, and they run side by side while a per-task cap keeps the bill bounded. You get one branch with all hundred files done.

Steps

Our repository, catalog, has 100 product pages in products/. Each needs front matter: a summary of at most 20 words and one category. check.py checks a file.

  1. In the repository, run codeaf and send any first message.
  2. Cap what the task may spend: type /crew cap task 3 and press Enter. CodeAF answers per-task limit · $3.
  3. Hand over all hundred in one brief:
/task Give all 100 product pages in products/ their front matter. Hand them all out at once: split the work into 100 parts, one part per file, and do not script it. Each part reads its one file, writes a summary in its own words (at most 20 words, what the product is and why someone would want it) and picks one category, puts the front matter the README describes above the title, and runs python3 check.py on that file. At the end run python3 check.py products/*.md and report how many pass.
  1. The Tasks column fills with Front matter for 001.md down to Front matter for 100.md, each with its own timer.
  2. When the run ends, land it:
if python3 check.py products/*.md passes on that branch, merge it into main

What you see

The run ended done · 2m26s · 100 files · branch kept, and its summary said all 100 pages got front matter through 100 per-file parts. On the branch, python3 check.py products/*.md printed 100 of 100 ok. Each summary was written for its product, for example "Lightweight recycled-aluminium desk lamp with a 20-hour battery, handmade in a Porto workshop and often given as a gift."

The words "split the work into 100 parts" and "do not script it" matter. Our first brief said only "one part per file", and one worker wrote a script that filled all 100 files in 40 seconds for $0.08, using each page's first sentence as its summary. Fast and cheap, but not what we asked for.

In our run

A hundred parts in 2 minutes and 26 seconds of wall time, for $1.52 against an estimate of $0.22, well inside the $3 cap. One part needed a second pass, and the categories came out 20, 20, 20, 19 and 21.

Make it yours

  • Go bigger. A run splits into at most 256 parts at a time and 1024 in all, so a few hundred files is still one brief.
  • Set the ceiling first. /crew cap task puts a hard dollar limit on the whole run. A very small cap also makes CodeAF pick a cheaper crew before it starts. See Budgets at every level.
  • Choose your trade. When the job really is mechanical, let it script: our scripted pass was 20 times cheaper. When each item needs judgement, ask for the parts.

Go further

Budgets at every level: Spending limits on the day, each conversation, each task and each team, so you can hand work off without a surprise bill.

Coming soon: describe the org you want and CodeAF builds it: teams, managers, budgets, standing orders.