Job 01 · Not yet tested
Which coding agent should I use to fix bugs and add features in my repo?
Agents that edit a real repository: fix three failing tests and add one feature, in a small TypeScript project, under one time limit.
People who want an agent to change a repo, not only talk about the diff.
The standard test
Pack 0.1.0 · specified · Small TypeScript repo
Fixed small TS repo with 3 failing tests + 1 feature; score tests passed, human fixes, time, cost
Setup
Every tool gets the same snapshot of a small TypeScript package. The test runner is already configured. Three tests fail for reasons written down before the run. A separate brief asks for one feature. The tool may edit files and run the test command. It does not get hints about the failures.
Inputs
The repository snapshot, the feature brief, and permission to edit and run tests. No other context.
Time limit
One wall-clock cap for every tool in the job. The number is published with the pack before the first run and does not move between tools.
What we record
- Tests passed
- Human fixes required
- Time
- Cost
- Tool version
- Plan
How the FitScore is built
FitScore = weighted average × 20, rounded to the nearest integer. The weighted average is the sum of (criterion score × weight) divided by 100.
- Output quality30
- Reliability15
- Cost and free tier15
- Speed10
- Setup friction10
- Agent, API, and MCP support10
- Privacy and data10
5 excellent · 4 good · 3 usable · 2 poor · 1 fails. Full method.
Candidates
Candidates for Coding agents. This order is the launch list, not a ranking. No FitScores are published.
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Version and plan recorded when a test is published
Questions this page answers
- Which coding agent should I use to fix bugs and add features in my repo?
- FitForTool has not published a FitScore for this job. The launch candidates are Claude Code, OpenAI Codex, Cursor, GitHub Copilot coding agent, Aider, Cline, and Gemini CLI. Each one is listed as not yet tested. A ranking is published only after every tool in the comparison has had the same test pack run, and a person has reviewed the result. Until that page shows a FitScore, there is no winner.
- How will FitForTool test coding agents?
- Fixed small TS repo with 3 failing tests + 1 feature; score tests passed, human fixes, time, cost. Every tool gets the same snapshot of a small TypeScript package. The test runner is already configured. Three tests fail for reasons written down before the run. A separate brief asks for one feature. The tool may edit files and run the test command. It does not get hints about the failures. We record tests passed, human fixes required, time, cost, tool version, and plan. Those notes are then scored from 1 to 5 on seven shared criteria. The pack has been specified and has not been run.
- Can a vendor pay to rank higher on this job?
- No. FitForTool does not sell position or score. A sponsorship, if we run one later, is labelled and sits outside the ranked list. An affiliate link is disclosed and does not change order. A vendor may ask for a retest, and we rerun the same pack. Sponsors do not see the result before it is public.
- What does "Not yet tested" mean?
- The tool is a candidate for this job. There is no published hands-on run, so there is no FitScore, no criterion score, and no rank. Research-only is a different label, used when we have read the docs and still have not run the pack. Neither label is a score.