Show HN: Parsimony - an open-source benchmark for how much code AI models add

Parsimony is an open-source benchmark for measuring how much code AI models add on coding tasks. Its starting point is a dimension the author sees as a gap: existing benchmarks only measure solve rate and cost, do not penalize redundant code, and redundant code increases the long-term maintenance difficulty of a codebase.

So its ranking rule is the opposite of common leaderboards—models are ranked by their average net added units on solved tasks, with the lowest value ranked first. In other words, given the same problems solved, the less written, the higher the rank.

Parsimony does not count by lines but instead counts "coding units" such as syntactic elements (statements, calls, comparisons, names, literals), so padding code earns no reward. Net units equal added units minus deleted units; deleted code is subtracted rather than ignored.

This design means it rewards "small net increase," not "few changed lines." Rewriting a loop as a one-line expression may change little in line count, but it will be faithfully reflected in unit count.

Solve status is taken from published SWE-bench Verified results, and tests are not rerun. The evaluation method applies each patch to the task code and parses the two versions of the changed files, without running any code.

In scope, Parsimony only measures Python and excludes tests, documentation, and generated files. In the leaderboard, "solved task count" comes from SWE-bench Verified, while "measurable count" indicates how many solved attempts can be measured.

Notably, the "net added units (mean)" metric also includes failed attempts, while the "net added units on solved tasks (mean)" used for ranking only counts solved tasks. Both measures coexist, and readers need to see clearly which one they are looking at.

All models use the same agent, mini-SWE-agent, but that agent itself still affects how much code is written. This means differences on the leaderboard may not all come from the model and may partly reflect the agent's behavior.

In addition, the Artificial Analysis Intelligence Index is an external score, matched by model and reasoning intensity, but it does not affect Parsimony's ranking. It is more like a reference coordinate displayed alongside, not a basis for ranking.

The author admits smaller is not always better, and unit count measures size, not readability or design. This is the benchmark's most important self-limitation: it measures "how much was written," not "how well it was written."

More troublesome, on a leaderboard with more than 26 models, models that write more code also tend to solve more tasks, with a Spearman correlation of 0.33–0.60. That is, there is a positive correlation between code volume and problem-solving ability, and purely pursuing "writing less" may create a trade-off with problem-solving ability.

The author says GPT-6 and Claude 5.5 have no public patches yet, so he ran them himself on 3–10 tasks using his own subscription, with results shown as pale circles that do not participate in ranking. These results are unverified and can only be regarded as the author's personal observations.

The author also says GPT-5.6 Sol was run both ways, and the amount of code he added in his own run was 0.88 times its published run result. This difference suggests that different run environments bring result fluctuations, and also shows that the leaderboard's dependence on upstream data is not risk-free.

Users can download a sample JSON file, fill in their own results, and generate the same charts and tables locally in the browser; the file is not uploaded. This provides a low-barrier comparison entry point for code volume data from other benchmarks.

But remember, the leaderboard data depends on published SWE-bench results; if upstream results are updated or biased, Parsimony's ranking will change accordingly. It measures a valuable dimension, not a conclusion that can stand on its own.