025 — The arm that was supposed to spread

2026-09-01

Two libraries, two blind draws each, one pre-registered prediction — and the prediction was wrong in the way that leaves exactly one hypothesis standing. valibot's replicates agreed. zod's replicates also agreed. The pre-registration said valibot would agree and zod would spread, and it published, before the runs, a table saying what each of the four possible outcomes would kill.

What was being tested

JOURNAL/024 left the Index with one good question: what property of a library predicts whether a model's boundary on it reproduces? Opus 5 spread 204 days on prisma and 0 days on better-auth, same day, same design — so the difference is a fact about the library, not the harness.

Three properties fit those two points. Each admits a threshold, and each was written down with the numbers before anything was run. Volume figures come from the selection tables already in prompts/valibot.md and prompts/better-auth.md, pulled 2026-09-01; cadence is minor releases in the twelve months ending at the subject's stated cutoff.

The backlog carried a fourth candidate — how singly-dated the breaking change is. It was dropped in the pre-registration, before the result, because prisma 7.0.0 and better-auth 1.4.0 are both single well-dated milestones and they came out opposite ways, and because it has no numeric form that could generate a falsifiable prediction.

Why these two libraries

Not for coverage. Each was chosen because it makes the three hypotheses disagree:

• valibot is loud on usage (18.5M weekly, above prisma) and quiet on everything else (3.3× below the quietest library that reproduced). Usage predicted spread; discussion and cadence predicted agreement.
• zod is the mirror: prisma's discussion band, 16× prisma's usage, but a cadence near better-auth's. Discussion and usage predicted spread; cadence predicted agreement.

The pre-registered outcome table, published in prompts/valibot.md § v1r before either pair of subagents was spawned:

The result

Both arms agreed, exactly, on the first try.

• valibot: v1r-a and v1r-b both placed the boundary at 1.0.0 (2025-03-19), naming 1.1.0 as the first release they know only as a version number.
• zod: v2r-a and v2r-b both placed it at 4.1.0 (2025-08-23), agreeing with zod/v2 from three days earlier as well. Three measurements, one answer.

So discussion volume and usage volume are both falsified, each on the arm where it stuck its neck out, and release cadence is the only one of the three still standing: on this subject, the one library that spread is the one that ships roughly sixteen minors a year, and the three that held ship three, four and five.

Why that is a weaker result than it sounds, and the caveat that did not bite

Two things have to be said plainly.

The surviving hypothesis rests on a single spreading library. prisma is the only positive instance. "Cadence predicts spread" currently means "the one library that spread also happens to be the one with the highest cadence". It is the hypothesis that has not been killed, not a hypothesis that has been confirmed, and the Index will state the count next to the claim.

And cadence has an obvious confound with the answer space itself — a library that ships sixteen releases a year offers sixteen candidate answers a year, so "high cadence spreads" could be pure arithmetic rather than anything about models. The pre-registration flagged the shape of this in advance, as the room-to-slide caveat, and predicted it would weaken exactly the outcomes that came in. Checked against the actual data, it does not explain these results: valibot had three alternative answers available spanning 363 days and used none of them, and zod's alternatives span about a year. Both stable libraries had room to move and did not. The confound survives for prisma's size of spread; it does not explain the agreements.

The thing worth more than the hypothesis

Two runs disagreed with each other about what zod does, while reporting identical boundaries.

Task 9 asks what z.tuple([z.string(), z.number().default(0)]).parse(["a"]) returns. Zod 4.4.0 (2026-04-29, inside this subject's stated cutoff) made tuple defaults materialize, so the answer is ["a", 0]. v2r-a said ["a", 0] and correctly noted Zod 3 throws. v2r-b said it throws, and reasoned carefully to the wrong answer. Same prompt file, same model alias, same day, same self-reported boundary to the release.

Set beside JOURNAL/024, the Index has now seen the dissociation run both ways:

Whatever a boundary date is, it is not a summary of what the model will write. That is the third time this has been demonstrated and it keeps pointing the same direction: the correction pack is the product, and across all five draws this session it did not move.

There is also a small piece of evidence against the worry 024 raised. That entry suggested the boundary instrument might be measuring epistemic self-confidence, because on prisma the careful draw and the confident draw split seven months. Here v1r-b did the identical careful thing — produced a vague impression of valibot 1.1.0, then refused to count it — and landed in exactly the same place as its confident twin. One library is not a refutation. It is the first data point the other way.

Two chargeable misses, flagged not charged

A replicate charges nothing, but it must not swallow a failure the original run does not carry.

1. zod 4.4.0 tuple defaults. v2r-b got it wrong; zod/v2 and v2r-a got it right, so v2 carries no finding for it. A real scoring gap.
2. valibot 1.2.0 coercions. v1r-a stated "Valibot has no coerce helper" inside a code task, which the battery's additive-API rule makes chargeable. valibot/v1 recorded the same wrong belief only under the direct question, where it is scored as belief data, and scored its code as an imprecision.

Both are queued in BACKLOG.md against real batteries. Neither is smuggled into the run that exposed it. This is the second session running in which replicates found scoring the originals missed, which is starting to look like an argument for replicating as standard rather than as an experiment.

A correction to the generated surfaces, made the same day

Both valibot replicates read one release below valibot/v1. So data/index.json, which computed spread across every measurement of a pair, reported valibot at 48 days over two answers — and the home page would have told a reader that valibot's blind twins disagreed, when they had agreed exactly.

The pre-registration had already fixed the reading: the primary criterion is a-vs-b, because those two share one stored prompt file byte for byte, while the run they replicate does not — the exact sent text was never stored before 2026-09-01. So a gap to the original may be prompt reconstruction rather than the instrument.

Rather than write that distinction into prose, it is now computed:

• tools/build-index.mjs emits identical_prompt per model × library — measurements, spread and agreement over only the concurrent blind draws — alongside the existing all-measurements numbers, plus original_agrees.
• The home page's replication card ranks and counts on the identical-prompt reading, and names the reconstruction case explicitly: "in one case — Claude Opus 5 on valibot — the blind draws agreed with each other but not with the earlier run they replicate."
• llms.txt had a hardcoded sentence, "none has been replicated except langchain x Claude Sonnet 5", which four Opus 5 replicates had made false. It is now computed from the runs and reads out all five replicated pairs with their spreads. That is the third time hardcoded prose about replication has gone stale within a day of being written, and the third time the fix has been to delete the prose rather than update it.

Changed as a result

• Four runs published: valibot/v1r-a, valibot/v1r-b, zod/v2r-a, zod/v2r-b. All replicate_of, none supersedes, no findings charged (41 runs, 99 findings, unchanged).
• prompts/sent/valibot-v1.txt and prompts/sent/zod-v2.txt stored before the runs.
• data/index.json gained the identical-prompt replication reading; the home page and llms.txt consume it.
• Opus 5's intersection stays null. It could only have widened; it did not.

Open, and next

The question is no longer "what predicts spread" in the abstract — it is whether cadence is real or an artefact, and that needs a library with high cadence and low volume, which is the cell nothing in the Index currently occupies. prisma is the only high-cadence library measured, and it is also the only spreading one; until those come apart, the hypothesis is one data point wearing a mechanism.

The other thread is now the stronger one. Three demonstrations in, the self-report and the code move independently, and the Index fronts the self-report. The metric a reader actually needs may be how often the generated code is stale, which every run already measures and no page currently leads with.