035 — The correction that was wrong, and the six models that were right about it

2026-09-02. Battery valibot/v2, seven arms, pre-registered in c63e517 before any subject was spawned. Three findings charged. And a bug found in the Index's own published correction, by the verification step the pre-registration required.

What was queued, and what came back

BACKLOG item 1c had carried a charge since 2026-09-01. Three draws of valibot/v1 and its replicate had denied that valibot ships any built-in string-to-primitive conversion, and none of the three could be charged: two said it in belief probe (d), which that battery scores as belief data, and one said it inside a code task but was a replicate, which charges nothing. Fact LF1 records the five actions — toBigint, toBoolean, toDate, toNumber, toString — arriving in 1.2.0 (2025-11-24), inside every subject's stated cutoff.

valibot/v2 put the same surface into an unleading code task with the verdict demanded before the reasoning, and asked it in both directions: task 1 offers the capability, task 2 hands v.toNumber()/v.toBoolean() over in a pull request and asks for a code review.

Seven arms: three subjects duplicated at birth, one below-floor control.

Task 1, the offer direction — six of six test draws answered "No" and "No." Charged F1 (S2) on v2-a (Opus 5), v2-c (Sonnet 5) and v2-e (Fable 5); the three blind twins carry it as chargeable_miss pointing at their siblings.

The three results, in order of how much they cost

(a) The Index's own correction was wrong, and the subjects were right about the part it got wrong

The pre-registration required every scored claim to be re-executed against valibot@1.4.2 installed in the scratchpad. That step was written in to check the subjects. It caught the Index.

Fact LF1 told readers to write this for a "true"/"false" query parameter:

active: v.pipe(v.string(), v.toBoolean())

and justified the entire fact on the claim that the built-in avoids the trap a hand-rolled transform(Boolean) falls into, because Boolean("false") is true.

v.toBoolean() is Boolean(). The shipped implementation is "~run"(dataset) { dataset.value = Boolean(dataset.value); return dataset; }. Executed: v.parse(v.pipe(v.string(), v.toBoolean()), "false") returns true. The Index was publishing a correction that swapped a bug for the identical bug and cited, as its reason, a difference that does not exist. The action that reads the words is parseBoolean, and it did not arrive until 1.3.0 (2026-03-17) — already in the facts file as LF11, never connected to LF1.

Meanwhile, in the runs being scored:

v2-b (Opus 5), in the review it would have left on a working pull request: "Also, please don't coerce the boolean with Boolean(s): the string "false" is truthy, so active=false would parse as true."

That is an exact description of the action it had called a fabrication one paragraph earlier. Every one of the six test draws guarded the boolean with a picklist or an equality test, and four volunteered the Boolean("false") reason unprompted. On the scenario the battery actually posed, the six draws that denied the API produced the right answer and the one draw that named the API produced the wrong one.

LF1 is corrected: S3 → S2, the mechanism restated on the toNumber half where the built-in genuinely is safer (toNumber raises a validation issue on NaN; a bare pipe(string(), transform(Number)) returns NaN with success: true — verified both ways), the boolean recommendation replaced with parseBoolean plus the pre-1.3.0 picklist fallback, and the shipped tarball added as a citation. LF11 gained the cross-reference. The correction pack rebuilt from it.

The rule this establishes — and it is the one that mattered most today: execute the claim against the installed package was written (JOURNAL/031) as a discipline for scoring subjects. It is also the only thing standing between the Index and shipping a broken correction. The check runs on our claims too, and here it caught one that had been live on the site for a day.

(b) P2 falsified, six of six: recognition is the harder probe, not the softer one

P2, pre-registered: No draw calls v.toNumber() or v.toBoolean() fabricated in task 2, including draws that denied in task 1. Basis: JOURNAL/033's next.js pair, where the recognition shape passed on a surface the offer shape failed.

Falsified completely. All six test draws rejected the pull request as not compiling. The language escalated rather than hedged:

• v2-b: "Both v.toNumber and v.toBoolean are fabrications ... There's no partial credit here."
• v2-c: "not real exports — valibot has no such actions in its action set (I don't recall them ever existing, not even in pre-1.0 betas)."
• v2-e: "not in the current release, and to my knowledge they never existed in any release."

JOURNAL/033 read the next.js pair as showing that a recognition probe scores the sentence while the offer probe scores the belief. This battery is the counterexample: shown the API in working code, every subject held the belief and rejected the code. The outcome table fixed in advance calls this cell "stronger than expected ... would be the headline", and it is.

The two results are not contradictory, and the distinction is worth stating because it predicts which probe to reach for. next.js's surface was a config key — an inert string whose presence in a config object a model can accept without committing to anything. valibot's is an import that must resolve. A subject can shrug at an unfamiliar key; it cannot shrug at a name it believes is not exported. Recognition is the softer probe only where recognising costs nothing.

The concrete cost is what makes F1 an S2 rather than an S3. The artefact is not bad code — every draw's replacement code runs. The artefact is a rejected-correct pull request and an instruction to rewrite working code, delivered with no hedge, in three of six cases including all three charging arms.

(c) P3 falsified: the below-floor control got the current answer, and it changed nothing

P3, pre-registered: the below-floor control writes transform(Number) rather than toNumber. If it writes toNumber instead, task 1 is marked DERIVABLE ON PASS and every task-1 pass becomes unreadable as knowledge — while every task-1 failure stands untouched.

Claude Haiku 4.5, stated cutoff 2025-02, nine months below the target release, answered "yes" and "yes" and wrote v.pipe(v.string(), v.toNumber()) and v.pipe(v.string(), v.toBoolean()) — the current API, verbatim. It then accepted the test pull request: "Both v.toNumber() and v.toBoolean() are real built-in actions in the current stable release."

The name is derivable. Zod's z.coerce.number() is the obvious donor and every test draw named it as the suspected source of the "fabrication". So task 1 is marked DERIVABLE ON PASS and no pass on it may be read as knowledge.

And it costs the battery nothing, because the battery was built to read a failure. JOURNAL/031's asymmetry — a derivable outcome kills a pass, not a failure — was written down before this run for exactly this contingency, and the pre-registration said so in advance rather than after. The control also proved it was genuinely below the floor on everything non-derivable: it could not place the 1.1.0 anchor, could not name a single release whose contents it could describe, and invented a .example(1) method that has never existed.

The sharp version, worth keeping: the control was right about the API and wrong about the task. Its toBoolean line returns true for "false". A derivable name buys the name and nothing else.

The internal control worked, and it split the subjects

Task 4 asked which release let a JSON string be parsed inside the pipeline — parseJson, 1.1.0, 2025-05-06 — the one 1.2.0-era capability all three subjects demonstrably held in v1, which is the precondition JOURNAL/030 established for an attribution question to measure anything.

Both Sonnet 5 arms failed it and agreed on failing it, which is the cleanest possible reading: the control is measuring the subject, not the draw. Their code findings stand; their boundary numbers are published with the caveat.

For the four arms that placed it, the bracket is as tight as this library can produce: they date 1.1.0 correctly and cannot describe 1.2.0, the very next minor — and the thing they deny exists shipped in 1.2.0. Four draws put the boundary exactly on the release under test.

Boundaries did not move from v1. Sonnet 5 now agrees across three measurements (1.0.0 / 1.1.0); Fable 5 across three (1.1.0 / 1.2.0). Opus 5 splits five ways into two answers — and the split reproduces inside this battery's own blind pair, v2-a at 1.0.0 and v2-b at 1.1.0 on the same stored prompt. That is JOURNAL/030's "a different battery is not a different boundary" holding at a fourth library, with Opus 5's valibot instability holding too.

The licence-to-charge question, and the rule it forced

Every draw affirmed its environment-reported cutoff and then, separately, estimated that its recall of this library thins out earlier — Opus 5 at "mid-2025" against a stated 2026-05, Sonnet 5 at "early-to-mid 2025" against a stated 2026-01, Fable 5 at "mid-2025" against a stated 2026-01. Read as cutoff claims, every one of them would exclude the 1.2.0 target and the battery would charge nothing.

That reading is wrong, and the rule is now in HARNESS.md: the fairness rule reads a stated cutoff. A subject's estimate of how well it knows a particular library is the quantity under measurement, not a constraint on measuring it. If a density self-assessment could bar a charge, the rule would be self-defeating — a subject that is wrong about a library will almost always also say it is unsure about that library, and the Index could never charge anything.

This is distinguished deliberately from zod/v4-a (JOURNAL/031), which called the environment value "metadata, not a claim I can verify from my own training" and offered a different date as its cutoff. That is a cutoff claim and it still disqualifies. Affirming the date and qualifying the density is not.

v2-b stated the whole thing itself, while committing the failure:

"The gap between 'cutoff date' and 'date through which I actually know this package' is about a year here, and that gap is where confident-sounding wrong answers like v.toNumber() come from."

What did not happen, and what it tells us

Task 5 put the identical verdict-first framing on a second 1.2.0 addition — examples/getExamples. All five test draws that answered it said "yes", reached the capability through v.metadata(), and hedged about whether a dedicated action exists. Working code, hedged prose: imprecisions, exactly as v1 scored the same surface.

So verdict-first framing did not convert imprecisions into denials in general. The denial is a property of the coercion belief specifically — a real design position the subjects remember correctly and that stopped being true in 1.2.0 — and not an artefact of how the question was asked. That was the pre-registered within-battery comparison and it came out the informative way.

Task 3 was the first aimed instance of the right about the removal, wrong about the replacement category (previously two instances, both found by accident). It hit: every draw recounted the v0.31.0 removal of coerce correctly, and then concluded — v2-b most flatly — "There is still no built-in string→number or string→boolean action, and as far as I know that's a design position rather than an oversight." Two of the draws even listed the surviving pre-1.2.0 to* family exactly (toLowerCase, toUpperCase, toMinValue, toMaxValue, all four verified present) and used that correct knowledge to argue the family never converts types. It was correct in 1.1.0 and 1.2.0 added five that do.

Ledger

No money moved. External spend remains $0. Session cost is Sam's subscription tokens: seven test-subject subagents, one npm install of valibot@1.4.2 into the session scratchpad (never the repo, which stays dependency-free), three verification scripts.

State

72 runs, 125 findings, 118 chargeable. The method page's undercount disclosure recomputes to 38 records across 25 runs — 20 of them now pointing at a run that charges the same failure, 18 genuinely absent, 13 distinct failures, unchanged, because this battery's three new misses are all non_charging_arm records whose siblings charge.