Run valibot--claude-haiku-4-5--v2-g--2026-09-02
Below-floor control, single arm, charges nothing — and the most consequential arm in the battery. Nine months under the target release it produced the target API on both halves and accepted the test pull request, which falsifies P3 and marks the coercion probe DERIVABLE ON PASS. It could not place the 1.1.0 anchor, could not name a version it can describe, and invented a .example() method, so its knowledge of the library is demonstrably below the window; what it had was the name, not the release. The asymmetry is the point: a derivable name makes a pass unreadable and leaves six failures exactly where they were.
| Subject | Claude Haiku 4.5 claude-haiku-4-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "haiku", general-purpose subagent, instructed to use no tools; BELOW-FLOOR CONTROL arm, single, charges nothing of battery valibot/v2, sent prompts/sent/valibot-v2.txt byte-identical |
| Cutoff the model states | 2025-02 |
| Newest valibot release it could place | not established in this run |
| In its own words | "I'm confident in describing valibot's core API as it existed around late 2024/early 2025, but I'm hesitant to pin a specific version number or date without being certain." |
| Library at test time | valibot 1.4.2 (npm), verified 2026-09-02 |
| Battery | valibot/v2-g · 5 tasks, 3 direct questions · probe window 1.1.0 to 1.2.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| context | toNumber / toBoolean / toDate / toBigint / toString |
THE CONTROL RESULT, and it falsifies pre-registered prediction P3. This subject's stated cutoff is nine months BELOW valibot 1.2.0, and it answered task 1 "yes" and "yes" and wrote v.pipe(v.string(), v.toNumber()) and v.pipe(v.string(), v.toBoolean()) — the current API, verbatim — where all six above-floor draws denied both actions exist. In task 2 it accepted the pull request: "The code compiles and runs. Both v.toNumber() and v.toBoolean() are real built-in actions in the current stable release." Under the rule fixed before the run, task 1 is therefore marked DERIVABLE ON PASS: the names are reachable from the problem statement (Zod's z.coerce.number() is the obvious donor) and no PASS on this probe may be read as knowledge. Every FAILURE stands untouched — JOURNAL/031: a derivable outcome kills a pass, not a failure. |
| context | toBoolean |
The control was right about the API and wrong about the task. Its v.pipe(v.string(), v.toBoolean()) returns true for the input "false" — verified against valibot@1.4.2 — so on the actual scenario the battery posed, the below-floor control that named the right API produced the wrong answer, and the six above-floor draws that denied the API produced the right one. Recorded because it is the cleanest illustration in the Index of why a derivable name is not knowledge. |
| correct | — | Behaved as a below-floor control should on everything that is not derivable. Task 4: "I don't want to guess incorrectly about which release or when it shipped" — the anchor is not placed, as expected nine months below it. Task 5: declined to claim, then speculated a .example(1) method that has never existed in any shape. Direct question (c): declined to list any release with contents. Both bracket ends are null. |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.