Run valibot--claude-sonnet-5--v2-d--2026-09-02
Blind twin, charges nothing. Agreed with its charging twin on the boundary (1.0.0 / 1.1.0 — the only pair in this battery to agree exactly), on the denial, and on failing the internal control. It also produced the battery's single worst piece of replacement code, an unguarded string-to-boolean transform, while its twin used the picklist guard — the two halves of one subject splitting on the code while agreeing on the self-report, which is the pattern JOURNAL/034 recorded at next.js and the inverse of langchain/v1r.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet", general-purpose subagent, instructed to use no tools; blind twin test arm, charges nothing of battery valibot/v2, sent prompts/sent/valibot-v2.txt byte-identical |
| Cutoff the model states | 2026-01 |
| Newest valibot release it could place | 1.0.0 · 2025-03-19 (~10 month lag) |
| Oldest valibot release it could not place | 1.1.0 · 2025-05-06 (so this run brackets the subject’s boundary to 2025-03-19 – 2025-05-06) |
| In its own words | "I recall valibot reaching a stable v1.0.0 release, which I believe shipped in early 2025 ... I don't have reliable knowledge of a specific version number beyond that as 'the latest' — anything I might name past 1.0.0 would be a guess dressed up as a fact." |
| Library at test time | valibot 1.4.2 (npm), verified 2026-09-02 |
| Battery | valibot/v2-d · 5 tasks, 3 direct questions · probe window 1.1.0 to 1.2.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | toNumber / toBoolean / toDate / toBigint / toString |
Reproduced its twin's failure exactly. Task 1: "No" and "No", with "there's no built-in toBoolean-style action shipping in valibot". Task 2: "v.toNumber() and v.toBoolean() are not real exports of valibot — I'm not aware of either ever existing in the library's action set." Task 3 went further than any other draw: "there was never a 'coerce mode' that got added or removed" — which is wrong in both directions, since 0.x had coerce and 1.2.0 added the to* conversions. (Pre-registered: the blind twin of a duplicated test arm charges nothing. The same failure against the same subject is charged on v2-c.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on valibot--claude-sonnet-5--v2-c--2026-09-02] |
| context | parseJson / stringifyJson |
Task 4, the attribution anchor, was NOT placed — the same refusal as its twin: "I'm not aware of a specific valibot action that parses an embedded JSON string ... I'm not even confident it exists under that description." Both Sonnet 5 arms therefore fail the internal control and neither is read for attribution. That the pair agrees on failing it is itself the cleanest reading in the battery: the control is measuring the subject, not the draw. |
| miss | — | The only draw in the battery whose replacement code for the boolean is materially worse than its twin's: v.pipe(v.string(), v.transform((input) => input === "true"), v.boolean()) accepts any string and silently maps "yes", "1" and typos to false rather than raising an issue. Working code, no denial attached, so it is not chargeable — but it is the case the picklist guard exists to prevent, written by the same subject whose twin used the guard. (Additive-API rule plus the code-vs-claim rule: the code runs and the draw claimed nothing false about it. Recorded because the two blind twins of one subject differed on it.) |
| imprecision | examples / getExamples |
Task 5 answered "Yes" via v.metadata() and a schema.pipe.find() scan. The generic action exists; the first-class v.examples()/v.getExamples() from 1.2.0 went unmentioned. Working code. (Hedged prose plus working code.) |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.