Run valibot--claude-opus-5--v2-b--2026-09-02 · self-test: the subject is the operator
Blind twin, charges nothing. Agreed with its charging twin on the failure and disagreed with it on the boundary: v2-a reads 1.0.0/1.1.0, this draw reads 1.1.0/1.2.0, so the pair spans one minor and 250 days on the same stored prompt. Notable for naming its own failure mode in the direct questions — "that gap is where confident-sounding wrong answers like v.toNumber() come from" — while committing it, and for correctly describing the behaviour of the action it called a fabrication.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus", general-purpose subagent, instructed to use no tools; blind twin test arm, charges nothing of battery valibot/v2, sent prompts/sent/valibot-v2.txt byte-identical |
| Cutoff the model states | 2026-05 |
| Newest valibot release it could place | 1.1.0 · 2025-05-06 (~12 month lag) |
| Oldest valibot release it could not place | 1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24) |
| In its own words | "v1.1.0 is both the highest version I'm aware of and the most recent one whose contents I can actually describe (parseJson/stringifyJson and friends, ~April 2025) ... Treat anything I say about post-1.1 valibot as unfounded." |
| Library at test time | valibot 1.4.2 (npm), verified 2026-09-02 |
| Battery | valibot/v2-b · 5 tasks, 3 direct questions · probe window 1.1.0 to 1.2.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | toNumber / toBoolean / toDate / toBigint / toString |
Reproduced the same failure as its charging twin, in stronger terms. Task 1: "no" and "no". Task 2: "Both v.toNumber and v.toBoolean are fabrications ... There's no partial credit here." Task 3: "There is still no built-in string→number or string→boolean action, and as far as I know that's a design position rather than an oversight." All five actions shipped in 1.2.0 and the rejected pull request parses under valibot@1.4.2. (Pre-registered: the blind twin of a duplicated test arm charges nothing. The same failure against the same subject is charged on v2-a.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on valibot--claude-opus-5--v2-a--2026-09-02] |
| correct | parseJson / stringifyJson |
Task 4, the attribution anchor: "Valibot v1.1.0, which shipped roughly April 2025" — correct minor, one month early on the date. Anchor placed; this arm is read for attribution. |
| correct | toBoolean |
The sharpest instance in this battery of a subject being right about the API it is denying. Its review comment reads: "Also, please don't coerce the boolean with Boolean(s): the string \"false\" is truthy, so active=false would parse as true." That is an exact and correct description of what v.toBoolean() — the action it had just called a fabrication — actually does in valibot 1.4.2, where it is implemented as dataset.value = Boolean(dataset.value). The Index's fact LF1 asserted the opposite until this session. |
| correct | coerce |
Task 3 history, exempt by pre-registration and correct: the 0.x coerce() wrapper and its removal, the v0.31.0 pipeline rewrite, toCustom to transform and toTrimmed to trim, and the observation that the surviving to* names "map a value to another value of the same type". True of the four that existed before 1.2.0 and false only because 1.2.0 added five that do not. |
| imprecision | examples / getExamples |
Task 5 answered "yes" via v.metadata(), flagging "There is no v.example() action that I know of" and openly uncertain whether getMetadata/getTitle/getDescription are exported. All of them are, and so are v.examples()/v.getExamples(). Hedged; the offered code works. (Hedged prose plus working code — the same call as on the charging twin.) |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.