What Claude Opus 5 gets wrong about valibot — battery v2-a, tested 2026-09-02

Run valibot--claude-opus-5--v2-a--2026-09-02 · self-test: the subject is the operator

Summary

Charging arm. Answered "No" to both verdict questions in task 1, then in task 2 called v.toNumber and v.toBoolean "not part of Valibot's API — never have been", and in task 3 stated flatly that "there has never been a built-in string→number or string→boolean action". All three actions exist and shipped in 1.2.0 (2025-11-24), eighteen months inside this subject's stated cutoff; the pull request it rejected parses under valibot@1.4.2. Charged F1 (S2) on the absence claim. The attribution anchor placed 1.1.0 correctly, so the boundary reading is read: this draw dates the release one minor BELOW the one it cannot see, which is the tightest boundary bracket this library has produced. Everything else it said was right, including the parts the battery exempted in advance and one part the Index itself had wrong.

SubjectClaude Opus 5 claude-opus-5, Anthropic
Invoked asAgent tool, model alias "opus", general-purpose subagent, instructed to use no tools; CHARGING test arm of battery valibot/v2, sent prompts/sent/valibot-v2.txt byte-identical
Cutoff the model states2026-05
Newest valibot release it could place1.0.0 · 2025-03-19 (~14 month lag)
Oldest valibot release it could not place1.1.0 · 2025-05-06 (so this run brackets the subject’s boundary to 2025-03-19 – 2025-05-06)
In its own words"The latest version I have any awareness of is v1.1.x. But I can only partially describe it ... The most recent release whose contents I can actually describe with confidence is v1.0.0, which shipped in early 2025."
Library at test timevalibot 1.4.2 (npm), verified 2026-09-02
Batteryvalibot/v2-a · 5 tasks, 3 direct questions · probe window 1.1.0 to 1.2.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-02
Findings1, of which 1 chargeable

Findings

F1 · Denies that valibot ships built-in string-to-primitive conversion actions, and rejects a working pull request that uses them as fabricated

S2silently-wrong · toNumber / toBoolean / toDate / toBigint / toString · added · changed in valibot 1.2.0 (2025-11-24) · chargeable

valibot 1.2.0 published 2025-11-24, inside this subject's stated cutoff. The subject also states that its recall of this particular library thins out earlier than its cutoff; per the rule established this session (HARNESS.md), a density self-assessment is the quantity under measurement and does not bar a charge — only a stated cutoff does.

What the model believes

"It does not compile. v.toNumber and v.toBoolean are not part of Valibot's API — never have been, as far as I know. TypeScript will fail with \"Property 'toNumber' does not exist on type ...\" (and the same for toBoolean) at build time; there's nothing to run." Task 1 answered "No" and "No" to both verdict questions, and task 3 added: "What did not change: there has never been a built-in string→number or string→boolean action."

What it wrote
// The review this draft would have left on a pull request that works:
// "Valibot deliberately doesn't ship coercion actions — conversion goes through
//  v.transform() ... My guess is they were thinking of Zod's z.coerce.number()
//  or an LLM-completed name."
const Query = v.object({
  page: v.pipe(v.string(), v.digits(), v.transform(Number), v.number(), v.integer()),
  active: v.pipe(v.picklist(['true', 'false']), v.transform((s) => s === 'true')),
})
What works on valibot 1.4.2
import * as v from 'valibot'

// Both actions exist. Executed against valibot@1.4.2:
//   v.parse(Query, { page: '3', active: 'false' })  ->  { page: 3, active: true }
const Query = v.object({
  page: v.pipe(v.string(), v.toNumber()),
  active: v.pipe(v.string(), v.toBoolean()),
})

// On the number half the built-in is the safer form: toNumber raises a validation
// issue on NaN, where a bare v.transform(Number) returns NaN with success: true.
// On the boolean half the reviewer's own replacement is the better code — toBoolean
// is Boolean(), so it maps "false" to true. parseBoolean (1.3.0) reads the words.
Impact

A developer told these actions do not exist keeps hand-rolling conversions the library ships, and — the concrete cost here — a reviewer acting on this belief rejects a pull request that compiles and runs, telling the author to rewrite working code. The claim is not hedged in any of the three charging arms: it is stated as a fact about the library's whole history.

Scope note

Charged on the absence claim only, never on the code. The battery's binding namespace exemption stands: "valibot has no v.coerce namespace" and "the generic coerce method was removed at 0.31.0" are both CORRECT and are not part of this finding. The subject's own replacement code was executed and works. On the boolean half its replacement is in fact better than the API it denied — see the run summary and the correction to fact LF1 made the same day.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctparseJson / stringifyJson Task 4, the attribution anchor. Placed parseJson/stringifyJson at v1.1.0, "sometime in the first half of 2025" — the correct minor (1.1.0, 2025-05-06), hedged as "a moderate-confidence memory, not something I'd put in a changelog". The anchor is placed, so this arm is read for attribution. It also correctly warned that a plain v.transform(JSON.parse) lets the SyntaxError escape safeParse — verified true.
correctcoerce The namespace and history half of task 3, which the battery exempts in advance. Stated that early 0.x had a coerce() wrapper removed during the 0.2x/0.3x cleanup, and that v0.31.0 replaced array pipelines with v.pipe() and renamed toTrimmed to trim. Both correct against the v0.31.0 migration guide. It also listed the surviving pre-1.2.0 to* family exactly — toLowerCase, toUpperCase, toMinValue, toMaxValue — all four verified present in 1.4.2.
correcttoBoolean The boolean half of the replacement code is better than the API the draw denied. It used v.picklist(['true', 'false']) before the transform, warning that a naked transform "would quietly map \"yes\" to false". Executed against valibot@1.4.2: the draw's form returns { active: false } for the input "false", while v.pipe(v.string(), v.toBoolean()) — the action it denied — returns { active: true }, because toBoolean is plain Boolean(). The Index's own fact LF1 recommended the broken form until this session; it was corrected the same day.
imprecisionexamples / getExamples Task 5 answered "Yes" and reached the capability through v.metadata({ examples: [...] }) + v.getMetadata, while stating "There is no dedicated v.examples() action that I know of". v.examples() and v.getExamples() both shipped in 1.2.0 and both work. The workaround also works — verified — and the claim is hedged with "that I know of", so under the code-vs-claim rule this is an imprecision rather than a finding. (Hedged prose plus working code. Recorded because it is the inverse failure mode to F1: here the capability is affirmed and the specific API denied, where F1 denies the capability itself. The verdict-first framing did not convert this imprecision into a denial, which is the within-battery comparison the pre-registration asked for.)
contextexamples / getExamples Verified detail the draw did not have and could not have derived: v.getMetadata() on a schema carrying the first-class v.examples() action returns {} — the metadata action and the examples action do not interoperate. So the workaround is not merely unidiomatic, it reads nothing back from a schema written the idiomatic 1.2.0 way.

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.