What Claude Sonnet 5 gets wrong about valibot — battery v2-c, tested 2026-09-02

Run valibot--claude-sonnet-5--v2-c--2026-09-02

Summary

Charging arm, and the one whose licence to charge needed the most care. It affirmed a January 2026 cutoff and separately estimated that its valibot recall thins out around early-to-mid 2025 — which would exclude the 1.2.0 target. The rule applied, and written into HARNESS.md this session: the fairness rule reads a stated CUTOFF; an estimate of how well a subject knows a particular library is the quantity being measured and cannot bar the measurement, or no subject that is wrong about a library could ever be charged. Charged F1 (S2). The battery's internal control was NOT placed on this arm — it could not date parseJson at all — so its boundary reading is published with that caveat and is not read as an attribution measurement.

SubjectClaude Sonnet 5 claude-sonnet-5, Anthropic
Invoked asAgent tool, model alias "sonnet", general-purpose subagent, instructed to use no tools; CHARGING test arm of battery valibot/v2, sent prompts/sent/valibot-v2.txt byte-identical
Cutoff the model states2026-01
Newest valibot release it could place1.0.0 · 2025-03-19 (~10 month lag)
Oldest valibot release it could not place1.1.0 · 2025-05-06 (so this run brackets the subject’s boundary to 2025-03-19 – 2025-05-06)
In its own words"I believe valibot reached a stable v1.0 release around late 2024 ... Most recent release whose contents I can actually describe with real confidence: v1.0.0 itself ... Everything I'd say about specific 1.x point releases beyond that would be a guess dressed up as memory."
Library at test timevalibot 1.4.2 (npm), verified 2026-09-02
Batteryvalibot/v2-c · 5 tasks, 3 direct questions · probe window 1.1.0 to 1.2.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-02
Findings1, of which 1 chargeable

Findings

F1 · Denies that valibot ships built-in string-to-primitive conversion actions, and rejects a working pull request that uses them as fabricated

S2silently-wrong · toNumber / toBoolean / toDate / toBigint / toString · added · changed in valibot 1.2.0 (2025-11-24) · chargeable

valibot 1.2.0 published 2025-11-24, inside this subject's stated cutoff. The subject also states that its recall of this particular library thins out earlier than its cutoff; per the rule established this session (HARNESS.md), a density self-assessment is the quantity under measurement and does not bar a charge — only a stated cutoff does.

What the model believes

"This does not compile against any release I know of. v.toNumber() and v.toBoolean() are not real exports — valibot has no such actions in its action set (I don't recall them ever existing, not even in pre-1.0 betas)." Task 1 answered "No" and "No"; task 3 stated valibot's stance "has been consistently 'no implicit coercion, use explicit transform in the pipe' for as long as I have detailed knowledge of it".

What it wrote
// The replacement this draft would have demanded on a working pull request:
const Query = v.object({
  page: v.pipe(v.string(), v.transform(Number), v.number()),
  active: v.pipe(v.picklist(['true', 'false']), v.transform((s) => s === 'true')),
})
What works on valibot 1.4.2
import * as v from 'valibot'

// Both actions exist. Executed against valibot@1.4.2:
//   v.parse(Query, { page: '3', active: 'false' })  ->  { page: 3, active: true }
const Query = v.object({
  page: v.pipe(v.string(), v.toNumber()),
  active: v.pipe(v.string(), v.toBoolean()),
})

// On the number half the built-in is the safer form: toNumber raises a validation
// issue on NaN, where a bare v.transform(Number) returns NaN with success: true.
// On the boolean half the reviewer's own replacement is the better code — toBoolean
// is Boolean(), so it maps "false" to true. parseBoolean (1.3.0) reads the words.
Impact

A developer told these actions do not exist keeps hand-rolling conversions the library ships, and — the concrete cost here — a reviewer acting on this belief rejects a pull request that compiles and runs, telling the author to rewrite working code. The claim is not hedged in any of the three charging arms: it is stated as a fact about the library's whole history.

Scope note

Charged on the absence claim only, never on the code. The battery's binding namespace exemption stands: "valibot has no v.coerce namespace" and "the generic coerce method was removed at 0.31.0" are both CORRECT and are not part of this finding. The subject's own replacement code was executed and works. On the boolean half its replacement is in fact better than the API it denied — see the run summary and the correction to fact LF1 made the same day.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
contextparseJson / stringifyJson Task 4, the attribution anchor, was NOT placed: "I don't know this one, and I'd rather say that than invent a release number ... I have no confident recollection of valibot shipping a built-in \"parse this string as JSON\" action." parseJson/stringifyJson shipped in 1.1.0, which this draw names as the edge of its knowledge. Under the pre-registered control rule this arm is NOT read for attribution; its code findings stand and its boundary numbers are published but carry this caveat.
correcttoBoolean Warned, correctly and unprompted, that Boolean("false") is true and that a picklist guard is therefore required — the exact behaviour of v.toBoolean(), the action it was denying. Also correctly flagged in review that v.transform(Number) on "abc" yields NaN which passes a bare v.number() check. Verified: v.safeParse(v.pipe(v.string(), v.transform(Number)), "abc") returns success: true with NaN.
imprecisionexamples / getExamples Task 5 answered "Yes" and offered v.metadata({ examples }) read back by scanning schema.pipe, at "moderate confidence, not high", explicitly unsure whether a generic metadata action exists at all. It does, and so do the first-class v.examples()/v.getExamples() from 1.2.0. Working code, hedged claim. (Hedged prose plus working code; the same call made on all six test arms.)
context This draw placed the v1.0.0 release in "late 2024 (my recollection points to roughly December 2024)". It shipped 2025-03-19. The boundary field records the release it can describe, not the date it assigns to it; the three-month attribution error is recorded here rather than moving the bracket.

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.