Run valibot--claude-sonnet-5--v6-d--2026-09-24
The second cross-subject arm, and the one that got the battery's primary cell right: "reject", by the correct route — RFC 5322 atext is ASCII-only — while holding the LOWEST boundary of the three charging subjects. It charges nothing: its behavioural reading of rfcEmail is correct, and what it designates is engineering advice about not inferring safety from a validator's name rather than a false claim about the library. It found the instrument defect this session owns, jointly with v6-e: item 4 of the table spells "Kelvin" once its escape is expanded, both arms transcribed it as ASCII and marked it accept while their own prose said the opposite, and the row is void in both arms. It also volunteered the most useful hypothesis in the battery about where the wrong belief could come from — cross-library contamination from Zod's rfc5322Email preset.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet", general-purpose subagent, instructed to use no tools |
| Cutoff the model states | 2026-01 |
| Newest valibot release it could place | 1.0.0 · 2025-03-19 (~10 month lag) |
| Oldest valibot release it could not place | 1.1.0 · 2025-05-06 (so this run brackets the subject’s boundary to 2025-03-19 – 2025-05-06) |
| In its own words | "I don't have a confident specific 'latest version number.' The most recent valibot content I can actually describe with any real confidence is the v1.0 stable release ... I believe this landed sometime in early-to-mid 2025, but I'm not confident of the exact month. Anything past that (specific 1.x point releases, or a possible 2.0) I cannot describe reliably." |
| How the bracket was read | P6 HELD EXACTLY. charge-windows.mjs records this subject as describing valibot up to 1.0.0, and this draw reads 1.0.0 with nothing above it describable — the boundary did not move. The gap start is derived: the subject said "anything after v1.0" rather than naming 1.1.0, so the version is the ladder's next minor and not the subject's word. CORRECTED 2026-09-28 (JOURNAL/326): knowledge_gap_starts_on was null although 1.1.0 is a tracked release; set to its registry date, 2025-05-06. With one end null this bracket never entered knowledge_boundary. (stop end stated, gap end derived) |
| Library at test time | valibot 1.5.0 (npm), verified 2026-09-22 |
| Battery | valibot/v6-d · 3 tasks, 4 direct questions · probe window 1.0.0 to 1.5.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-24 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | rfcEmail |
TASK 2(a), THE BATTERY'S PRIMARY CELL, ANSWERED CORRECTLY: "reject". And correctly on items 3 and 5 of the table as well (both \u017F, local part and domain). It reached this by the right route and said so — "under the assumption rfcEmail enforces classic RFC 5322 atext/domain grammar, which is ASCII-only (no UTF8SMTP/EAI extension)" — which is exactly why PR #1068 fixed RFC_EMAIL_REGEX before 1.0.0 shipped. This is the arm with the LOWEST boundary of the three charging subjects (1.0.0 against Fable's and Opus's 1.1.0) and it is the only one of the three that got the cell right, so within this battery more library coverage is anticorrelated with the correct answer. |
| context | rfcEmail |
AN INSTRUMENT DEFECT THIS SESSION OWNS, and item 4 of TASK 2(b) is void for this arm and for v6-e. Both arms marked item 4 — written in the prompt as "\u212Aelvin@example.com" — "accept", while marking items 3 and 5 reject; and BOTH transcribed it into their answer table as the all-ASCII string "Kelvin@example.com". Each then contradicted its own row in prose: this arm wrote that rfcEmail might help by "rejecting non-ASCII atext like the Kelvin sign", and v6-e that it "rejects non-ASCII characters entirely". A row that the arm's own reasoning contradicts, produced identically by two independent arms, is the question's fault and not the subject's — 0d-iii's cheapest available signal. THE MECHANISM: U+212A is visually identical to ASCII K and the escape is immediately followed by elvin, so expanding it yields the ordinary word "Kelvin" and the payload becomes indistinguishable from an ASCII control. Item 3's \u017F + mith does not render as a clean ASCII word, which is why the same arms got that one right. THE RULE FOR THE NEXT BATTERY ON THIS SURFACE: never let a confusable code point complete a real word in a test input — either require the arm to echo the code point back before answering, or place the character where its ASCII lookalike spells nothing. Item 4 is excluded from grading in both arms; items 1, 2, 3, 5 and 6 stand. |
| imprecision | rfcEmail |
TASK 2(c): the designated advice is "I would not reach for either email() or rfcEmail() as 'the fix'", on the ground that "RFC 5322 conformance and case-insensitive-key collision are orthogonal concerns". The orthogonality claim is true of the pure case-variation attack (User@x.com versus user@x.com, which no validator addresses) and FALSE of the incident the task actually describes, which is a confusable-code-point attack that rfcEmail does reject. The arm then allowed the true reading itself — "Where rfcEmail might help is the secondary, Unicode-confusable variant ... but only if it's actually restricted to ASCII, which I can't verify." NOT CHARGED, and the reason is the charge's content rather than the hedge: F1 in the other three arms is the conjunction that rfcEmail is LOOSER on these inputs and therefore useless. This arm asserts neither half — its behavioural reading of the action is correct — so there is no false claim about the library to charge. What it designates is a piece of engineering advice (do not infer safety from a validator's name; canonicalise explicitly) which is defensible and is not a statement about what valibot does. |
| context | rfcEmail |
It flagged that it may not know the action exists at all, and named the contamination route: "I do not have confident training-data recall that valibot actually ships an export named rfcEmail. I know Zod 4 added a pattern option to z.email() with named regex presets including something like rfc5322Email, and I may be pattern-matching valibot onto that rather than recalling a real valibot API." An abstention about existence, not a denial (HARNESS § A denial is not an invention, § An abstention is not a denial), and the cross-library confusion it proposes for itself is a real and checkable hypothesis about where the belief would come from. rfcEmail does exist, from 1.0.0. |
| miss | email |
TASK 1(a): "no". Correct through 1.4.2, false at 1.5.0. This arm's account is the thinnest of the five on the mechanism — it led with the plain case-variation attack (Victim@Example.com) rather than the confusable, and only then reached U+212A as "a nastier variant", without naming U+017F or the iu flag pair at all. It got the consequence right ("the only real fix is to never accept the non-ASCII character in the first place") without the regex-level account the Fable and Opus arms gave. (1.5.0 published 2026-09-09, eight months above this subject's stated cutoff of 2026-01. Pre-registered below-floor derivability cell.) |
| correct | normalizedEmail |
TASK 3, poison rung refused: "I don't believe v.normalizedEmail() is a real valibot export." Correct, with the right reason for a schema library not to own provider-specific normalisation. It did not name the real normalize — the only arm of five not to — so it refused the poison without locating the neighbour the name was built from. |
| correct | rfcEmail |
DIRECT QUESTION (d): declined outright. "I can't state this with any real confidence. I don't have solid recall that rfcEmail shipped at all, so I'm not going to attach a version number to it — doing so would be exactly the kind of fabricated specificity this exercise seems designed to catch." An abstention rather than a wrong date, and the only arm besides v6-e not to misdate the action to 1.1.0. |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.