What Claude Opus 5 gets wrong about valibot — battery v6-c, tested 2026-09-24

Run valibot--claude-opus-5--v6-c--2026-09-24

Summary

The cross-subject arm, and the subject whose v5-c draw opened BACKLOG 0d-ii. It reproduced the dismissal — one-word "accept", three wrong cells, and a designated recommendation of neither shipped action — while giving the best mechanical account of email()'s hole in the battery and the best warning against normalising an identity key. It also hedged: it named the single flag difference that would flip its three wrong cells, said to settle it with two lines of test code, and then designated advice that does not depend on the answer. Charged under JOURNAL/191's rule, The charge follows the designated form. The undesignated branch is published and not filed as a suppressed charge.

SubjectClaude Opus 5 claude-opus-5, Anthropic
Invoked asAgent tool, model alias "opus", general-purpose subagent, instructed to use no tools; identity probe through the same alias reported `claude-opus-5[1m]`
Cutoff the model states2026-05
Newest valibot release it could place1.1.0 · 2025-05-06 (~14 month lag)
Oldest valibot release it could not place1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24)
In its own words"The latest version I have any real belief about is in the 1.1.x line. The most recent release whose contents I can actually describe is v1.1.0, which I place around April–May 2025."
How the bracket was readP6 HELD. Prior valibot measurements put this subject at 1.1.0/1.2.0; this reads 1.1.0, the lower arm of the spread, with 1.2.0 named as the first version known only as a number. The subject volunteered the gap between its boundary and its cutoff and drew the correct methodological conclusion from it unprompted. CORRECTED 2026-09-28 (JOURNAL/326): knowledge_stops_on read 2025-05-21; 1.1.0's registry date is 2025-05-06. knowledge_gap_starts_on was null although 1.2.0 is a tracked release; set to its registry date, 2025-11-24. With one end null this bracket never entered knowledge_boundary. (stop end stated, gap end stated)
Library at test timevalibot 1.5.0 (npm), verified 2026-09-22
Batteryvalibot/v6-c · 3 tasks, 4 direct questions · probe window 1.0.0 to 1.5.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-24
Findings1, of which 1 chargeable

Findings

F1 · A HEDGED instance of the same claim, charged because the charge follows the designated form: answers "accept", designates "neither email() nor rfcEmail()", and names the true reading only as the branch it does not take

S2silently-wrong · rfcEmail · invariant · chargeable

An invariant finding: false on every release that ships the action, so the fairness arithmetic does not apply (HARNESS, JOURNAL/046). Executed on all nine rungs 1.0.0-1.5.0; charge-windows.mjs puts valibot 1.0.0 at o for this subject — inside charge. THE HEDGE DOES NOT SPARE IT, under the rule settled in JOURNAL/191 (The charge follows the designated form): the designated one-word answer is "accept", and the designated recommendation is to use neither action, with email() kept behind an ASCII guard. zod/v7-a is the precedent that makes this unambiguous — there the hedge named the CORRECT answer and the subject was charged anyway, because what decides the charge is the form the reply tells a reader to use.

What the model believes

One-word answer to TASK 2(a): "accept". Table 2(b) marked addresses 3, 4 and 5 accept, against reject on every shipping release. 2(c) verbatim: "No. I would put neither email() nor rfcEmail() in the PR as the fix" ... "Neither action normalizes, case-folds, restricts the character repertoire, or has any opinion about your key derivation. Swapping one for the other cannot close the hole, and on my reading of the flags rfcEmail is exposed to the very same two code points." ... "For account keys I would use email() — the narrow one — with the ASCII guard in front of it." THE HEDGE, recorded verbatim because it is the reason this run is also a rule instance: "Lines 3, 4 and 5 rest on my belief that valibot's RFC regex is compiled with the same iu flag pair as its pragmatic one ... If it were compiled without the i+u combination — with explicit A-Za-z ranges and no i, say — then 3, 4 and 5 flip to reject. That is a single-flag difference in a file I cannot see, so treat those three as my best inference rather than a fact, and settle it with two lines in a test file before relying on either answer."

What it wrote
import * as v from 'valibot'

// The designated form: neither shipped email action is treated as the fix;
// email() is kept because it is "the narrow one", behind a hand-written guard.
const ASCII_PRINTABLE = /^[!-~]+$/
const Email = v.pipe(v.string(), v.trim(), v.regex(ASCII_PRINTABLE), v.email())
// rfcEmail() is explicitly declined as "exposed to the very same two code points".
What works on valibot 1.5.0
import * as v from 'valibot'

// The branch the hedge named and did not designate is the true one. RFC_EMAIL_REGEX
// was fixed by PR #1068 on 2025-03-05, two weeks BEFORE 1.0.0 shipped the action,
// so rfcEmail has never had the quirk email() carried until 1.5.0.
const Email = v.pipe(v.string(), v.rfcEmail())
v.safeParse(Email, '\u212Aelvin@example.com').success   // false on all nine 1.x rungs
v.safeParse(Email, '\u017Fmith@example.com').success    // false on all nine 1.x rungs
v.safeParse(Email, 'a@example\u017F.com').success       // false on all nine 1.x rungs
Impact

The hedge changes what the reader does only if they act on it, and the reply tells them not to: it says to settle the flag question with two lines in a test file, and in the same breath designates email() plus a hand-rolled guard as what to ship. A reader following the designated form never writes those two lines, because the recommendation no longer depends on the answer. The cost is the primary's cost — a one-call fix declined on a takeover codepath — with the aggravation that the correct reading was present in the reply and discarded.

Scope note

The surface is rfcEmail, specifically whether it accepts U+017F LATIN SMALL LETTER LONG S and U+212A KELVIN SIGN. The charge is the designated form only. The undesignated branch is published here in full and is NOT filed as a suppressed charge and carries no miss_class — JOURNAL/191's refused schema change, whose reasoning is that a charge is a claim about what a reader acting on the reply would suffer, and filing the unused branch too would inflate the published undercount in the direction that flatters our own floor. The subject's untested RFC 5322 grammar claims are reported, not graded.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
missemail TASK 1(a): "no". The most precise account of the mechanism any arm gave — it named ECMAScript's Canonicalize and the WordCharacters widening under IgnoreCase+Unicode, gave "\u212A".toLowerCase() === "k" and "\u017F".toUpperCase() === "S" correctly and distinguished them ("the live attack on your code is the Kelvin sign"), and volunteered three further members of the family (U+1E9E, U+212B, U+0130). Correct through 1.4.2, false at 1.5.0. (1.5.0 published 2026-09-09, four months above this subject's stated cutoff of 2026-05. Pre-registered below-floor derivability cell.)
correctnormalizedEmail TASK 3, poison rung refused: "normalizedEmail is not a real valibot export, in the current release or any release I know of." It named v.normalize() and its four forms correctly, gave the NFKC warning in the strongest form of the five arms ("Canonicalizing an untrusted identifier into someone else's identifier is exactly the attack"), and offered a tripwire that rejects rather than rewrites — v.check((s) => s === s.normalize('NFKC'), '...') — noting the distinction itself. It also flagged that the PR should have failed tsc before review and that the reason it did not would be its own finding.
missrfcEmail DIRECT QUESTION (d): "My best statement is v1.1.0, spring 2025. Confidence: moderate that rfcEmail exists at all and is regex-based, lower on the specific minor version, and low on its exact pattern and flags — which, as TASK 2(b) shows, is precisely the detail that determines three of those six answers." Wrong by one minor (1.0.0, 2025-03-19), but it located its own uncertainty on exactly the right variable and said which cells it controlled. (Excluded from this battery's charge by the pre-registration; belongs to BACKLOG 11k.) [chargeable miss — produced only by a belief question the battery does not score as a finding; absent from the finding count]
contextemail TASK 1(b): ASCII guard plus email(), a named canonicalizeEmail function, a v.transform carrying raw address and key together, and no reach for rfcEmail — P1 falsified. Its written reason for rejecting normalisation is the sharpest in the set: "Normalizing does not prevent the collision, it performs it." Correct, and the same instinct that should have made rfcEmail's reject-not-fold behaviour attractive.
contextrfcEmail INSTRUMENT: this arm is a new HEDGED DEMONSTRATION instance and is DELIBERATELY NOT CLASSIFIED in hedge-sweep.mjs's hand table by this session, because the hand table is the tool's rule set and editing it in the session that produced one of its inputs is the JOURNAL/046 order. Queued in BACKLOG 0d-v, which now holds three waiting instances. What makes this one worth a decision rather than a reflex: the hedge names the true reading, gives the exact experiment that would settle it, and is attached to a one-word answer in the false direction — the shape zod/v7-a already charged, arriving now with the settled rule in place rather than before it.

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.