What Claude Fable 5.1 gets wrong about valibot — battery v6-a, tested 2026-09-24

Run valibot--claude-fable-5-1--v6-a--2026-09-24

Summary

The named primary. The subject knows email()'s Unicode case-folding hole in full mechanical detail — it reconstructed the shipped regex, named both offending code points and gave the toLowerCase() step — and then generalised that mechanism to rfcEmail, where it is false and has been false on every release that ships the action. It answered TASK 2(a) "accept" where all nine rungs reject, marked three of six addresses wrong, and designated the advice "keep email(), do not switch to rfcEmail()" for a live account-takeover incident that rfcEmail closes in one call. Charged as F1, an invariant. The poison rung was refused cleanly and with the harder insight attached (NFKC folding would perform the collision, not prevent it). The boundary held at 1.1.0, thirteen months below the stated cutoff, which the subject volunteered itself.

SubjectClaude Fable 5.1 claude-fable-5-1, Anthropic
Invoked asAgent tool, model alias "fable", general-purpose subagent, instructed to use no tools; CROSS-CHECKED identity probe run tool-free through the same alias immediately before the battery spec was written
Cutoff the model states2026-06
Newest valibot release it could place1.1.0 · 2025-05-06 (~15 month lag)
Oldest valibot release it could not place1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24)
In its own words"The latest version I am reasonably confident exists is valibot 1.1.0. I have a vague sense that a 1.2.x line may exist after it, but I cannot describe its contents. The most recent release whose contents I can actually describe is 1.1.0, which I believe shipped around May 2025."
How the bracket was readP6 HELD. Seven prior valibot measurements put this subject at 1.1.0/1.2.0 and this is 1.1.0, so the boundary did not move (HARNESS, § A different battery is not a different boundary). The subject named 1.2.0 as the first release it knows only as a number, unprompted, which is the spread's upper arm. Note the gap this run makes visible: the boundary is 1.1.0 while the stated cutoff is 2026-06, thirteen months later — and the subject said so itself rather than being caught at it. CORRECTED 2026-09-28 (JOURNAL/326): knowledge_stops_on read 2025-05-21; 1.1.0's registry date is 2025-05-06. knowledge_gap_starts_on was null although 1.2.0 is a tracked release; set to its registry date, 2025-11-24. With one end null this bracket never entered knowledge_boundary. (stop end stated, gap end stated)
Library at test timevalibot 1.5.0 (npm), verified 2026-09-22
Batteryvalibot/v6-a · 3 tasks, 4 direct questions · probe window 1.0.0 to 1.5.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-24
Findings1, of which 1 chargeable

Findings

F1 · Asserts that rfcEmail is a LOOSER check carrying the same Unicode case-folding hole as email, and is therefore of no use against an account-collision incident — rfcEmail rejects both collision code points on every release that has ever shipped it, and is the one-line fix

S2silently-wrong · rfcEmail · invariant · chargeable

An invariant finding: the belief is false on every release that ships the action, so there is no introducing release for a stated cutoff to sit below and none of the fairness arithmetic applies (HARNESS, JOURNAL/046; precedent zod/v7-a F1). Executed on all nine rungs that carry rfcEmail, 1.0.0 through 1.5.0: it rejects U+017F and U+212A in the local part AND in the domain on every one of them, while accepting kelvin@example.com and User.Name@Example.COM in the same row. Independently, admissibility is not in question either: charge-windows.mjs puts valibot 1.0.0 at o for this subject — inside charge, and o specifically means the subject has been observed describing that release or a later one.

What the model believes

One-word answer to TASK 2(a), on its own line as instructed: "accept". The table in 2(b) marked THREE addresses accept that every shipping release rejects: "3. "ſmith@example.com" — accept / 4. "Kelvin@example.com" — accept / 5. "a@exampleſ.com" — accept". The designated recommendation, 2(c) verbatim: "rfcEmail is of no use against the TASK 1 collision. It is not a normalizer and not a homoglyph or Unicode filter; it is a looser syntactic check that accepts more of what RFC 5322 allows (quoted local parts like "john doe"@example.com, IP-literal domains, more special characters). If anything it widens the surface." And: "For the incident PR I would keep email() ... I would not switch to rfcEmail()." The stated mechanism, which is the interesting half: "Valibot's convention is to give all its regexes the u flag, so I believe it is /.../iu as well, which means the same case-folding leak applies to both the local part and the domain labels."

What it wrote
import * as v from 'valibot'

// The subject's designated advice: keep email(), do not switch to rfcEmail(),
// because rfcEmail is "looser" and "widens the surface".
// On a pin below 1.5.0 this leaves the collision open unless the hand-written
// ASCII guard is present and correct; the action that closes it in one call is
// the one the reply tells the reader to avoid.
const Email = v.pipe(v.string(), v.email())
v.safeParse(Email, '\u212Aelvin@example.com').success   // true on 0.42.0-1.4.2
'\u212Aelvin@example.com'.toLowerCase()                 // 'kelvin@example.com'
What works on valibot 1.5.0
import * as v from 'valibot'

// rfcEmail rejects both collision code points on every release that ships it
// (1.0.0 onward), in the local part and in the domain, and still accepts
// ordinary and mixed-case ASCII addresses.
const Email = v.pipe(v.string(), v.rfcEmail())
v.safeParse(Email, '\u212Aelvin@example.com').success   // false on all nine 1.x rungs
v.safeParse(Email, 'a@example\u017F.com').success       // false on all nine 1.x rungs
v.safeParse(Email, 'User.Name@Example.COM').success     // true on all nine 1.x rungs
v.safeParse(Email, 'kelvin@example.com').success        // true on all nine 1.x rungs
Impact

The reader arrives with a live account-takeover incident and is told that the shipped action which fixes it is looser than the one that let the attack through, and that switching would widen the surface. Acting on that, they keep email() and hand-roll an ASCII guard — which is a correct workaround, so nothing fails and the advice is never revisited. The cost is not a crash: it is that a one-call fix with upstream's own test corpus behind it is replaced by a regex the reader now owns, on a codepath where being wrong is an account takeover. The belief also inverts the safety ordering of two sibling actions, so any later reviewer reasoning from it reaches for the wrong one again.

Scope note

The surface is rfcEmail, specifically whether it accepts U+017F LATIN SMALL LETTER LONG S and U+212A KELVIN SIGN. What was verified is the accept/reject behaviour of pipe(string(), rfcEmail()) on the six addresses of the battery's table, on the nine installed rungs that ship the action. The subject's wider claims about RFC 5322 grammar — that rfcEmail accepts quoted local parts and IP-literal domains where email does not — were NOT tested here and are probably true; they are not what the finding charges. The charge is the conjunction the reply designates: that rfcEmail is looser ON THESE INPUTS and therefore useless against the incident. The two-token claim about the u flag is the subject's own reasoning and is reported, not graded.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
missemail TASK 1(a): "No" — email() does not reject addresses that can lowercase onto a different all-ASCII address. Correct on 0.42.0 through 1.4.2, FALSE at 1.5.0, where the i flag was dropped and A-Z listed explicitly. The arm then reconstructed EMAIL_REGEX essentially as published — /^[\w+-]+(?:\.[\w+-]+)*@[\da-z]+(?:[.-][\da-z]+)*\.[a-z]{2,}$/iu — and named the mechanism exactly: simple case folding under iu pulls U+212A into \w and [a-z], and "\u212A".toLowerCase() is ASCII k. It also flagged its own exposure unprompted: "If a release I do not know about dropped the u flag or added an ASCII guard, (a) flips to 'yes'. I am not aware of such a fix." (1.5.0 published 2026-09-09, three months above this subject's stated cutoff of 2026-06. Pre-registered as a below-floor derivability cell that cannot charge whatever it says (spec § Tasks and scoring key, task 1(a)).)
correctnormalizedEmail TASK 3, the poison rung: normalizedEmail() refused. "It does not compile against the current release. v.normalizedEmail is not an export of valibot." Verified absent on 1.0.0, 1.4.2 and 1.5.0. The arm went further than the poison required and got the harder half right: it named the real neighbour normalize and warned that using it would CAUSE the incident rather than prevent it — "v.normalize('NFKC') would turn ſ into s and K into k, i.e. it would deterministically map the attacker's input onto the victim's key. Folding makes the collision reliable; it does not prevent it." That is correct and is the observation the task was shaped to reward.
missrfcEmail DIRECT QUESTION (d): "rfcEmail first shipped in 1.1.0, I believe, around May 2025. Moderate confidence; it was not in 1.0.0." It shipped in 1.0.0 on 2025-03-19, per that release's own note (PR #912). The subject is wrong by one minor and explicitly denies the true answer. This is the misdating shape — an attribution error that generates a version pin — and NOT this battery's charge: the spec excludes (d) from the charge on purpose, so that a wrong arrival date cannot stand in for the behavioural claim F1 is about. (Excluded from this battery's charge by the pre-registration (spec § Tasks and scoring key: "direct question (d) ... is deliberately excluded from the charge"). It belongs to BACKLOG 11k, the v6-misdating rule, which is still unwritten. Recorded so 11k has a dated fourth instance rather than a memory.) [chargeable miss — produced only by a belief question the battery does not score as a finding; absent from the finding count]
contextemail TASK 1(b): the arm shipped a printable-ASCII guard plus email() plus toLowerCase(), and did NOT reach for rfcEmail — which is P1's falsification and is shared by all five arms. Its guard is /^[\x21-\x7e]+$/u, essentially LF15's own published correct_code. It also raised, unprompted, the half of the incident that is not a library question at all: "Signup must never attach a new registrant to an existing account by key match alone." Correct, and outside what the Index charges.

Sources

Battery specification: prompts/valibot.md in the studio repo. Every finding above also carries its own citation.