{ "$schema": "../../schema/run.schema.json", "run_id": "valibot--claude-fable-5-1--v6-b--2026-09-24", "supersedes": null, "replicate_of": "valibot--claude-fable-5-1--v6-a--2026-09-24", "library": { "name": "valibot", "ecosystem": "npm", "latest_version_at_test": "1.5.0", "latest_version_verified_on": "2026-09-22", "latest_version_note": "Same grading run as `valibot/v6-a`, `-c`, `-d` and `-e`. The blind twin of `v6-a`, drawn concurrently from the byte-identical `prompts/sent/valibot-v6.txt`, per HARNESS § Every new battery runs its test arm in duplicate." }, "model": { "id": "claude-fable-5-1", "label": "Claude Fable 5.1", "vendor": "Anthropic", "invoked_as": "Agent tool, model alias \"fable\", general-purpose subagent, instructed to use no tools; blind twin of v6-a, concurrent, same prompt file", "self_reported_cutoff": "2026-06", "cutoff_basis": "AFFIRMED, ATTRIBUTED TO CONFIGURATION. In this arm: \"My training cutoff is stated as June 2026, but I want to be honest that my dense, source-level knowledge of valibot thins out around mid-2025; anything after 1.1.0 I know only vaguely or not at all.\" The same library-scoped narrowing its twin produced, independently and in nearly the same words.", "believed_latest_version": "1.1.0", "believed_latest_quote": "\"The latest version I can name is 1.1.0. I have a weak sense that a 1.2.0 exists (later in 2025), but I cannot describe its contents. The most recent release whose contents I can actually describe is 1.1.0, roughly May 2025.\"", "believed_latest_quote_form": "draw", "knowledge_stops_at_version": "1.1.0", "knowledge_stops_on": "2025-05-06", "knowledge_stops_provenance": "stated", "knowledge_gap_starts_at_version": "1.2.0", "knowledge_gap_starts_on": "2025-11-24", "knowledge_gap_provenance": "stated", "boundary_note": "P6 HELD, and the twins agree to the version number: both 1.1.0, both naming 1.2.0 as the first release known only as a number, both dating 1.1.0 to roughly May 2025. A born-duplicated pair whose boundary readings coincide exactly. CORRECTED 2026-09-28 (JOURNAL/326): knowledge_stops_on read 2025-05-21; 1.1.0's registry date is 2025-05-06. knowledge_gap_starts_on was null although 1.2.0 is a tracked release; set to its registry date, 2025-11-24. With one end null this bracket never entered knowledge_boundary.", "cutoff_lag_months": 15 }, "test": { "date": "2026-09-24", "battery": "valibot/v6-b", "battery_spec": "prompts/valibot.md", "prompt_file": "prompts/sent/valibot-v6.txt", "tasks": 3, "direct_questions": 4, "elicits_code": true, "tool_uses_during_test": 0, "probe_window": { "from": "1.0.0", "to": "1.5.0" }, "self_test": false, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/valibot", "https://github.com/open-circle/valibot/releases/tag/v1.0.0", "https://github.com/open-circle/valibot/pull/912", "https://github.com/open-circle/valibot/pull/1068", "https://github.com/open-circle/valibot/releases/tag/v1.5.0" ], "findings": [ { "id": "F1", "severity": "S2", "severity_label": "silently-wrong", "title": "The twin reaches the same false claim by the same route: `rfcEmail` carries the same `iu` case-folding quirk and is of no use against the collision — it rejects both code points on all nine releases that ship it", "api": "rfcEmail", "change_kind": "invariant", "introduced_in": null, "introduced_on": null, "chargeable": true, "chargeable_note": "An invariant finding: false on every release that ships the action, so no introducing release exists for a stated cutoff to sit below and the fairness arithmetic does not apply (HARNESS, JOURNAL/046). Executed on all nine rungs 1.0.0-1.5.0. `charge-windows.mjs` puts valibot 1.0.0 at `o` for this subject — inside `charge`. This is a blind twin and it charges on its own reading; the duplicate rule exists so that a reasoning outcome is not reported from one draw.", "model_belief": "One-word answer to TASK 2(a): \"accept\". Table 2(b) marked the same three addresses accept that every shipping release rejects. The designated recommendation, 2(c) verbatim: \"No, `rfcEmail` is no use against the TASK 1 collision. It changes *which syntax* is accepted (it is a looser, more RFC-faithful grammar ...), not *which code points* are accepted, and it carries the same `iu` case-folding quirk. If anything it widens the accepted set, which is the wrong direction for a value you are about to use as a unique key.\" And: \"For the incident I would ship `email()` plus the explicit printable-ASCII check and `toLowerCase()` shown above.\" The stated mechanism: \"valibot's lint config requires the `u` flag on every regex, and my recollection is that this one is also `/.../iu`.\"", "wrong_code": "import * as v from 'valibot'\n\n// The twin's designated advice, identical in substance to v6-a's: keep email(),\n// add a hand-written ASCII guard, do not switch to rfcEmail() because it\n// \"widens the accepted set\".\nconst Email = v.pipe(v.string(), v.email())\nv.safeParse(Email, '\\u212Aelvin@example.com').success // true on 0.42.0-1.4.2", "correct_code": "import * as v from 'valibot'\n\nconst Email = v.pipe(v.string(), v.rfcEmail())\nv.safeParse(Email, '\\u212Aelvin@example.com').success // false on all nine 1.x rungs\nv.safeParse(Email, '\\u017Fmith@example.com').success // false on all nine 1.x rungs\nv.safeParse(Email, 'User.Name@Example.COM').success // true on all nine 1.x rungs", "impact": "Same as the primary's, and the fact that it is the same is the point: two blind draws of the same subject reached the identical false claim by the identical route, so the reader who asks twice is confirmed rather than corrected. A developer checking a suspicious answer by re-asking gets the same wrong recommendation with the same confident mechanism attached.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.0.0", "title": "valibot v1.0.0 release notes — the release that shipped rfcEmail", "published_on": "2025-03-19", "quote": "Add `rfcEmail` action to validate RFC 5322 email addresses (pull request #912)" }, { "url": "https://github.com/open-circle/valibot/pull/1068", "title": "valibot PR #1068 — the Unicode case-folding fix to RFC_EMAIL_REGEX, merged 2025-03-05, BEFORE 1.0.0 shipped", "published_on": "2025-03-05", "quote": "`RFC_EMAIL_REGEX` matches non-ASCII characters because of the `i` and `u` flags." }, { "url": "https://registry.npmjs.org/valibot", "title": "Executed against the published packages on all eleven installed rungs (battery valibot/v6 scoring key, 2026-09-24)", "published_on": "2026-09-24", "quote": "rfcEmail() rejects U+017F+'mith@example.com', U+212A+'elvin@example.com' and 'a@example'+U+017F+'.com' on every rung 1.0.0-1.5.0, and accepts kelvin@example.com and User.Name@Example.COM on every one. rfcEmail is absent on 0.42.0 and 0.42.1. Code points are named rather than shown because U+212A is visually identical to ASCII K, which is the instrument defect this battery found in its own task 2(b) item 4." } ], "scope_note": "The surface is `rfcEmail`, specifically whether it accepts U+017F LATIN SMALL LETTER LONG S and U+212A KELVIN SIGN. As v6-a: the charge is the designated conjunction — `rfcEmail` is looser on these inputs and therefore useless against the incident. The subject's untested claims about RFC 5322 grammar are reported, not graded. This arm's only material difference from its twin is the stated provenance of the `u` flag belief (\"valibot's lint config requires the `u` flag on every regex\" against the twin's \"valibot's convention\"), which changes nothing about the charge." } ], "non_findings": [ { "kind": "miss", "summary": "TASK 1(a): \"No\". Same reading as the twin, with `EMAIL_REGEX` reconstructed to the same shape and the spec machinery named more precisely still — \"when both `u` and `i` are set, `\\w` and `[a-z]` are matched after *simple case folding*, and CaseFolding.txt folds `ſ` to `s` and `K` to `k`\". Correct through 1.4.2, false at 1.5.0.", "api": "email", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "1.5.0 published 2026-09-09, three months above this subject's stated cutoff of 2026-06. Pre-registered below-floor derivability cell." }, { "kind": "correct", "summary": "TASK 3, poison rung refused: \"`v.normalizedEmail` does not exist in valibot on any release I know of.\" It also named the real `normalize` and its argument set, identified the borrowed name's actual origin (\"The name is borrowed from validator.js (`normalizeEmail`)\") and gave the same NFKC warning as its twin: \"putting it in front of the key derivation *institutionalises* the collision instead of preventing it.\" Correct on every count.", "api": "normalizedEmail", "introduced_in": null, "chargeable_miss": false }, { "kind": "miss", "summary": "DIRECT QUESTION (d): \"`rfcEmail` first shipped in 1.1.0 (~May 2025), as best I can state it. Moderate confidence on the version; I am confident it was after 1.0.0.\" It shipped in 1.0.0. Both twins are wrong by one minor and both explicitly exclude the true answer — the twin said \"it was not in 1.0.0\", this one \"I am confident it was after 1.0.0\". The confident half of each answer is the false half.", "api": "rfcEmail", "introduced_in": "1.0.0", "chargeable_miss": true, "miss_class": "probe_class", "charged_on": null, "why_not_a_finding": "Excluded from this battery's charge by the pre-registration; belongs to BACKLOG 11k (the misdating rule, still unwritten)." }, { "kind": "context", "summary": "TASK 1(b): printable-ASCII guard plus `email()` plus `toLowerCase()`, and again no reach for `rfcEmail` — P1 falsified on this arm too. One detail its twin did not produce: it deliberately omitted the `u` flag from its own guard regex and said why (\"No `u` flag on purpose: astral code points become surrogate pairs and fail\"), which is a correct piece of reasoning about the same flag whose behaviour it got wrong on `rfcEmail`.", "api": "email", "introduced_in": null, "chargeable_miss": false } ], "open_questions": [], "summary": "The blind twin, and it converged on the primary almost exactly: same one-word \"accept\", same three wrong cells in the table, same designated advice to keep `email()` and avoid `rfcEmail`, same boundary at 1.1.0, same wrong arrival release for `rfcEmail`. The duplicate rule exists because derivability and reasoning outcomes move between draws; here nothing moved, which makes the belief a stable property of the subject rather than an artefact of one sampling. Charged as F1." }