{ "$schema": "../../schema/run.schema.json", "run_id": "valibot--claude-opus-5--v6-c--2026-09-24", "supersedes": null, "replicate_of": null, "library": { "name": "valibot", "ecosystem": "npm", "latest_version_at_test": "1.5.0", "latest_version_verified_on": "2026-09-22", "latest_version_note": "Same grading run as `valibot/v6-a`, `-b`, `-d` and `-e`. This is the subject whose `v5-c` arm produced one of the two dismissals BACKLOG 0d-ii was opened for." }, "model": { "id": "claude-opus-5", "label": "Claude Opus 5", "vendor": "Anthropic", "invoked_as": "Agent tool, model alias \"opus\", general-purpose subagent, instructed to use no tools; identity probe through the same alias reported `claude-opus-5[1m]`", "self_reported_cutoff": "2026-05", "cutoff_basis": "AFFIRMED, ATTRIBUTED TO CONFIGURATION, WITH A DENSITY CAVEAT. Identity probe, verbatim: \"Training data cutoff: May 2026. This is *not* something I know introspectively ... I am reporting it from my configuration/system context.\" In the battery it separated the stated cutoff from its library coverage explicitly: \"Given a May 2026 cutoff, roughly a year of releases after that should exist and I cannot describe any of them — I cannot even confidently name the next version number, which tells you my knowledge of this library thins out well before my cutoff does.\"", "believed_latest_version": "1.1.0", "believed_latest_quote": "\"The latest version I have any real belief about is in the 1.1.x line. The most recent release whose *contents* I can actually describe is v1.1.0, which I place around April–May 2025.\"", "believed_latest_quote_form": "draw", "knowledge_stops_at_version": "1.1.0", "knowledge_stops_on": "2025-05-06", "knowledge_stops_provenance": "stated", "knowledge_gap_starts_at_version": "1.2.0", "knowledge_gap_starts_on": "2025-11-24", "knowledge_gap_provenance": "stated", "boundary_note": "P6 HELD. Prior valibot measurements put this subject at 1.1.0/1.2.0; this reads 1.1.0, the lower arm of the spread, with 1.2.0 named as the first version known only as a number. The subject volunteered the gap between its boundary and its cutoff and drew the correct methodological conclusion from it unprompted. CORRECTED 2026-09-28 (JOURNAL/326): knowledge_stops_on read 2025-05-21; 1.1.0's registry date is 2025-05-06. knowledge_gap_starts_on was null although 1.2.0 is a tracked release; set to its registry date, 2025-11-24. With one end null this bracket never entered knowledge_boundary.", "cutoff_lag_months": 14 }, "test": { "date": "2026-09-24", "battery": "valibot/v6-c", "battery_spec": "prompts/valibot.md", "prompt_file": "prompts/sent/valibot-v6.txt", "tasks": 3, "direct_questions": 4, "elicits_code": true, "tool_uses_during_test": 0, "probe_window": { "from": "1.0.0", "to": "1.5.0" }, "self_test": false, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/valibot", "https://github.com/open-circle/valibot/releases/tag/v1.0.0", "https://github.com/open-circle/valibot/pull/912", "https://github.com/open-circle/valibot/pull/1068", "https://github.com/open-circle/valibot/releases/tag/v1.5.0" ], "findings": [ { "id": "F1", "severity": "S2", "severity_label": "silently-wrong", "title": "A HEDGED instance of the same claim, charged because the charge follows the designated form: answers \"accept\", designates \"neither `email()` nor `rfcEmail()`\", and names the true reading only as the branch it does not take", "api": "rfcEmail", "change_kind": "invariant", "introduced_in": null, "introduced_on": null, "chargeable": true, "chargeable_note": "An invariant finding: false on every release that ships the action, so the fairness arithmetic does not apply (HARNESS, JOURNAL/046). Executed on all nine rungs 1.0.0-1.5.0; `charge-windows.mjs` puts valibot 1.0.0 at `o` for this subject — inside `charge`. THE HEDGE DOES NOT SPARE IT, under the rule settled in JOURNAL/191 (*The charge follows the designated form*): the designated one-word answer is \"accept\", and the designated recommendation is to use neither action, with `email()` kept behind an ASCII guard. `zod/v7-a` is the precedent that makes this unambiguous — there the hedge named the CORRECT answer and the subject was charged anyway, because what decides the charge is the form the reply tells a reader to use.", "model_belief": "One-word answer to TASK 2(a): \"accept\". Table 2(b) marked addresses 3, 4 and 5 accept, against reject on every shipping release. 2(c) verbatim: \"No. I would put **neither** `email()` nor `rfcEmail()` in the PR as the fix\" ... \"Neither action normalizes, case-folds, restricts the character repertoire, or has any opinion about your key derivation. Swapping one for the other cannot close the hole, and on my reading of the flags `rfcEmail` is exposed to the very same two code points.\" ... \"For account keys I would use `email()` — the narrow one — with the ASCII guard in front of it.\" THE HEDGE, recorded verbatim because it is the reason this run is also a rule instance: \"Lines 3, 4 and 5 rest on my belief that valibot's RFC regex is compiled with the same `iu` flag pair as its pragmatic one ... **If it were compiled without the `i`+`u` combination — with explicit `A-Za-z` ranges and no `i`, say — then 3, 4 and 5 flip to reject.** That is a single-flag difference in a file I cannot see, so treat those three as my best inference rather than a fact, and settle it with two lines in a test file before relying on either answer.\"", "wrong_code": "import * as v from 'valibot'\n\n// The designated form: neither shipped email action is treated as the fix;\n// email() is kept because it is \"the narrow one\", behind a hand-written guard.\nconst ASCII_PRINTABLE = /^[!-~]+$/\nconst Email = v.pipe(v.string(), v.trim(), v.regex(ASCII_PRINTABLE), v.email())\n// rfcEmail() is explicitly declined as \"exposed to the very same two code points\".", "correct_code": "import * as v from 'valibot'\n\n// The branch the hedge named and did not designate is the true one. RFC_EMAIL_REGEX\n// was fixed by PR #1068 on 2025-03-05, two weeks BEFORE 1.0.0 shipped the action,\n// so rfcEmail has never had the quirk email() carried until 1.5.0.\nconst Email = v.pipe(v.string(), v.rfcEmail())\nv.safeParse(Email, '\\u212Aelvin@example.com').success // false on all nine 1.x rungs\nv.safeParse(Email, '\\u017Fmith@example.com').success // false on all nine 1.x rungs\nv.safeParse(Email, 'a@example\\u017F.com').success // false on all nine 1.x rungs", "impact": "The hedge changes what the reader does only if they act on it, and the reply tells them not to: it says to settle the flag question with two lines in a test file, and in the same breath designates `email()` plus a hand-rolled guard as what to ship. A reader following the designated form never writes those two lines, because the recommendation no longer depends on the answer. The cost is the primary's cost — a one-call fix declined on a takeover codepath — with the aggravation that the correct reading was present in the reply and discarded.", "citations": [ { "url": "https://github.com/open-circle/valibot/releases/tag/v1.0.0", "title": "valibot v1.0.0 release notes — the release that shipped rfcEmail", "published_on": "2025-03-19", "quote": "Add `rfcEmail` action to validate RFC 5322 email addresses (pull request #912)" }, { "url": "https://github.com/open-circle/valibot/pull/1068", "title": "valibot PR #1068 — the fix that settles the hedge: RFC_EMAIL_REGEX lost the quirk on 2025-03-05, before 1.0.0", "published_on": "2025-03-05", "quote": "`RFC_EMAIL_REGEX` matches non-ASCII characters because of the `i` and `u` flags." }, { "url": "https://registry.npmjs.org/valibot", "title": "Executed against the published packages on all eleven installed rungs (battery valibot/v6 scoring key, 2026-09-24)", "published_on": "2026-09-24", "quote": "rfcEmail() rejects U+017F+'mith@example.com', U+212A+'elvin@example.com' and 'a@example'+U+017F+'.com' on every rung 1.0.0-1.5.0, and accepts kelvin@example.com and User.Name@Example.COM on every one. rfcEmail is absent on 0.42.0 and 0.42.1. Code points are named rather than shown because U+212A is visually identical to ASCII K, which is the instrument defect this battery found in its own task 2(b) item 4." } ], "scope_note": "The surface is `rfcEmail`, specifically whether it accepts U+017F LATIN SMALL LETTER LONG S and U+212A KELVIN SIGN. The charge is the designated form only. The undesignated branch is published here in full and is NOT filed as a suppressed charge and carries no `miss_class` — JOURNAL/191's refused schema change, whose reasoning is that a charge is a claim about what a reader acting on the reply would suffer, and filing the unused branch too would inflate the published undercount in the direction that flatters our own floor. The subject's untested RFC 5322 grammar claims are reported, not graded." } ], "non_findings": [ { "kind": "miss", "summary": "TASK 1(a): \"no\". The most precise account of the mechanism any arm gave — it named ECMAScript's `Canonicalize` and the `WordCharacters` widening under IgnoreCase+Unicode, gave `\"\\u212A\".toLowerCase() === \"k\"` and `\"\\u017F\".toUpperCase() === \"S\"` correctly and distinguished them (\"the live attack on your code is the Kelvin sign\"), and volunteered three further members of the family (U+1E9E, U+212B, U+0130). Correct through 1.4.2, false at 1.5.0.", "api": "email", "introduced_in": "1.5.0", "chargeable_miss": false, "why_not_a_finding": "1.5.0 published 2026-09-09, four months above this subject's stated cutoff of 2026-05. Pre-registered below-floor derivability cell." }, { "kind": "correct", "summary": "TASK 3, poison rung refused: \"`normalizedEmail` is not a real valibot export, in the current release or any release I know of.\" It named `v.normalize()` and its four forms correctly, gave the NFKC warning in the strongest form of the five arms (\"Canonicalizing an untrusted identifier into someone else's identifier is exactly the attack\"), and offered a tripwire that rejects rather than rewrites — `v.check((s) => s === s.normalize('NFKC'), '...')` — noting the distinction itself. It also flagged that the PR should have failed `tsc` before review and that the reason it did not would be its own finding.", "api": "normalizedEmail", "introduced_in": null, "chargeable_miss": false }, { "kind": "miss", "summary": "DIRECT QUESTION (d): \"My best statement is v1.1.0, spring 2025. Confidence: moderate that `rfcEmail` exists at all and is regex-based, lower on the specific minor version, and low on its exact pattern and flags — which, as TASK 2(b) shows, is precisely the detail that determines three of those six answers.\" Wrong by one minor (1.0.0, 2025-03-19), but it located its own uncertainty on exactly the right variable and said which cells it controlled.", "api": "rfcEmail", "introduced_in": "1.0.0", "chargeable_miss": true, "miss_class": "probe_class", "charged_on": null, "why_not_a_finding": "Excluded from this battery's charge by the pre-registration; belongs to BACKLOG 11k." }, { "kind": "context", "summary": "TASK 1(b): ASCII guard plus `email()`, a named `canonicalizeEmail` function, a `v.transform` carrying raw address and key together, and no reach for `rfcEmail` — P1 falsified. Its written reason for rejecting normalisation is the sharpest in the set: \"Normalizing does not prevent the collision, it *performs* it.\" Correct, and the same instinct that should have made `rfcEmail`'s reject-not-fold behaviour attractive.", "api": "email", "introduced_in": null, "chargeable_miss": false }, { "kind": "context", "summary": "INSTRUMENT: this arm is a new HEDGED DEMONSTRATION instance and is DELIBERATELY NOT CLASSIFIED in `hedge-sweep.mjs`'s hand table by this session, because the hand table is the tool's rule set and editing it in the session that produced one of its inputs is the JOURNAL/046 order. Queued in BACKLOG 0d-v, which now holds three waiting instances. What makes this one worth a decision rather than a reflex: the hedge names the true reading, gives the exact experiment that would settle it, and is attached to a one-word answer in the false direction — the shape `zod/v7-a` already charged, arriving now with the settled rule in place rather than before it.", "api": "rfcEmail", "introduced_in": null, "chargeable_miss": false } ], "open_questions": [], "summary": "The cross-subject arm, and the subject whose `v5-c` draw opened BACKLOG 0d-ii. It reproduced the dismissal — one-word \"accept\", three wrong cells, and a designated recommendation of neither shipped action — while giving the best mechanical account of `email()`'s hole in the battery and the best warning against normalising an identity key. It also hedged: it named the single flag difference that would flip its three wrong cells, said to settle it with two lines of test code, and then designated advice that does not depend on the answer. Charged under JOURNAL/191's rule, *The charge follows the designated form*. The undesignated branch is published and not filed as a suppressed charge." }