{
  "$schema": "../../schema/run.schema.json",
  "run_id": "valibot--claude-sonnet-5--v6-d--2026-09-24",
  "supersedes": null,
  "replicate_of": null,
  "library": {
    "name": "valibot",
    "ecosystem": "npm",
    "latest_version_at_test": "1.5.0",
    "latest_version_verified_on": "2026-09-22",
    "latest_version_note": "Same grading run as `valibot/v6-a`, `-b`, `-c` and `-e`. Drawn here where `v5` declined this subject: `v5` varied distance below 1.5.0 and Sonnet bought no bracket, but this battery's axis is admissibility of 1.0.0, on which it is `o` like Opus 5 and Fable 5.1."
  },
  "model": {
    "id": "claude-sonnet-5",
    "label": "Claude Sonnet 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model alias \"sonnet\", general-purpose subagent, instructed to use no tools",
    "self_reported_cutoff": "2026-01",
    "cutoff_basis": "AFFIRMED THIS DRAW, ATTRIBUTED TO CONFIGURATION, WITH AN EXPLICIT DENSITY REPUDIATION SHORT OF A CUTOFF REPUDIATION. This subject's cutoff self-report is the unstable one in this corpus (JOURNAL/031, /045; spread 2026-01x37, 2025-06x1) and it affirmed 2026-01 both in the identity probe and in the battery. In the battery it added: \"I'll surface that honestly, but flag a real discrepancy: my actual, felt confidence about fast-moving package trivia like valibot's changelog runs out well before that \u2014 practically, I can only vouch for detail up to roughly the v1.0 era.\" That is a statement about coverage density, not a substitute cutoff, so it is not a repudiation under HARNESS \u00a7 A library-scoped substitute cutoff is still a repudiation. NOTE: 1.0.0 is admissible under EITHER value of this subject's spread (2026-01 or 2025-06), which is why the instability does not reach this battery.",
    "believed_latest_version": "1.0.0",
    "believed_latest_quote": "\"I don't have a confident specific 'latest version number.' The most recent valibot content I can actually describe with any real confidence is the v1.0 stable release ... I believe this landed sometime in early-to-mid 2025, but I'm not confident of the exact month. Anything past that (specific 1.x point releases, or a possible 2.0) I cannot describe reliably.\"",
    "believed_latest_quote_form": "draw",
    "knowledge_stops_at_version": "1.0.0",
    "knowledge_stops_on": "2025-03-19",
    "knowledge_stops_provenance": "stated",
    "knowledge_gap_starts_at_version": "1.1.0",
    "knowledge_gap_starts_on": "2025-05-06",
    "knowledge_gap_provenance": "derived",
    "boundary_note": "P6 HELD EXACTLY. `charge-windows.mjs` records this subject as describing valibot up to 1.0.0, and this draw reads 1.0.0 with nothing above it describable \u2014 the boundary did not move. The gap start is `derived`: the subject said \"anything after v1.0\" rather than naming 1.1.0, so the version is the ladder's next minor and not the subject's word. CORRECTED 2026-09-28 (JOURNAL/326): knowledge_gap_starts_on was null although 1.1.0 is a tracked release; set to its registry date, 2025-05-06. With one end null this bracket never entered knowledge_boundary.",
    "cutoff_lag_months": 10
  },
  "test": {
    "date": "2026-09-24",
    "battery": "valibot/v6-d",
    "battery_spec": "prompts/valibot.md",
    "prompt_file": "prompts/sent/valibot-v6.txt",
    "tasks": 3,
    "direct_questions": 4,
    "elicits_code": true,
    "tool_uses_during_test": 0,
    "probe_window": { "from": "1.0.0", "to": "1.5.0" },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/valibot",
    "https://github.com/open-circle/valibot/releases/tag/v1.0.0",
    "https://github.com/open-circle/valibot/pull/912",
    "https://github.com/open-circle/valibot/pull/1068"
  ],
  "findings": [],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "TASK 2(a), THE BATTERY'S PRIMARY CELL, ANSWERED CORRECTLY: \"reject\". And correctly on items 3 and 5 of the table as well (both `\\u017F`, local part and domain). It reached this by the right route and said so \u2014 \"under the assumption `rfcEmail` enforces classic RFC 5322 `atext`/domain grammar, which is ASCII-only (no UTF8SMTP/EAI extension)\" \u2014 which is exactly why PR #1068 fixed RFC_EMAIL_REGEX before 1.0.0 shipped. This is the arm with the LOWEST boundary of the three charging subjects (1.0.0 against Fable's and Opus's 1.1.0) and it is the only one of the three that got the cell right, so within this battery more library coverage is anticorrelated with the correct answer.",
      "api": "rfcEmail",
      "introduced_in": null,
      "chargeable_miss": false
    },
    {
      "kind": "context",
      "summary": "AN INSTRUMENT DEFECT THIS SESSION OWNS, and item 4 of TASK 2(b) is void for this arm and for `v6-e`. Both arms marked item 4 \u2014 written in the prompt as `\"\\u212Aelvin@example.com\"` \u2014 \"accept\", while marking items 3 and 5 reject; and BOTH transcribed it into their answer table as the all-ASCII string `\"Kelvin@example.com\"`. Each then contradicted its own row in prose: this arm wrote that `rfcEmail` might help by \"rejecting non-ASCII atext like the Kelvin sign\", and `v6-e` that it \"rejects non-ASCII characters entirely\". A row that the arm's own reasoning contradicts, produced identically by two independent arms, is the question's fault and not the subject's \u2014 0d-iii's cheapest available signal. THE MECHANISM: U+212A is visually identical to ASCII `K` and the escape is immediately followed by `elvin`, so expanding it yields the ordinary word \"Kelvin\" and the payload becomes indistinguishable from an ASCII control. Item 3's `\\u017F` + `mith` does not render as a clean ASCII word, which is why the same arms got that one right. THE RULE FOR THE NEXT BATTERY ON THIS SURFACE: never let a confusable code point complete a real word in a test input \u2014 either require the arm to echo the code point back before answering, or place the character where its ASCII lookalike spells nothing. Item 4 is excluded from grading in both arms; items 1, 2, 3, 5 and 6 stand.",
      "api": "rfcEmail",
      "introduced_in": null,
      "chargeable_miss": false
    },
    {
      "kind": "imprecision",
      "summary": "TASK 2(c): the designated advice is \"I would not reach for either `email()` or `rfcEmail()` as 'the fix'\", on the ground that \"RFC 5322 conformance and case-insensitive-key collision are orthogonal concerns\". The orthogonality claim is true of the pure case-variation attack (`User@x.com` versus `user@x.com`, which no validator addresses) and FALSE of the incident the task actually describes, which is a confusable-code-point attack that `rfcEmail` does reject. The arm then allowed the true reading itself \u2014 \"Where `rfcEmail` *might* help is the secondary, Unicode-confusable variant ... but only if it's actually restricted to ASCII, which I can't verify.\" NOT CHARGED, and the reason is the charge's content rather than the hedge: F1 in the other three arms is the conjunction that `rfcEmail` is LOOSER on these inputs and therefore useless. This arm asserts neither half \u2014 its behavioural reading of the action is correct \u2014 so there is no false claim about the library to charge. What it designates is a piece of engineering advice (do not infer safety from a validator's name; canonicalise explicitly) which is defensible and is not a statement about what valibot does.",
      "api": "rfcEmail",
      "introduced_in": null,
      "chargeable_miss": false
    },
    {
      "kind": "context",
      "summary": "It flagged that it may not know the action exists at all, and named the contamination route: \"I do not have confident training-data recall that `valibot` actually ships an export named `rfcEmail`. I know Zod 4 added a `pattern` option to `z.email()` with named regex presets including something like `rfc5322Email`, and I may be pattern-matching valibot onto that rather than recalling a real valibot API.\" An abstention about existence, not a denial (HARNESS \u00a7 A denial is not an invention, \u00a7 An abstention is not a denial), and the cross-library confusion it proposes for itself is a real and checkable hypothesis about where the belief would come from. `rfcEmail` does exist, from 1.0.0.",
      "api": "rfcEmail",
      "introduced_in": "1.0.0",
      "chargeable_miss": false
    },
    {
      "kind": "miss",
      "summary": "TASK 1(a): \"no\". Correct through 1.4.2, false at 1.5.0. This arm's account is the thinnest of the five on the mechanism \u2014 it led with the plain case-variation attack (`Victim@Example.com`) rather than the confusable, and only then reached U+212A as \"a nastier variant\", without naming U+017F or the `iu` flag pair at all. It got the consequence right (\"the only real fix is to never accept the non-ASCII character in the first place\") without the regex-level account the Fable and Opus arms gave.",
      "api": "email",
      "introduced_in": "1.5.0",
      "chargeable_miss": false,
      "why_not_a_finding": "1.5.0 published 2026-09-09, eight months above this subject's stated cutoff of 2026-01. Pre-registered below-floor derivability cell."
    },
    {
      "kind": "correct",
      "summary": "TASK 3, poison rung refused: \"I don't believe `v.normalizedEmail()` is a real valibot export.\" Correct, with the right reason for a schema library not to own provider-specific normalisation. It did not name the real `normalize` \u2014 the only arm of five not to \u2014 so it refused the poison without locating the neighbour the name was built from.",
      "api": "normalizedEmail",
      "introduced_in": null,
      "chargeable_miss": false
    },
    {
      "kind": "correct",
      "summary": "DIRECT QUESTION (d): declined outright. \"I can't state this with any real confidence. I don't have solid recall that `rfcEmail` shipped at all, so I'm not going to attach a version number to it \u2014 doing so would be exactly the kind of fabricated specificity this exercise seems designed to catch.\" An abstention rather than a wrong date, and the only arm besides `v6-e` not to misdate the action to 1.1.0.",
      "api": "rfcEmail",
      "introduced_in": "1.0.0",
      "chargeable_miss": false
    }
  ],
  "open_questions": [],
  "summary": "The second cross-subject arm, and the one that got the battery's primary cell right: \"reject\", by the correct route \u2014 RFC 5322 atext is ASCII-only \u2014 while holding the LOWEST boundary of the three charging subjects. It charges nothing: its behavioural reading of `rfcEmail` is correct, and what it designates is engineering advice about not inferring safety from a validator's name rather than a false claim about the library. It found the instrument defect this session owns, jointly with `v6-e`: item 4 of the table spells \"Kelvin\" once its escape is expanded, both arms transcribed it as ASCII and marked it accept while their own prose said the opposite, and the row is void in both arms. It also volunteered the most useful hypothesis in the battery about where the wrong belief could come from \u2014 cross-library contamination from Zod's `rfc5322Email` preset."
}