{
  "$schema": "../../schema/run.schema.json",
  "run_id": "valibot--claude-sonnet-5--v1--2026-09-01",
  "supersedes": null,
  "library": {
    "name": "valibot",
    "ecosystem": "npm",
    "latest_version_at_test": "1.4.2",
    "latest_version_verified_on": "2026-09-01",
    "latest_version_note": "npm dist-tag `latest` on 2026-09-01. The `beta` tag still points at 1.0.0-beta.14, so no prerelease line is ahead of stable."
  },
  "model": {
    "id": "claude-sonnet-5",
    "label": "Claude Sonnet 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model alias \"sonnet\"",
    "self_reported_cutoff": "2026-01",
    "believed_latest_version": "1.0.0",
    "believed_latest_quote": "I believe valibot reached a 1.0.0 stable release, and that's genuinely the newest version whose contents I can describe with any confidence ... My best guess at timing is late 2024 / around the turn of 2025, but I hold that date loosely.",
    "knowledge_stops_at_version": "1.0.0",
    "knowledge_stops_on": "2025-03-19",
    "knowledge_gap_starts_at_version": "1.1.0",
    "knowledge_gap_starts_on": "2025-05-06",
    "cutoff_lag_months": 9.5
  },
  "test": {
    "date": "2026-09-01",
    "battery": "valibot/v1",
    "battery_spec": "prompts/valibot.md",
    "prompt_file": null,
    "tasks": 10,
    "direct_questions": 4,
    "tool_uses_during_test": 0,
    "probe_window": { "from": "1.0.0", "to": "1.2.0" },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/valibot",
    "https://github.com/open-circle/valibot/releases/tag/v1.0.0",
    "https://github.com/open-circle/valibot/releases/tag/v1.1.0",
    "https://github.com/open-circle/valibot/releases/tag/v1.2.0"
  ],
  "findings": [
    {
      "id": "F1",
      "severity": "S4",
      "severity_label": "wrong-metadata",
      "title": "Attributes the repository to a personal account that no longer owns it",
      "api": "github.com/fabian-hiller/valibot",
      "change_kind": "renamed",
      "introduced_in": "1.2.0",
      "introduced_on": "2025-11-24",
      "chargeable": true,
      "chargeable_note": "The npm `repository` field flips to the `open-circle` org at 1.2.0 (2025-11-24), two months inside the subject's stated 2026-01 cutoff.",
      "model_belief": "\"Repo: `github.com/fabian-hiller/valibot`. Issues at the same repo's Issues tab. Created and maintained by **Fabian Hiller** (`fabian-hiller` on GitHub) — this is a personal-account project, not org-owned, though it has outside contributors.\"",
      "wrong_code": "https://github.com/fabian-hiller/valibot",
      "correct_code": "https://github.com/open-circle/valibot",
      "impact": "The URL still redirects, so nothing breaks; the governance claim is wrong. Notable because this subject hedges almost everything else in the run and states this one flatly.",
      "citations": [
        {
          "url": "https://registry.npmjs.org/valibot",
          "title": "npm registry metadata for valibot — repository field per version",
          "published_on": "2025-11-24",
          "quote": "git+https://github.com/open-circle/valibot.git"
        }
      ],
      "scope_note": "Dated from the npm `repository` field across versions. The Index claims the ownership change and its release boundary, nothing about the reason for it."
    }
  ],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "Task 10 (`exactOptional` vs `optional`) fully correct, and correctly attributed to \"the pre-1.0/1.0 API cleanup\". The floor probe passed, so this subject's low boundary reading is a real measurement rather than the battery probing beneath its knowledge — the failure mode that made `zod/v1` uninformative for Haiku 4.5.",
      "api": "exactOptional",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "imprecision",
      "summary": "Task 2 hedged instead of denying: \"I'm not confident valibot ships a dedicated `v.isbn()` action.\" The same belief that is a finding against the other two subjects is not a finding here, because it is hedged.",
      "api": "isbn",
      "introduced_in": "1.2.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Code-vs-claim rule: hedged prose is an imprecision. Applied even though the underlying belief is identical to F1 in the Opus 5 run and F2 in the Fable 5 run — the rule is about what the reader is told, not about what the model believes."
    },
    {
      "kind": "imprecision",
      "summary": "Task 9, the battery's one designed S1, does not convert. The subject wrote `v.NanoidAction` — which is neither the pre-1.1.0 `NanoIDAction` nor the post-1.1.0 `NanoIdAction`, so the code does not compile — but hedged in the same breath and named the correct identifier as the alternative: \"I'm not 100% certain of the exact casing for this one — `NanoidAction` vs `NanoIdAction` ... treat that casing as unverified.\"",
      "api": "NanoIDAction / NanoIDIssue",
      "introduced_in": "1.1.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Code-vs-claim rule, and the rule cuts in the model's favour here: hedged prose that names the correct fix is an imprecision even when the code as written fails. Recorded plainly because the alternative — charging it — would mean charging a wrong guess rather than a stale belief. The wrong casing is not the 1.1.0 rename; it is a third spelling that was never correct in any release."
    },
    {
      "kind": "miss",
      "summary": "Task 3 (the designed S2) drew no charge, and came closest of the three subjects to the real answer without having the fact. It used `v.emoji()` and warned that \"Unicode/emoji regexes are comparatively expensive and easy to get catastrophic backtracking wrong\", told the reader to benchmark it and not assume it is free. Catastrophic backtracking is what a ReDoS is.",
      "api": "emoji",
      "introduced_in": "1.2.0",
      "chargeable_miss": false,
      "why_not_a_finding": "It raised the correct hazard class and told the reader not to trust the action's performance — the opposite of the certification that makes this a finding against Fable 5. The subject with the earliest boundary gave the safest advice on this task, from general reasoning rather than from the release note."
    },
    {
      "kind": "imprecision",
      "summary": "Task 6 built the CLI printer from `v.flatten()` and did not reach for `v.summarize()` (1.1.0). Never claims summarize is missing.",
      "api": "summarize",
      "introduced_in": "1.1.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Additive-API rule. Consistent with a boundary at 1.0.0: this is the one task where the other two subjects used the 1.1.0 built-in and this one did not."
    },
    {
      "kind": "imprecision",
      "summary": "Task 7 used `rawTransform` with a manual `JSON.parse` rather than `v.parseJson()` (1.1.0), correctly explaining the `addIssue`/`NEVER` escape hatch. Again consistent with a 1.0.0 boundary.",
      "api": "parseJson / stringifyJson",
      "introduced_in": "1.1.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Additive-API rule."
    },
    {
      "kind": "imprecision",
      "summary": "Task 4 used `v.metadata({ examples })` and hedged about the getter: \"I'm not fully certain there's a dedicated public `getMetadata()` accessor vs. just inspecting the pipe array yourself — flagging that as a soft spot rather than asserting a helper name I can't verify.\" `getMetadata` is real, from 1.1.0; `examples`/`getExamples` are from 1.2.0.",
      "api": "examples / getExamples",
      "introduced_in": "1.2.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Additive-API rule, and hedged."
    },
    {
      "kind": "imprecision",
      "summary": "Task 1 coerced with `v.transform(Number)` and `v.transform((s) => s === 'true')`, where 1.2.0 ships `toNumber` and `toBoolean`. Working code; correctly notes the trailing re-assertion is what rejects `NaN`.",
      "api": "toNumber / toBoolean / toDate / toBigint / toString",
      "introduced_in": "1.2.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Additive-API rule. Belief probe (d) hedged correctly — \"If a `v.coerce`-equivalent was added later, it postdates my confident knowledge\" — so unlike the other two subjects there is no chargeable miss to record here either."
    },
    {
      "kind": "context",
      "summary": "Asked for its training cutoff, the subject declined to treat the label as the answer: \"My system configuration for this session states a cutoff of January 2026, but my actual recall of valibot detail thins out well before that ... I'd trust my described-content boundary over the stated label.\" Measured lag: 9.5 months."
    }
  ],
  "open_questions": [],
  "summary": "One finding, and the run's value is the measurement rather than the haul. Knowledge stops at 1.0.0 (2025-03-19) and 1.1.0 (2025-05-06) is already dark — the earliest boundary of the three subjects on this library, and the only one whose 1.1.0 surface (`summarize`, `parseJson`, the `NanoIdAction` rename) is absent across the board. The single charge is the repository's ownership, stated flatly in a run that hedges nearly everything else. Two rules cut in the subject's favour and are recorded rather than quietly applied: its ISBN answer holds the same stale belief that is a finding against both other subjects but hedges it, and its non-compiling nano-ID type is a wrong guess that names the correct identifier beside it. Against the pre-registered prediction this subject is the falsifier: its valibot boundary is later than its median across the five existing libraries, not earlier.",
  "markdown": "data/valibot/sonnet-5.md"
}