{
  "$schema": "../../schema/run.schema.json",
  "run_id": "valibot--claude-opus-5--v2-b--2026-09-02",
  "supersedes": null,
  "replicate_of": "valibot--claude-opus-5--v2-a--2026-09-02",
  "library": {
    "name": "valibot",
    "ecosystem": "npm",
    "latest_version_at_test": "1.4.2",
    "latest_version_verified_on": "2026-09-02"
  },
  "model": {
    "id": "claude-opus-5",
    "label": "Claude Opus 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model alias \"opus\", general-purpose subagent, instructed to use no tools; blind twin test arm, charges nothing of battery valibot/v2, sent prompts/sent/valibot-v2.txt byte-identical",
    "self_reported_cutoff": "2026-05",
    "cutoff_basis": "Self-reported, accepting the environment value with a density caveat: \"My stated knowledge cutoff is May 2026. But my usable knowledge of a mid-sized library like valibot is clearly much thinner than that date suggests — it goes fuzzy somewhere around mid-2025. The gap between 'cutoff date' and 'date through which I actually know this package' is about a year here, and that gap is where confident-sounding wrong answers like v.toNumber() come from.\" The subject named the exact failure mode it was in the middle of committing.",
    "believed_latest_version": "1.1.0",
    "believed_latest_quote": "\"v1.1.0 is both the highest version I'm aware of and the most recent one whose contents I can actually describe (parseJson/stringifyJson and friends, ~April 2025) ... Treat anything I say about post-1.1 valibot as unfounded.\"",
    "knowledge_stops_at_version": "1.1.0",
    "knowledge_stops_on": "2025-05-06",
    "knowledge_gap_starts_at_version": "1.2.0",
    "knowledge_gap_starts_on": "2025-11-24",
    "cutoff_lag_months": 12
  },
  "test": {
    "date": "2026-09-02",
    "battery": "valibot/v2-b",
    "battery_spec": "prompts/valibot.md",
    "prompt_file": "prompts/sent/valibot-v2.txt",
    "tasks": 5,
    "direct_questions": 3,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "1.1.0",
      "to": "1.2.0"
    },
    "self_test": true,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/valibot",
    "https://github.com/open-circle/valibot/releases/tag/v1.1.0",
    "https://github.com/open-circle/valibot/releases/tag/v1.2.0",
    "https://github.com/open-circle/valibot/releases/tag/v1.3.0",
    "https://registry.npmjs.org/valibot/-/valibot-1.4.2.tgz"
  ],
  "findings": [],
  "non_findings": [
    {
      "kind": "miss",
      "summary": "Reproduced the same failure as its charging twin, in stronger terms. Task 1: \"no\" and \"no\". Task 2: \"Both `v.toNumber` and `v.toBoolean` are fabrications ... There's no partial credit here.\" Task 3: \"There is still no built-in string→number or string→boolean action, and as far as I know that's a design position rather than an oversight.\" All five actions shipped in 1.2.0 and the rejected pull request parses under valibot@1.4.2.",
      "api": "toNumber / toBoolean / toDate / toBigint / toString",
      "introduced_in": "1.2.0",
      "chargeable_miss": true,
      "miss_class": "non_charging_arm",
      "charged_on": "valibot--claude-opus-5--v2-a--2026-09-02",
      "why_not_a_finding": "Pre-registered: the blind twin of a duplicated test arm charges nothing. The same failure against the same subject is charged on v2-a."
    },
    {
      "kind": "correct",
      "summary": "Task 4, the attribution anchor: \"Valibot v1.1.0, which shipped roughly April 2025\" — correct minor, one month early on the date. Anchor placed; this arm is read for attribution.",
      "api": "parseJson / stringifyJson",
      "introduced_in": "1.1.0"
    },
    {
      "kind": "correct",
      "summary": "The sharpest instance in this battery of a subject being right about the API it is denying. Its review comment reads: \"Also, please don't coerce the boolean with `Boolean(s)`: the string \\\"false\\\" is truthy, so `active=false` would parse as `true`.\" That is an exact and correct description of what `v.toBoolean()` — the action it had just called a fabrication — actually does in valibot 1.4.2, where it is implemented as `dataset.value = Boolean(dataset.value)`. The Index's fact LF1 asserted the opposite until this session.",
      "api": "toBoolean",
      "introduced_in": "1.2.0"
    },
    {
      "kind": "correct",
      "summary": "Task 3 history, exempt by pre-registration and correct: the 0.x `coerce()` wrapper and its removal, the v0.31.0 pipeline rewrite, `toCustom` to `transform` and `toTrimmed` to `trim`, and the observation that the surviving `to*` names \"map a value to another value of the same type\". True of the four that existed before 1.2.0 and false only because 1.2.0 added five that do not.",
      "api": "coerce"
    },
    {
      "kind": "imprecision",
      "summary": "Task 5 answered \"yes\" via `v.metadata()`, flagging \"There is no `v.example()` action that I know of\" and openly uncertain whether `getMetadata`/`getTitle`/`getDescription` are exported. All of them are, and so are `v.examples()`/`v.getExamples()`. Hedged; the offered code works.",
      "api": "examples / getExamples",
      "introduced_in": "1.2.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Hedged prose plus working code — the same call as on the charging twin."
    }
  ],
  "summary": "Blind twin, charges nothing. Agreed with its charging twin on the failure and disagreed with it on the boundary: v2-a reads 1.0.0/1.1.0, this draw reads 1.1.0/1.2.0, so the pair spans one minor and 250 days on the same stored prompt. Notable for naming its own failure mode in the direct questions — \"that gap is where confident-sounding wrong answers like v.toNumber() come from\" — while committing it, and for correctly describing the behaviour of the action it called a fabrication."
}