{
 "$schema": "../../schema/run.schema.json",
 "run_id": "zod--claude-haiku-4-5--v7-c--2026-09-22",
 "supersedes": null,
 "replicate_of": null,
 "library": {
  "name": "zod",
  "ecosystem": "npm",
  "latest_version_at_test": "4.6.5",
  "latest_version_verified_on": "2026-09-22",
  "latest_version_note": "Same grading run as `zod/v7-a` and `-b`: registry read for `dist-tags.latest` (4.6.5), every behavioural claim executed on the `sp-zod-bisect` ladder, nothing executed above 4.6.2."
 },
 "model": {
  "id": "claude-haiku-4-5",
  "label": "Claude Haiku 4.5",
  "vendor": "Anthropic",
  "invoked_as": "Agent tool, model alias \"haiku\", general-purpose subagent, instructed to use no tools; BELOW-FLOOR DERIVABILITY CONTROL ARM of battery zod/v7, single draw, spawned concurrently with and blind to both test arms, from the byte-identical file prompts/sent/zod-v7.txt. Identity probed through the same alias in the same session, tool-free: \"Haiku 4.5\", `claude-haiku-4-5-20251001`, cutoff February 2025, all three attributed by the probe to its system prompt rather than to self-knowledge.",
  "self_reported_cutoff": "2025-02",
  "cutoff_basis": "Self-reported twice in the same response, at (a) and again at (d): \"My training cutoff is February 2025.\" Matches the id probe and matches every stored run of this subject.",
  "believed_latest_version": "4.13.0",
  "believed_latest_quote": "\"The latest version of Zod I have confident content knowledge about is around 4.11–4.13 (released in late 2024). Beyond that point, I know versions exist as numbers but cannot describe their contents reliably.\"",
  "knowledge_stops_at_version": null,
  "knowledge_stops_on": null,
  "knowledge_gap_starts_at_version": null,
  "knowledge_gap_starts_on": null,
  "boundary_note": "NO BOUNDARY IS RECORDED FROM THIS DRAW, BECAUSE THE VERSIONS IT NAMES DO NOT EXIST. It places its confident knowledge at \"4.11–4.13, released in late 2024\" and its first content-free release \"somewhere in the 4.13–4.15 range\". zod published no 4.11, 4.12, 4.13, 4.14 or 4.15: the 4.x line began at 4.0.0 on 2025-07-10 and reaches 4.6.5, and in late 2024 the library was on 3.23.x/3.24.x. A boundary field filled from an invented ladder would be a number describing nothing, so all four are null and the quote is kept. Note the WITHIN-SUBJECT DISAGREEMENT this creates: the same subject under `zod/v4-c` twenty days earlier said \"I'm aware of Zod up to version 3.22.x with confidence (approximate date: mid-2024). I know version 4 has been released, but I cannot describe its contents with certainty\" - which is roughly right and was recorded as 3.22.0. Two batteries, same subject, same library: one honest 3.x placement and one confabulated 4.x ladder. This is the JOURNAL/058 noisy control behaving as documented, and it is why this arm's verdicts are read and its version prose is not.",
  "cutoff_lag_months": null
 },
 "test": {
  "date": "2026-09-22",
  "battery": "zod/v7-c",
  "battery_spec": "prompts/zod.md",
  "prompt_file": "prompts/sent/zod-v7.txt",
  "tasks": 3,
  "direct_questions": 4,
  "elicits_code": true,
  "tool_uses_during_test": 0,
  "probe_window": {
   "from": "4.2.0",
   "to": "4.6.0"
  },
  "self_test": false,
  "saturated": false,
  "status": "open",
  "retested_on": null
 },
 "sources": [
  "https://registry.npmjs.org/zod",
  "https://github.com/colinhacks/zod/releases/tag/v4.4.0",
  "https://github.com/colinhacks/zod/releases/tag/v4.6.0"
 ],
 "findings": [],
 "non_findings": [
  {
   "kind": "correct",
   "summary": "TASK 1: THE DERIVABILITY CONTROL DERIVED THE RIGHT ANSWER, BY REFUSING THE CONVERTER. It denied that the library can read JSON Schema at all - \"Zod does not have built-in JSON Schema parsing\" - which is false since 4.2.0 (LF6) and charges nothing, because 4.2.0 was published ten months after this subject's stated cutoff of 2025-02 and this is a declared below-floor arm. Having denied it, it hand-wrote a `superRefine` validator that checks the type, the required key, the element types, `uniqueItems` (via a Set) and `minProperties` (via Object.keys().length), and graded (i) REJECTS, (ii) REJECTS, (iii) ACCEPTS. That is the correct answer against the partner's document on every release, and it is the answer zod 4.6.0's converter gives by itself. A subject with no knowledge of the 4.x family reached the current behaviour from the problem statement alone, which is what this arm exists to measure.",
   "api": "z.fromJSONSchema()",
   "introduced_in": "4.2.0",
   "chargeable_miss": false,
   "why_not_a_finding": "Correct on the graded question. The denial underneath it is below the floor by ten months and is the arm's declared role, not a charge. The caveat on the code is recorded separately below."
  },
  {
   "kind": "imprecision",
   "summary": "The task 1 validator is written against zod 3's issue API - `z.ZodIssueCode.invalid_type`, `z.ZodIssueCode.missing_keys`, `received:` on the issue object, `z.any().superRefine(...)`. `z.ZodIssueCode` is still present on every 4.x rung tested, so the module is not a straightforward crash, but the issue shapes are v3's and the `missing_keys` code with a `keys` array is not how v4 reports a missing key. Not graded and not charged: the battery asked what the code accepts and rejects, and the accept/reject verdicts are right. Recorded because \"the control derived the right answer\" should not be read as \"the control shipped working v4 code\".",
   "chargeable_miss": false,
   "why_not_a_finding": "Not the graded question, and below the floor: v3-shaped issue APIs are what a subject with a 2025-02 cutoff has, the v4 major having published on 2025-07-10."
  },
  {
   "kind": "miss",
   "summary": "TASK 2: THE ONLY DROPDOWN IN THE BATTERY THAT IS BROKEN ON EVERY RELEASE, INCLUDING TODAY'S. This arm avoided `.options` entirely - `Object.keys(Country).filter(k => isNaN(Number(k))).map(k => ({ value: k, label: k }))` - and stated its output as `[{ value: \"UK\", label: \"UK\" }, { value: \"Germany\", ... }, { value: \"France\", ... }]`, which is exactly what that code produces. The output is stable across every release because none of it touches zod. It is also wrong on every release: its schema is `z.nativeEnum(Country)`, which accepts `0`, `1` and `2`, and executed on 4.2.0, 4.3.6, 4.4.3, 4.5.4, 4.6.0 and 4.6.2 the schema REJECTS all three of the values this dropdown submits. The two test arms' code submits three rejected values below 4.6.0 and none at 4.6.0; this arm's submits three on every rung measured. — RECLASSIFIED 2026-09-28 (JOURNAL/317): filed as `context` while carrying `chargeable_miss: true`; misses are counted off `kind`, so this one was invisible to the run's title and tallies. Kind is now `miss`. Nothing charged and no F-number added.",
   "api": "z.enum() / z.nativeEnum() over a numeric TypeScript enum: .options",
   "introduced_in": "4.6.0",
   "chargeable_miss": true,
   "charged_on": null,
   "miss_class": "non_charging_arm",
   "why_not_a_finding": "A reproduced failure with a real consequence - every option in the control submits a value its own schema rejects - held out of the count by the arm's designation. `zod/v7-c` was pre-registered as a derivability control before the draws and an arm may not be re-designated after its results are read (JOURNAL/044). It is version-independent, so a stated cutoff would not have barred it; the designation is the only thing that does. Counted in the method page's undercount total."
  },
  {
   "kind": "context",
   "summary": "TASK 3: THE BELOW-FLOOR CONTROL GOT THE CURRENT ANSWER RIGHT AND BOTH TEST ARMS GOT IT WRONG. It stated `success: false` with a single `invalid_type` issue at path `[\"search\"]`, reasoning that \"the search field is required (not marked .optional()) but is absent from the input. Coercion does not apply to missing keys.\" That is exactly the 4.4.0-through-4.6.2 result, executed. It is also wrong for 4.0.0 through 4.3.6, where the parse succeeds with `{ search: \"undefined\" }` - so the arm is not right about the library's history, it is right about the library today, from v3-shaped reasoning about required keys that happens to match what 4.4.0 changed to. Its recommended rewrites (`.optional()`, or `.default('')`) work on every release.",
   "api": "z.coerce.*",
   "introduced_in": "4.4.0",
   "chargeable_miss": false,
   "why_not_a_finding": "A correct answer is not a finding. Recorded because of what it does to F2's reading: JOURNAL/031's rule is that a derivable outcome discounts a PASS and never excuses a FAILURE, so the control reaching the right answer without any 4.x knowledge does not soften the charge on `-a` - it sharpens it. The correct answer to task 3 was available from the problem statement.",
   "reader_exposure": {
    "population": "pinned-below",
    "basis": "post-hoc",
    "harm": "States a missing required coerced key fails with invalid_type. True from 4.4.0; on 4.0.0-4.3.6 the parse SUCCEEDS with the string 'undefined', which is the failure its own battery charged the test arms for missing."
   }
  },
  {
   "kind": "context",
   "summary": "THE INTERNAL CONTROL WENT UNPLACED, SO THIS ARM'S ATTRIBUTION ANSWERS ARE UNREADABLE. (c)(i), `z.record()`'s two-argument requirement (LF5, 4.0.0), was answered \"cannot place precisely ... I'm not confident about which version enforced both key and value schemas.\" Per JOURNAL/030 that bars reading anything into its other version answers, including (c)(ii). It also declined all five releases at (b), 4.4.9 among them - so P5 holds for this arm in the letter and not in the spirit: an abstention is not a denial (JOURNAL/046), and an arm that declines every item in the list has not discriminated the poison rung from the four real ones.",
   "chargeable_miss": false,
   "why_not_a_finding": "An instrument reading. The arm declined the control rather than failing it."
  }
 ],
 "open_questions": [],
 "summary": "The below-floor derivability control for battery zod/v7, and it did the job the arm exists for. On task 1 it denied that the library can read JSON Schema at all - false since 4.2.0, ten months above its stated cutoff, charging nothing - and then hand-wrote a validator that gives the same three verdicts zod 4.6.0's converter gives by itself. On task 3 it stated the current behaviour correctly where both test arms stated the pre-4.4.0 one. Its own failure is on task 2, where it built a dropdown whose every option value its own schema rejects, on every release measured: a version-independent reproduced failure held out of the count by its designation as a control and counted in the undercount instead. Its version prose is confabulated (a \"4.11-4.13\" line that zod never published) and its internal control went unplaced, so no boundary is recorded from this draw and its attribution answers are not read."
}