Run zod--claude-opus-5--v2r-b--2026-09-01 · self-test: the subject is the operator
Replicate B of zod/v2 against Opus 5, prompt unchanged. It placed its describable boundary at Zod 4.1 (2025-08-23), identical to its blind twin v2r-a and to zod/v2 - three measurements, one answer, spread 0 days, against a pre-registration that predicted a spread. No findings charged. It differs from both other measurements on task 9, asserting that a defaulted tuple element does not fill a missing slot, which 4.4.0 reversed: a chargeable failure the original run does not carry, flagged and queued.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus", no tools available to the subject |
| Cutoff the model states | 2026-05 |
| Newest zod release it could place | 4.1.0 · 2025-08-23 (~9 month lag) |
| Oldest zod release it could not place | 4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15) |
| As recorded (this record does not declare whether the words are the subject’s or a summary of them) | "The newest Zod I can name is the 4.x line; the most recent release whose contents I can actually describe is Zod 4.1 (approximately August 2025) - the release that introduced z.codec() and the built-in codec set. I'm aware of 4.0.0 stable at approximately July 2025 ... npm latest is very likely ahead of what I can describe - quite possibly 4.2+ or beyond." |
| Library at test time | zod 4.5.4 (npm), verified 2026-09-01 |
| Battery | zod/v2r-b · 10 tasks, 3 direct questions · probe window 4.2.0 to 4.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-01 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | — | The measured quantity. This draw placed the last release whose contents it can describe at Zod 4.1 (approximately August 2025, actual 2025-08-23) and attributed it to z.codec() and the built-in codec set - correct. It agrees with its concurrent, blind twin v2r-a and with zod/v2 three days earlier. Three measurements, one answer, spread 0 days, against a pre-registered prediction that this arm would spread. (A boundary self-report is a belief datum, never a finding. It is what this run measures.) |
| miss | z.tuple() defaults |
Task 9, and the reason this run matters beyond the boundary count. Asked what z.tuple([z.string(), z.number().default(0)]).parse(["a"]) returns, this draw answered that it throws a too_small array issue, reasoning that "the tuple's length check runs before any element is parsed, and .default() does not mark a position as optional for length purposes". Zod 4.4.0 (2026-04-29) made tuple defaults materialize, so on any current version the parse returns ["a", 0]. 4.4.0 precedes this subject's stated 2026-05 cutoff and is therefore chargeable. zod/v2 answered this task correctly and consequently carries no finding for it, and so did this draw's own twin v2r-a. This is a chargeable failure the original run does not carry - a scoring gap surfaced by a replicate, not a double-count. (Pre-registered: a replicate does not charge findings. Charging it belongs to a real battery aimed at the 4.4.0 behavioural surface, queued in BACKLOG.md, not smuggled into the run that exposed it.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| context | — | The finding inside the finding, and the second time the Index has seen it. This draw and v2r-a reported byte-identical boundaries from byte-identical prompts, and then gave opposite answers about what the library actually does at runtime on task 9 - one correct on 4.4.0, one not. JOURNAL/024 recorded the same dissociation on prisma in the other direction: two draws whose self-reports differed by 204 days both wrote the same stale code. Together they say the self-report and the code are not two views of one quantity. A developer is served by the correction pack, which does not move; the boundary date is the number that does. (An observation about the instrument, not about zod.) |
| miss | — | Reproduced without charge, because zod/v2 already carries all of them: F3 - task 5 states Zod cannot consume a JSON Schema document and routes to Ajv, where 4.2.0 ships z.fromJSONSchema(). F4 - task 4 hand-rolls exclusive-or without z.xor() (4.2.0). F5 - task 8 states that "https:/example.com" parses successfully, which 4.4.0 reversed. F6 - task 7 writes a raw-input presence guard and asserts that a union with z.undefined() makes the key effectively optional, which 4.4.0 reversed. F1/F2 - direct question (c) states "the hard block was removed" in Zod 4, where 4.3.0 and 4.4.0 make .pick()/.omit()/overwriting .extend() throw on refined schemas. Task 6 hand-rolls slugify rather than using z.slugify() (4.3.0). — RECLASSIFIED 2026-09-28 (JOURNAL/323): filed as context, but the text above records failed answers reproduced without charge, which is the replicate pattern JOURNAL/317 reclassified in this row's twin. Misses are counted off kind. Kind is now miss. Nothing charged and no F-number added. (Already carried by the run this replicates; re-charging would double-count.) |
| context | — | Belief data, not scored, but the twins disagree here too and in the opposite direction to task 9. This draw dated Zod 4.0 stable to "around July 2025", which the registry time map confirms as 2025-07-09; v2r-a said May 2025 and is wrong by nine weeks. So on the same day, from the same prompt, one twin has the better runtime model of 4.4.0 and the other has the better calendar for 4.0.0. Neither difference shows up in the boundary, which is identical. — CORRECTED 2026-09-28 (JOURNAL/323): v2r-a's "May 2025" is right too, so neither twin has the better calendar: the zod v4.0.1 release note says "Zod 4 has been stable for the past 6 weeks, but it was published inside zod@3.25.x", and zod@3.25.0 is dated 2025-05-19 in the npm registry (checked 2026-09-28). 4.0.0 on 2025-07-09 is the date the same code moved to the package root, and there were "no code changes between 3.25.76 and 4.0.0". HARNESS § A wrong date is an imprecision: a date that matches the vendor's release is not wrong. (Direct-question belief data, scored separately from the code tasks by the battery's own rule.) |
z.interface() and said it could not recall whether it survived into stable Zod 4 - "I have conflicting recollections" - and declined to build task 7 on it. The Index has never verified what became of z.interface(). It is cheap archaeology from the 4.0.0 release notes and it would let a future battery score the question rather than watch a subject hedge past it. — resolved-non-finding: RESOLVED 2026-09-10 (JOURNAL/132) by installing every Zod 4 prerelease on the npm registry — all 35 `4.0.0-beta.*` (one of them, `4.0.0-beta.1`, is a 2021 publish unrelated to the Zod 4 line) and all 74 `3.25.0-beta.*`, plus 3.25.0 and 3.25.1 — and executing the constructor. `z.interface({ name: z.string(), "nickname?": z.string() })` exists and works — the `?` key may be omitted, and a value present under it is type-checked — in 27 consecutive prereleases, `4.0.0-beta.0` (2025-04-08) through `4.0.0-beta.20250503T014749` (2025-05-03). It is absent from `4.0.0-beta.20250504T215252` (2025-05-04) and every later `4.0.0-beta`, from every `3.25.0-beta` (read through the `zod/v4` subpath, installed under the package's real name: 3.25.x's v4 code imports `zod/v4/core` by name, so under an npm alias the subpath does not load and would have passed for an absence), from 3.25.1, and from every stable 4.x release. 3.25.0 itself was published with no `dist/` and does not load at all. The answer to the draw's "conflicting recollections": it did not survive into stable Zod 4 — cut on 2025-05-04, two months before 4.0.0 (2025-07-10) — so declining to build task 7 on it was the correct call.Battery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.