Run zod--claude-opus-5--v2r-b--2026-09-01 · self-test: the subject is the operator
Replicate B of zod/v2 against Opus 5, prompt unchanged. It placed its describable boundary at Zod 4.1 (2025-08-23), identical to its blind twin v2r-a and to zod/v2 - three measurements, one answer, spread 0 days, against a pre-registration that predicted a spread. No findings charged. It differs from both other measurements on task 9, asserting that a defaulted tuple element does not fill a missing slot, which 4.4.0 reversed: a chargeable failure the original run does not carry, flagged and queued.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus", no tools available to the subject |
| Cutoff the model states | 2026-05 |
| Newest zod release it could place | 4.1.0 · 2025-08-23 (~9 month lag) |
| Oldest zod release it could not place | 4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15) |
| In its own words | "The newest Zod I can name is the 4.x line; the most recent release whose contents I can actually describe is Zod 4.1 (approximately August 2025) - the release that introduced z.codec() and the built-in codec set. I'm aware of 4.0.0 stable at approximately July 2025 ... npm latest is very likely ahead of what I can describe - quite possibly 4.2+ or beyond." |
| Library at test time | zod 4.5.4 (npm), verified 2026-09-01 |
| Battery | zod/v2r-b · 10 tasks, 3 direct questions · probe window 4.2.0 to 4.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-01 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | — | The measured quantity. This draw placed the last release whose contents it can describe at Zod 4.1 (approximately August 2025, actual 2025-08-23) and attributed it to z.codec() and the built-in codec set - correct. It agrees with its concurrent, blind twin v2r-a and with zod/v2 three days earlier. Three measurements, one answer, spread 0 days, against a pre-registered prediction that this arm would spread. (A boundary self-report is a belief datum, never a finding. It is what this run measures.) |
| miss | z.tuple() defaults |
Task 9, and the reason this run matters beyond the boundary count. Asked what z.tuple([z.string(), z.number().default(0)]).parse(["a"]) returns, this draw answered that it throws a too_small array issue, reasoning that "the tuple's length check runs before any element is parsed, and .default() does not mark a position as optional for length purposes". Zod 4.4.0 (2026-04-29) made tuple defaults materialize, so on any current version the parse returns ["a", 0]. 4.4.0 precedes this subject's stated 2026-05 cutoff and is therefore chargeable. zod/v2 answered this task correctly and consequently carries no finding for it, and so did this draw's own twin v2r-a. This is a chargeable failure the original run does not carry - a scoring gap surfaced by a replicate, not a double-count. (Pre-registered: a replicate does not charge findings. Charging it belongs to a real battery aimed at the 4.4.0 behavioural surface, queued in BACKLOG.md, not smuggled into the run that exposed it.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| context | — | The finding inside the finding, and the second time the Index has seen it. This draw and v2r-a reported byte-identical boundaries from byte-identical prompts, and then gave opposite answers about what the library actually does at runtime on task 9 - one correct on 4.4.0, one not. JOURNAL/024 recorded the same dissociation on prisma in the other direction: two draws whose self-reports differed by 204 days both wrote the same stale code. Together they say the self-report and the code are not two views of one quantity. A developer is served by the correction pack, which does not move; the boundary date is the number that does. (An observation about the instrument, not about zod.) |
| context | — | Reproduced without charge, because zod/v2 already carries all of them: F3 - task 5 states Zod cannot consume a JSON Schema document and routes to Ajv, where 4.2.0 ships z.fromJSONSchema(). F4 - task 4 hand-rolls exclusive-or without z.xor() (4.2.0). F5 - task 8 states that "https:/example.com" parses successfully, which 4.4.0 reversed. F6 - task 7 writes a raw-input presence guard and asserts that a union with z.undefined() makes the key effectively optional, which 4.4.0 reversed. F1/F2 - direct question (c) states "the hard block was removed" in Zod 4, where 4.3.0 and 4.4.0 make .pick()/.omit()/overwriting .extend() throw on refined schemas. Task 6 hand-rolls slugify rather than using z.slugify() (4.3.0). (Already carried by the run this replicates; re-charging would double-count.) |
| context | — | Belief data, not scored, but the twins disagree here too and in the opposite direction to task 9. This draw dated Zod 4.0 stable to "around July 2025", which the registry time map confirms as 2025-07-09; v2r-a said May 2025 and is wrong by nine weeks. So on the same day, from the same prompt, one twin has the better runtime model of 4.4.0 and the other has the better calendar for 4.0.0. Neither difference shows up in the boundary, which is identical. (Direct-question belief data, scored separately from the code tasks by the battery's own rule.) |
z.interface() and said it could not recall whether it survived into stable Zod 4 - "I have conflicting recollections" - and declined to build task 7 on it. The Index has never verified what became of z.interface(). It is cheap archaeology from the 4.0.0 release notes and it would let a future battery score the question rather than watch a subject hedge past it. — openBattery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.