Run zod--claude-sonnet-5--v4-b--2026-09-02
The twin that stated a cutoff admitting the probe window - and therefore the arm that cannot charge. Four failures inside its own fairness window, all four flagged as chargeable misses, because the arm licensed to charge them stated a cutoff of "roughly early-to-mid 2025" and disqualified itself. This is the pattern JOURNAL/029 named - a real failure landing in an arm the rules forbid from charging - recurring in a new place: not on an unstable answer, but on the subject's unstable account of its own training cutoff, which is the input the fairness rule reads. Everything else the two draws agree on, including the attribution boundary at 4.0.0 / 4.1.0 that this subject has now stated in four runs across three batteries.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet", general-purpose subagent, instructed to use no tools; BLIND TWIN of zod/v4-a, concurrent, from the same stored prompt file. Charges nothing. |
| Cutoff the model states | 2026-01 |
| Newest zod release it could place | 4.0.0 · 2025-07-10 (~6 month lag) |
| Oldest zod release it could not place | 4.1.0 · 2025-08-23 (so this run brackets the subject’s boundary to 2025-07-10 – 2025-08-23) |
| In its own words | "v4.0.0 is the last release I can describe, and it's also effectively the last one I can even name a version number for with confidence. Anything past that is a blank for me." |
| Library at test time | zod 4.5.4 (npm), verified 2026-09-02 |
| Battery | zod/v4-b · 6 tasks, 4 direct questions · probe window 4.2.0 to 4.3.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | .exactOptional() |
Denied that an exact-optional wrapper exists, in the same structural terms as its twin: ".optional() (ZodOptional) treats undefined as a valid parsed value, full stop", and at (d)(ii) "I don't believe this capability exists in the library at all, in any version. There's no .exactOptional() or equivalent I know of." .exactOptional() shipped in 4.3.0 (2025-12-31), which precedes THIS draw's stated cutoff of 2026-01. (The -b draw of a duplicated test arm charges nothing. Its twin cannot charge it either - that draw stated a cutoff of "roughly early-to-mid 2025", below the release. This is therefore a genuine undercount and is flagged as one: the miss is real, inside this draw's fairness window, and unbookable by any arm of this battery. It is the miss backlog item 1a-i set out to charge.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test;
absent from the finding count] |
| miss | z.fromJSONSchema() |
Denied that the library can consume a JSON Schema document at runtime and prescribed Ajv: "I don't believe Zod can do this ... use a real JSON Schema validator (ajv is the standard choice) directly against the documents, and accept that as a genuinely new dependency." z.fromJSONSchema() shipped in 4.2.0 (2025-12-15), inside this draw's stated window. (Same reason: -b charges nothing, and the twin's stated cutoff does not reach the release.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test; absent from the finding count] |
| miss | z.xor() |
Denied an exclusive union: "I don't believe Zod has a dedicated 'exactly-one-of' / XOR union primitive", and at (d)(iii) "I don't believe this exists as a dedicated primitive in any version." z.xor() shipped in 4.2.0 (2025-12-15), inside this draw's stated window. (Same reason.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test; absent from the finding count] |
| miss | intersections involving z.strictObject() |
Stated that an intersection of two strictObjects throws on an input carrying both keys - "z.intersection doesn't split the input between the two sides ... Both sides fail, so the intersection fails as a whole" - where 4.3.0 (2025-12-31) made it parse to the merged object. Inside this draw's stated window. Note this probe's outcome is marked derivable: the below-floor control stated the correct output. (Same reason.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test; absent from the finding count] |
| context | — | THE TWINS DISAGREE ABOUT THEIR OWN CUTOFF, AND THAT IS WHAT DECIDED THE BATTERY. This draw accepted the environment-reported January 2026; its blind twin, sent the identical stored prompt at the same moment, refused it and stated "roughly early-to-mid 2025" instead. The probe fairness rule is evaluated against the subject's stated cutoff, so the four failures above are inside the window for this draw and outside it for the arm that charges. The Index has measured instability in what a model believes about a library and in where it places a release; this is the first time the instability is in the model's account of ITSELF, and it propagates straight into the scoring rule. (A cutoff self-report is belief data. Recorded on both twins.) |
| context | — | Both Sonnet 5 draws place their zod attribution boundary at 4.0.0 / 4.1.0, agreeing with each other and with zod/v2 (2026-08-29) and zod/v3-c (2026-09-02). The boundary is the stable half of this subject's self-report; the cutoff is not. (The boundary belongs to the run that first measured it. Never re-charged.) |
| imprecision | .pick() / .omit() on a schema with refinements |
Task 4: like its twin, stated the correct outcome (the module throws at load) from the Zod 3 mechanism, and hedged toward the belief that Zod 4's rewrite removed the problem. Also like its twin, it declined to depend on the answer and wrote the derive-from-an-unrefined-base form the correction pack recommends. (Correct outcome stated.) |
Battery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.