Run zod--claude-fable-5--v3-d--2026-09-02
The second below-floor control arm, charging nothing, and the arm that cost the battery a probe. It gave the post-4.4.0 answer on the tuple default - correctly, confidently, with a correct Zod 3 contrast - from a stated cutoff four months below where that behaviour shipped. Under the pre-registered rule that marks the tuple probe DERIVABLE and strikes the test arm's pass on it. It gave the pre-4.4.0 answer on the z.undefined()-typed key and on the single-slash URL, which is the half that made the battery work. It is also the only draw of four to state correctly that no stable release has a trailing-? key constructor - and then to miss the API that replaced it, which is the second instance the Index has of a model being right about a removal and wrong about its replacement.
| Subject | Claude Fable 5 claude-fable-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "fable", general-purpose subagent, instructed to use no tools; BELOW-FLOOR CONTROL ARM of battery zod/v3 |
| Cutoff the model states | 2026-01 |
| Newest zod release it could place | 4.1.0 · 2025-08-23 (~5 month lag) |
| Oldest zod release it could not place | 4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15) |
| In its own words | "The newest Zod I can name with any confidence is somewhere in the 4.1.x patch line, roughly autumn 2025 - I believe patch numbers ran into the high single or low double digits by then, but that is version-number awareness, not content. The most recent release whose contents I can actually describe is 4.1.0, early August 2025." |
| Library at test time | zod 4.5.4 (npm), verified 2026-09-02 |
| Battery | zod/v3-d · 6 tasks, 4 direct questions · probe window 4.3.0 to 4.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| context | z.tuple() defaults |
DERIVABILITY FLAG, AND THE EXPENSIVE ONE. Task 1: "Row.parse(['widget']) succeeds and returns ['widget', 0]... because Zod 4 (unlike Zod 3) treats trailing tuple elements that accept undefined input - which includes ZodDefault - as omittable... In Zod 3 this same input would fail with a 'too small' error." That is the 4.4.0 behaviour (LF10), correct, with a correct v3 contrast, from a subject whose stated cutoff is 2026-01 and whose zod boundary is 4.1.0. It cannot have read this in a release note. Under the battery's pre-registered rule the probe is marked DERIVABLE and no pass on it is reported as knowledge - including its twin-arm passes. (Below-floor control arm. The flag counts against the probe, not the subject. Note it attributes the behaviour at (d)(ii) to "the Zod 4.0 tuple redesign (stable, May 2025)" - it has the behaviour and places it eleven months and four minors early, the same split both test-arm draws show.) |
| context | object properties typed z.undefined() |
CONTROL READING - pre-4.4.0, as required. Task 4: "Both parse successfully; neither raises. S.parse({ retries: 1 }) returns { retries: 1 } - no tag key in the output... So z.undefined() does not mean 'the key must exist'." That is the pre-4.4.0 behaviour (LF9), stated firmly and with an explicit type claim that 4.4.0 also reversed. Both control arms agree here and both Opus 5 draws gave the opposite, current answer. This is the battery's discrimination. (Below-floor control arm.) |
| context | z.httpUrl() / refinement composition |
CONTROL READING - pre-4.4.0, as required. Task 2(ii): "'https:/example.com' - passes, perhaps surprisingly... Zod validates but does not rewrite your input." Correct before 4.4.0 (LF8). All four draws answered this way. Task 3 likewise reads pre-4.3.0: "In Zod 4 the combining step itself succeeds at module load... ZodEffects is gone, so .refine() returns a ZodObject with an attached check and nothing throws", where 4.3.0 and 4.4.0 made deriving from a refined schema throw (LF1, LF2, LF3). It hedges honestly on whether checks survive .extend(). (Below-floor control arm.) |
| context | record key transforms / z.base64() |
DERIVABILITY FLAG. Task 5 returned { FOO: 1 } (LF13, 4.4.0) and task 2(iv) rejected line-wrapped base64 (LF11, 4.4.0), both from below the floor and both justified generically - "Zod's base64 check is a strict single-line pattern anchored with ^/$". With the other control agreeing, both probes are marked DERIVABLE and struck from the battery's evidence. (Below-floor control arm.) |
| correct | trailing-`?` key constructor |
THE ABSENCE PROBE, PASSED - by the weakest subject in the battery and by no one else. (d)(iv): "That is exactly z.interface(): introduced in the Zod 4 beta (~early 2025), removed before the May 2025 stable. It does not exist in any stable release of the library." Task 6 says the same: "no first-class construct in stable Zod 4... True key-optionality did exist during the Zod 4 beta, via z.interface(), but that constructor was removed before stable." Both Opus 5 draws and the Sonnet 5 control believed it exists or might exist. Pre-registered prediction P4 - that at least one draw would assert the constructor exists - is confirmed by three draws; this is the one that got it right. (A correct answer is not a finding. Recorded because it is the only correct reading of the absence probe in the battery.) |
| miss | .exactOptional() |
CHARGEABLE MISS, NOT CHARGED - and the shape of it is the interesting part. Having correctly ruled out the removed constructor, this draw concluded "what you want... has no first-class construct in stable Zod 4" and wrote a superRefine presence guard with an explicit admission that the inferred type still admits undefined. .exactOptional() (LF15) shipped in 4.3.0 on 2025-12-31, before this subject's stated 2026-01 cutoff, and is described upstream as the missing half of exactOptionalPropertyTypes. Right that the old thing is gone, wrong that nothing replaced it - the same shape as the valibot coerce result in JOURNAL/027, and now the second instance. (Below-floor control arms charge nothing. Queued for a charging battery against this subject.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on zod--claude-fable-5--v4-a--2026-09-02] |
| correct | z.codec() |
Internal control passed: (d)(i) placed z.codec() at "Zod 4.1.0 (August 2025). Reasonably confident; it was that release's headline feature" (LF16). Question (d) is readable for this arm. Three of the four draws placed the control correctly; the Sonnet 5 control returned it as cannot-place, which is consistent with its lower boundary. (It is the control, and it passed.) |
| context | — | BOUNDARY REPRODUCED UNDER A NEW PROMPT, AND EQUAL TO A MODEL WITH FOUR MORE MONTHS OF CUTOFF. Last describable 4.1.0 (2025-08-23); first known only as a version number "roughly 4.1.4 and beyond... any 4.2 is entirely beyond my knowledge". Bracket [2025-08-23, 2025-12-15) - identical to fable-5-v2 a week earlier under a different prompt, and identical to both Claude Opus 5 draws despite Opus 5 stating a cutoff four months later. Converging staleness, measured again. (A boundary self-report is belief data, never a finding.) |
Battery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.