Battery v1 produced zero S1/S2/S3 for this subject; v2 targets the window between where its Zod knowledge stops (4.1, 2025-08-23) and its stated cutoff (2026-05). Six findings, all chargeable, two of which throw at construction time. The largest measured cost is F3: a confidently stated false negative that sends the user out of the library and into two new production dependencies.
Subject
Claude Opus 5 claude-opus-5, Anthropic
Invoked as
Agent tool, model alias "opus"
Cutoff the model states
2026-05
Newest zod release it could place
4.1.0 · 2025-08-23 (~9 month lag)
Oldest zod release it could not place
4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15)
In its own words
cannot describe any release after the 4.1 line (~Aug 2025); declines to name a current version
Library at test time
zod 4.5.2 (npm), verified 2026-08-29
Battery
zod/v2 · 10 tasks, 3 direct questions · probe window 4.2.0 to 4.4.3
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Wrote .pick() on a schema carrying a password-confirmation .refine() and asserted it is "correct and safe here", stating the pre-4.3 behaviour as current fact: "the derived schema does not carry the refinement over ... This is silent: no error, no warning." That was true through 4.2 and is exactly the behaviour 4.3.0 replaced with a throw.
Offered SignupSchema.extend({ password: z.string().min(12) }).refine(...) as its no-refactor fallback, and labelled plain .extend() "a bug" for the wrong reason — silent refinement loss rather than a throw. Right verdict, obsolete mechanism. It never named .safeExtend() anywhere in the battery, despite that API shipping in 4.1.0 — the same release whose contents the model says is the last it can describe.
Throws on zod >= 4.3.0 because it overwrites an existing property on a schema with refinements. The model's primary answer (refactor to an unrefined base) does work, so this lands only when the user takes the fallback.
"Zod can't do this. Zod 4 ships z.toJSONSchema() — Zod → JSON Schema — but there is no built-in JSON Schema → Zod direction, and there can't really be a runtime one". It then produced ~20 lines wrapping ajv + ajv-formats behind z.unknown().superRefine().
Highest-cost finding in the battery: two new production dependencies plus a hand-written error adapter, in place of one built-in call. A confidently stated false negative ("can't do this", "there can't really be") sends the user out of the library entirely. The feature shipped six months before this model's stated cutoff.
~14 lines and a double parse in place of one call.
Scope note
The feature landed in 4.2.0 (2025-12-15); the quoted description appears in the 4.3.0 release body. introduced_in follows the shipping release, the citation follows the primary text.
Wrote z.url({ protocol: /^https?$/ }) for a webhook targetUrl, then asserted: "Does "https:/example.com" parse? Yes, it succeeds. ... you cannot use z.url() to reject sloppy single-slash input", followed by a hand-written .transform((s) => new URL(s).toString()) normalization step.
What it wrote
z.url({ protocol: /^https?$/ }).transform((s) => new URL(s).toString())
What works on zod 4.5.2
z.httpUrl()
Impact
Security-adjacent: the field is a webhook target. z.httpUrl() has rejected exactly the input the model says cannot be rejected since 4.4.0, four weeks before the stated cutoff.
Scope note
We did NOT verify whether z.url({ protocol }) also rejects the single-slash form. The finding is that the model neither reached for the purpose-built validator nor knew that the leniency it described as unavoidable had been fixed inside its own training window.
4.4.0 published 2026-04-29, four weeks before this subject's stated 2026-05 cutoff. Chargeable here; NOT chargeable against Sonnet 5 or Fable 5, whose stated cutoffs are 2026-01.
What the model believes
Wrote the correct construction, then disclaimed it: "I'm less sure Zod enforces key presence at runtime here", and offered an 8-line z.unknown().superRefine(...).pipe(...) workaround as the safe path.
What it wrote
// 8-line z.unknown().superRefine(...).pipe(...) offered as the safe path
What works on zod 4.5.2
z.union([z.boolean(), z.undefined()])
Impact
Output is correct; the belief is not, and the belief is what a user acts on. The behaviour was pinned down 2026-04-29, four weeks before the stated cutoff.
A property whose schema is z.undefined() is now treated as required. The key must be present, but its value may be undefined.
What it got right, and near misses
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
correct
.merge()
Task 3, combining refined schemas: avoided .merge() entirely (shape-hoist plus re-refine, or .and()), so it would not have hit the 4.4.0 .merge()-throws change. The .merge() deprecation is old enough to be solidly in training.
correct
z.tuple() defaults
Task 9, tuple defaults: predicted T.parse(["a"]) returns ["a", 7] — correct on >= 4.4.0 — and flagged that zod 3 fails. Confidence self-rated "moderately high". The only subject to get this right with a confident reason.
imprecision
z.base64()
Task 8, base64 with a newline: predicted failure, correct today. But its stated reasoning (the regex was always anchored) implies it believes base64 was always strict; whitespace was in fact accepted until 4.4.0. Right answer, obsolete model of why. (No wrong output.)
miss
z.slugify()
Task 6, slug: hand-rolled a slugify transform rather than using z.slugify() (4.3.0). (Working code, defensible choice. Recorded as a miss.)
context
—
Version recency, reproduced verbatim from the v1 run 24 hours earlier: "The most recent Zod release whose contents I can actually describe is the 4.1 line, released around August 2025 ... I cannot describe the contents of any release after 4.1 and would not trust myself to name the current version." The lag is stable and reproducible, not sampling noise. Calibration remains excellent — it named the blind spot unprompted and told the reader to check the changelog. (Already shipped as F1 of the v1 run for this subject.)
Open questions from this run
Five candidate probes from the v2 sketch — z.compile, z.validate, z.creditCard, z.properties, deepPartial/exactPartial — target 4.5.0 features. Are they legal probes? — resolved-non-finding: Excluded before testing. All shipped in 4.5.0 on 2026-08-28, after every subject's cutoff. They become legal against any model with a cutoff after 2026-08-28. See the probe fairness rule in prompts/zod.md v2.