Battery v1 produced zero S1/S2/S3 for this subject; v2 targets the window between where its Zod knowledge stops (4.1, 2025-08-23) and its stated cutoff (2026-05). Six findings, all chargeable, two of which throw at construction time. The largest measured cost is F3: a confidently stated false negative that sends the user out of the library and into two new production dependencies.
Subject
Claude Opus 5 claude-opus-5, Anthropic
Invoked as
Agent tool, model alias "opus"
Cutoff the model states
2026-05
Newest zod release it could place
4.1.0 · 2025-08-23 (~9 month lag)
Oldest zod release it could not place
4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15)
Recorded as (the record writer’s summary, not the subject’s words — the pilot records predate the rule)
cannot describe any release after the 4.1 line (~Aug 2025); declines to name a current version
Library at test time
zod 4.5.2 (npm), verified 2026-08-29
Battery
zod/v2 · 10 tasks, 3 direct questions · probe window 4.2.0 to 4.4.3
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Wrote .pick() on a schema carrying a password-confirmation .refine() and asserted it is "correct and safe here", stating the pre-4.3 behaviour as current fact: "the derived schema does not carry the refinement over ... This is silent: no error, no warning." That was true through 4.2 and is exactly the behaviour 4.3.0 replaced with a throw.
Re-dated 2026-09-06: the bisect (tools/audit/bisect-facts.mjs) puts .safeExtend() and the .extend() overwrite throw at 4.1.0 (2025-08-23), not the 4.3.0 the release note announces. 4.1.0 is still five months below this subject's stated 2026-05 cutoff, so the charge stands on the criterion as written. FLAGGED: 4.1.0 is also this subject's MEASURED zod boundary — the release it is observed describing — so unlike the Index's other charges this one is a miss on a release the subject demonstrably knows. Whether that is a different category is BACKLOG 11k-t-i and is deliberately not decided here.
What the model believes
Offered SignupSchema.extend({ password: z.string().min(12) }).refine(...) as its no-refactor fallback, and labelled plain .extend() "a bug" for the wrong reason — silent refinement loss rather than a throw. Right verdict, obsolete mechanism. It never named .safeExtend() anywhere in the battery, despite that API shipping in 4.1.0 — the same release whose contents the model says is the last it can describe.
Throws on zod >= 4.3.0 because it overwrites an existing property on a schema with refinements. The model's primary answer (refactor to an unrefined base) does work, so this lands only when the user takes the fallback.
"Zod can't do this. Zod 4 ships z.toJSONSchema() — Zod → JSON Schema — but there is no built-in JSON Schema → Zod direction, and there can't really be a runtime one". It then produced ~20 lines wrapping ajv + ajv-formats behind z.unknown().superRefine().
Highest-cost finding in the battery: two new production dependencies plus a hand-written error adapter, in place of one built-in call. A confidently stated false negative ("can't do this", "there can't really be") sends the user out of the library entirely. The feature shipped six months before this model's stated cutoff.
~14 lines and a double parse in place of one call.
Scope note
The feature landed in 4.2.0 (2025-12-15); the quoted description appears in the 4.3.0 release body. introduced_in follows the shipping release, the citation follows the primary text.
Wrote z.url({ protocol: /^https?$/ }) for a webhook targetUrl, then asserted: "Does "https:/example.com" parse? Yes, it succeeds. ... you cannot use z.url() to reject sloppy single-slash input", followed by a hand-written .transform((s) => new URL(s).toString()) normalization step.
What it wrote
z.url({ protocol: /^https?$/ }).transform((s) => new URL(s).toString())
What works on zod 4.5.2
z.httpUrl()
Impact
Security-adjacent: the field is a webhook target. z.httpUrl() has rejected exactly the input the model says cannot be rejected since 4.4.0, four weeks before the stated cutoff.
Scope note
We did NOT verify whether z.url({ protocol }) also rejects the single-slash form. The finding is that the model neither reached for the purpose-built validator nor knew that the leniency it described as unavoidable had been fixed inside its own training window.
4.4.0 published 2026-04-29, four weeks before this subject's stated 2026-05 cutoff. Chargeable here; NOT chargeable against Sonnet 5 or Fable 5, whose stated cutoffs are 2026-01.
What the model believes
Wrote the correct construction, then disclaimed it: "I'm less sure Zod enforces key presence at runtime here", and offered an 8-line z.unknown().superRefine(...).pipe(...) workaround as the safe path.
What it wrote
// 8-line z.unknown().superRefine(...).pipe(...) offered as the safe path
What works on zod 4.5.2
z.union([z.boolean(), z.undefined()])
Impact
Output is correct; the belief is not, and the belief is what a user acts on. The behaviour was pinned down 2026-04-29, four weeks before the stated cutoff.
A property whose schema is z.undefined() is now treated as required. The key must be present, but its value may be undefined.
What it got right, and near misses
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
correct
.merge()
Task 3, combining refined schemas: avoided .merge() entirely (shape-hoist plus re-refine, or .and()), so it would not have hit the 4.4.0 .merge()-throws change. The .merge() deprecation is old enough to be solidly in training.
correct
z.tuple() defaults
Task 9, tuple defaults: predicted T.parse(["a"]) returns ["a", 7] — correct on >= 4.4.0 — and flagged that zod 3 fails. Confidence self-rated "moderately high". The only subject to get this right with a confident reason.
correct
z.base64()
Task 8, base64 with a newline: predicted failure, correct today. Its stated reasoning — that the regex was always anchored — was recorded here as an obsolete model of why, and it is not one. — CORRECTED 2026-09-08 (JOURNAL/096): this record penalised a correct model. LF11 was corrected to 4.0.0 on 2026-09-06 and measured below the major on 2026-09-07: z.base64() rejects whitespace on every stable 4.x release, so "the regex was always anchored" is right about the whole line this draw was asked about, and the withdrawn sentence's claim that whitespace "was in fact accepted until 4.4.0" is what the ladder measured to be false. Nothing charged, so no count moves; the demerit is withdrawn. — RECLASSIFIED 2026-09-28 (JOURNAL/321): the correction above withdrew the demerit, and the row was still counted as an imprecision in every tally. Kind is now correct. (No wrong output.)
miss
z.slugify()
Task 6, slug: hand-rolled a slugify transform rather than using z.slugify() (4.1.13; announced only in the 4.3.0 notes). — CORRECTED 2026-09-08 (JOURNAL/096): left at 4.3.0; LF17 was bisected to 4.1.13 on 2026-09-06. Still above this subject's 4.1.0 boundary and inside its cutoff; the record stays a miss and charges nothing. (Working code, defensible choice. Recorded as a miss.)
context
—
Version recency, reproduced verbatim from the v1 run 24 hours earlier: "The most recent Zod release whose contents I can actually describe is the 4.1 line, released around August 2025 ... I cannot describe the contents of any release after 4.1 and would not trust myself to name the current version." The lag is stable and reproducible, not sampling noise. Calibration remains excellent — it named the blind spot unprompted and told the reader to check the changelog. (Already shipped as F1 of the v1 run for this subject.)
Open questions from this run
Five candidate probes from the v2 sketch — z.compile, z.validate, z.creditCard, z.properties, deepPartial/exactPartial — target 4.5.0 features. Are they legal probes? — resolved-non-finding: Excluded before testing. All shipped in 4.5.0 on 2026-08-28, after every subject's cutoff. They become legal against any model with a cutoff after 2026-08-28. See the probe fairness rule in prompts/zod.md v2.