Run zod--claude-opus-5--v2r-a--2026-09-01 · self-test: the subject is the operator
Replicate A of zod/v2 against Opus 5, prompt unchanged. It placed its describable boundary at Zod 4.1 (2025-08-23), agreeing with its concurrent blind twin v2r-b and with zod/v2 - three measurements, one answer, spread 0 days. The zod arm was pre-registered to spread and did not, which falsifies both volume hypotheses in the predictor test. No findings charged. It differs from its twin on task 9's runtime behaviour while agreeing on the boundary exactly.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus", no tools available to the subject |
| Cutoff the model states | 2026-05 |
| Newest zod release it could place | 4.1.0 · 2025-08-23 (~9 month lag) |
| Oldest zod release it could not place | 4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15) |
| In its own words | "The newest release I can describe the contents of is Zod 4.1, around August 2025 - codecs and z.encode/z.decode/z.safeEncode/z.safeDecode, .safeExtend(), and further parser performance work on top of the 4.0 line. ... I cannot reliably name the actual current version. Given my cutoff, the true latest is very likely 4.1.x or a later 4.x minor released after my confident knowledge thins out, and I'd be guessing at the number." |
| Library at test time | zod 4.5.4 (npm), verified 2026-09-01 |
| Battery | zod/v2r-a · 10 tasks, 3 direct questions · probe window 4.2.0 to 4.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-01 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | — | The measured quantity. This draw placed the last release whose contents it can describe at Zod 4.1 (2025-08-23) and attributed it correctly - codecs, z.encode/z.decode, .safeExtend() - then declined to name a current version. Its concurrent, blind twin v2r-b gave the same boundary, and so did zod/v2 three days earlier. Three measurements, one answer, spread 0 days. The zod arm was pre-registered to spread; it did not. (A boundary self-report is a belief datum, never a finding. It is what this run measures.) |
| context | — | Reproduced without charge, because zod/v2 already carries all of them: F3 - task 5 states that no JSON Schema to Zod path exists and sends the user to Ajv, where 4.2.0 ships z.fromJSONSchema(). F4 - task 4 hand-rolls exclusive-or three different ways without reaching z.xor() (4.2.0). F5 - task 8 states that "https:/example.com" parses successfully and prescribes a manual normalization workaround; 4.4.0 made z.httpUrl() reject it. F6 - task 7 writes a superRefine presence guard piped into an object schema and states that a z.undefined()-accepting property does not enforce key presence, which 4.4.0 reversed. F1/F2 - direct question (c) states that Zod 4 removed Zod 3's hard block on .pick()/.extend() over refined schemas, where 4.3.0 and 4.4.0 make them throw. Also reproduced: task 6 hand-rolls a slugify transform rather than using z.slugify() (4.3.0). (Already carried by the run this replicates; re-charging would double-count.) |
| correct | z.tuple() defaults |
Task 9, and the place where the two draws part company. Asked what a tuple with a defaulted second element returns for a one-element input, this draw answered ["a", 0] - correct on 4.4.0 and later, where tuple defaults materialize - and correctly flagged that Zod 3 throws instead. It also volunteered that this was the claim it would most want verified. zod/v2 answered the same way. Its twin v2r-b answered that the parse throws, which is wrong on 4.4.0. Same prompt, same model, same day, same self-reported boundary, opposite answers about runtime behaviour. (A correct answer is not a stale prior.) |
| imprecision | z.base64() |
Task 8, the base64 half: predicted that a line-wrapped base64 string fails, which is correct on 4.4.0 and later, but reasoned from a belief that z.base64() was always a whitespace-free regex. 4.4.0 is the release that made base64 reject whitespace, so the right answer rests on a wrong model of the library's history. zod/v2 recorded the identical pattern. Both draws reproduce it. (The stated behaviour is correct today; only the reasoning is stale, and a replicate does not charge in any case.) |
| context | — | Belief data where the draws differ from each other without either being scored. This draw dated Zod 4.0 stable to "May 2025"; v2r-b dated it to "around July 2025". The registry time map gives 2025-07-09, so the twin is right and this draw is nine weeks early - on a release both of them can describe in detail. A boundary that agrees exactly while the date attached to a release inside it disagrees by two months is worth noting: the version ordering is stable, the calendar is not. (Direct-question belief data, scored separately from the code tasks by the battery's own rule.) |
.safeExtend() as a 4.1 addition that "exists precisely because plain .extend() handles existing object-level checks unsoundly" while simultaneously asserting that Zod 4 lifted the restriction on .pick()/.extend() over refined schemas. Those two statements are in tension and the battery has no way to score a subject that holds both. The Index's finding F2 charges the second; the first is closer to right than anything zod/v2 recorded. — openBattery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.