Run zod--claude-haiku-4-5--v4-c--2026-09-02
The below-floor control, and the arm that decided what the rest of the battery was allowed to claim. Five of six probes came back not derivable: a subject whose zod knowledge stops before the 4.0 major produced none of the post-4.2 answers, which is what makes the test arms' denials readable as real gaps. The sixth is the interesting one - asked what an intersection of two strict object schemas parses to, this control gave the correct 4.3.0 answer that every test draw got wrong, and gave it from reasoning that is wrong about how intersections work in any version. The probe's outcome is therefore marked derivable and its mechanism is not, which is the per-component flag JOURNAL/030 introduced, used here for the first time on a probe where the control out-answered the test arm.
| Subject | Claude Haiku 4.5 claude-haiku-4-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "haiku", general-purpose subagent, instructed to use no tools; BELOW-FLOOR CONTROL ARM of battery zod/v4 |
| Cutoff the model states | 2025-02 |
| Newest zod release it could place | 3.22.0 (release date not established in this run) |
| Oldest zod release it could not place | 4.0.0 · 2025-07-10 (the lower end of the bracket is not established in this run, so it bounds the boundary from above only) |
| In its own words | "I'm aware of Zod up to version 3.22.x with confidence (approximate date: mid-2024). I know version 4 has been released, but I cannot describe its contents with certainty." |
| Library at test time | zod 4.5.4 (npm), verified 2026-09-02 |
| Battery | zod/v4-c · 6 tasks, 4 direct questions · probe window 4.2.0 to 4.3.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| context | intersections involving z.strictObject() |
PROBE 5 IS DERIVABLE ON ITS OUTCOME. Asked what an intersection of two strictObjects parses { a: "x", b: "y" } to, this control - eighteen months below the floor and unable to describe any zod 4 release - gave the correct post-4.3.0 answer, { a: "x", b: "y" }, while every draw of both test subjects said it throws. Per JOURNAL/030's rule that the derivable flag applies per component: the OUTCOME is derivable, the MECHANISM is not - this draw reasoned that the two schemas "are unified into a single schema requiring both a and b", which is not what an intersection does in any zod version, and it doubted in the same breath that z.strictObject exists at all ("If z.strictObject() is not a real Zod API - the actual call would be z.object().strict()"). It reached the right value along a route that would have been wrong before 4.3.0 too. (A derivable outcome makes a PASS unreadable as knowledge. It does not excuse a failure, so the test arms are still charged on it - see the scope notes on those findings. Recorded here because the flag counts against the probe, not against any subject.) |
| context | — | THE OTHER FIVE PROBES ARE NOT DERIVABLE. This control denied that an exact-optional wrapper exists ("Zod 4 doesn't have native support for this"), denied an exclusive union ("Does not exist as a built-in"), denied runtime JSON Schema consumption ("Zod cannot do this ... it's architecturally the wrong direction and doesn't exist in maintained form"), hand-rolled the slug chain, and gave the pre-4.3 answer on the refined-schema derivation ("The module loads without error ... the refinement does not carry through .pick()"). So on five of six probes a subject that cannot know the answer does not produce it, which is what makes the test arms' failures readable as failures rather than as an unanswerable battery. (It is the control doing its job.) |
| context | — | THE CONTROL IS BLUNT, AND THE RUN SHOWS IT. This subject's boundary sits below zod 4 entirely - it names 3.22.x as its last describable release and 4.0 as the first it knows only as a number - so its failures carry no information about the 4.2/4.3 window under test. It also volunteered doubt about z.strictObject and misplaced the two-argument dictionary constructor at "Zod 3.0 or 3.1", where fact LF5 puts it at the 4.0.0 major. zod/v3 had control arms four months below its floor; this one is eighteen months below, because it is the only below-floor subject available for a 4.2/4.3 target. A blunt control can still say a probe is not derivable, which is all this arm was asked for. (Belief data from a below-floor subject, outside the fairness window by construction. This arm charges nothing.) |
| imprecision | z.record() |
THE INTERNAL CONTROL FAILED FOR THIS SUBJECT. (d)(i) placed the two-argument requirement on the dictionary constructor at "Zod 3.0 or 3.1 (estimate: late 2022-early 2023)", where it is a breaking change of the 4.0.0 major (fact LF5). Under JOURNAL/030's rule, none of this draw's other attribution answers are read. (Marked as an estimate, from a below-floor control arm that charges nothing. Recorded because it is what makes the rest of this draw's question (d) unreadable.) |
Battery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.