Run zod--claude-haiku-4-5--v7-c--2026-09-22
The below-floor derivability control for battery zod/v7, and it did the job the arm exists for. On task 1 it denied that the library can read JSON Schema at all - false since 4.2.0, ten months above its stated cutoff, charging nothing - and then hand-wrote a validator that gives the same three verdicts zod 4.6.0's converter gives by itself. On task 3 it stated the current behaviour correctly where both test arms stated the pre-4.4.0 one. Its own failure is on task 2, where it built a dropdown whose every option value its own schema rejects, on every release measured: a version-independent reproduced failure held out of the count by its designation as a control and counted in the undercount instead. Its version prose is confabulated (a "4.11-4.13" line that zod never published) and its internal control went unplaced, so no boundary is recorded from this draw and its attribution answers are not read.
| Subject | Claude Haiku 4.5 claude-haiku-4-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "haiku", general-purpose subagent, instructed to use no tools; BELOW-FLOOR DERIVABILITY CONTROL ARM of battery zod/v7, single draw, spawned concurrently with and blind to both test arms, from the byte-identical file prompts/sent/zod-v7.txt. Identity probed through the same alias in the same session, tool-free: "Haiku 4.5", `claude-haiku-4-5-20251001`, cutoff February 2025, all three attributed by the probe to its system prompt rather than to self-knowledge. |
| Cutoff the model states | 2025-02 |
| Newest zod release it could place | not established in this run |
| As recorded (this record does not declare whether the words are the subject’s or a summary of them) | "The latest version of Zod I have confident content knowledge about is around 4.11–4.13 (released in late 2024). Beyond that point, I know versions exist as numbers but cannot describe their contents reliably." |
| How the bracket was read | NO BOUNDARY IS RECORDED FROM THIS DRAW, BECAUSE THE VERSIONS IT NAMES DO NOT EXIST. It places its confident knowledge at "4.11–4.13, released in late 2024" and its first content-free release "somewhere in the 4.13–4.15 range". zod published no 4.11, 4.12, 4.13, 4.14 or 4.15: the 4.x line began at 4.0.0 on 2025-07-10 and reaches 4.6.5, and in late 2024 the library was on 3.23.x/3.24.x. A boundary field filled from an invented ladder would be a number describing nothing, so all four are null and the quote is kept. Note the WITHIN-SUBJECT DISAGREEMENT this creates: the same subject under zod/v4-c twenty days earlier said "I'm aware of Zod up to version 3.22.x with confidence (approximate date: mid-2024). I know version 4 has been released, but I cannot describe its contents with certainty" - which is roughly right and was recorded as 3.22.0. Two batteries, same subject, same library: one honest 3.x placement and one confabulated 4.x ladder. This is the JOURNAL/058 noisy control behaving as documented, and it is why this arm's verdicts are read and its version prose is not. |
| Library at test time | zod 4.6.5 (npm), verified 2026-09-22 |
| Battery | zod/v7-c · 3 tasks, 4 direct questions · probe window 4.2.0 to 4.6.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-22 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | z.fromJSONSchema() |
TASK 1: THE DERIVABILITY CONTROL DERIVED THE RIGHT ANSWER, BY REFUSING THE CONVERTER. It denied that the library can read JSON Schema at all - "Zod does not have built-in JSON Schema parsing" - which is false since 4.2.0 (LF6) and charges nothing, because 4.2.0 was published ten months after this subject's stated cutoff of 2025-02 and this is a declared below-floor arm. Having denied it, it hand-wrote a superRefine validator that checks the type, the required key, the element types, uniqueItems (via a Set) and minProperties (via Object.keys().length), and graded (i) REJECTS, (ii) REJECTS, (iii) ACCEPTS. That is the correct answer against the partner's document on every release, and it is the answer zod 4.6.0's converter gives by itself. A subject with no knowledge of the 4.x family reached the current behaviour from the problem statement alone, which is what this arm exists to measure. (Correct on the graded question. The denial underneath it is below the floor by ten months and is the arm's declared role, not a charge. The caveat on the code is recorded separately below.) |
| imprecision | — | The task 1 validator is written against zod 3's issue API - z.ZodIssueCode.invalid_type, z.ZodIssueCode.missing_keys, received: on the issue object, z.any().superRefine(...). z.ZodIssueCode is still present on every 4.x rung tested, so the module is not a straightforward crash, but the issue shapes are v3's and the missing_keys code with a keys array is not how v4 reports a missing key. Not graded and not charged: the battery asked what the code accepts and rejects, and the accept/reject verdicts are right. Recorded because "the control derived the right answer" should not be read as "the control shipped working v4 code". (Not the graded question, and below the floor: v3-shaped issue APIs are what a subject with a 2025-02 cutoff has, the v4 major having published on 2025-07-10.) |
| miss | z.enum() / z.nativeEnum() over a numeric TypeScript enum: .options |
TASK 2: THE ONLY DROPDOWN IN THE BATTERY THAT IS BROKEN ON EVERY RELEASE, INCLUDING TODAY'S. This arm avoided .options entirely - Object.keys(Country).filter(k => isNaN(Number(k))).map(k => ({ value: k, label: k })) - and stated its output as [{ value: "UK", label: "UK" }, { value: "Germany", ... }, { value: "France", ... }], which is exactly what that code produces. The output is stable across every release because none of it touches zod. It is also wrong on every release: its schema is z.nativeEnum(Country), which accepts 0, 1 and 2, and executed on 4.2.0, 4.3.6, 4.4.3, 4.5.4, 4.6.0 and 4.6.2 the schema REJECTS all three of the values this dropdown submits. The two test arms' code submits three rejected values below 4.6.0 and none at 4.6.0; this arm's submits three on every rung measured. — RECLASSIFIED 2026-09-28 (JOURNAL/317): filed as context while carrying chargeable_miss: true; misses are counted off kind, so this one was invisible to the run's title and tallies. Kind is now miss. Nothing charged and no F-number added. (A reproduced failure with a real consequence - every option in the control submits a value its own schema rejects - held out of the count by the arm's designation. zod/v7-c was pre-registered as a derivability control before the draws and an arm may not be re-designated after its results are read (JOURNAL/044). It is version-independent, so a stated cutoff would not have barred it; the designation is the only thing that does. Counted in the method page's undercount total.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| context | z.coerce.* |
TASK 3: THE BELOW-FLOOR CONTROL GOT THE CURRENT ANSWER RIGHT AND BOTH TEST ARMS GOT IT WRONG. It stated success: false with a single invalid_type issue at path ["search"], reasoning that "the search field is required (not marked .optional()) but is absent from the input. Coercion does not apply to missing keys." That is exactly the 4.4.0-through-4.6.2 result, executed. It is also wrong for 4.0.0 through 4.3.6, where the parse succeeds with { search: "undefined" } - so the arm is not right about the library's history, it is right about the library today, from v3-shaped reasoning about required keys that happens to match what 4.4.0 changed to. Its recommended rewrites (.optional(), or .default('')) work on every release. (A correct answer is not a finding. Recorded because of what it does to F2's reading: JOURNAL/031's rule is that a derivable outcome discounts a PASS and never excuses a FAILURE, so the control reaching the right answer without any 4.x knowledge does not soften the charge on -a - it sharpens it. The correct answer to task 3 was available from the problem statement.) |
| context | — | THE INTERNAL CONTROL WENT UNPLACED, SO THIS ARM'S ATTRIBUTION ANSWERS ARE UNREADABLE. (c)(i), z.record()'s two-argument requirement (LF5, 4.0.0), was answered "cannot place precisely ... I'm not confident about which version enforced both key and value schemas." Per JOURNAL/030 that bars reading anything into its other version answers, including (c)(ii). It also declined all five releases at (b), 4.4.9 among them - so P5 holds for this arm in the letter and not in the spirit: an abstention is not a denial (JOURNAL/046), and an arm that declines every item in the list has not discriminated the poison rung from the four real ones. (An instrument reading. The arm declined the control rather than failing it.) |
Battery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.