Run zod--claude-haiku-4-5--v1--2026-08-28
The only subject whose Zod knowledge predates the v4 line entirely. Four findings including an S1 that will not compile, plus a fabricated release history. Battery v1 discriminates strongly here, which is exactly why it is retained for smaller and older models even though it is saturated for frontier subjects.
| Subject | Claude Haiku 4.5 claude-haiku-4-5-20251001, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "haiku" |
| Cutoff the model states | not stated |
| Newest zod release it could place | 3.x (release date not established in this run) |
| Oldest zod release it could not place | 4.0.0 · 2025-07-10 (the lower end of the bracket is not established in this run, so it bounds the boundary from above only) |
| In its own words | claims v3 released Sept 2024 — also false; v3 shipped 2021 |
| Library at test time | zod 4.5 (npm), verified 2026-08-28 |
| Battery | zod/v1 · 6 tasks, 3 direct questions |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-08-28 |
| Findings | 5, of which 5 chargeable |
Wrote z.record(z.number()) for string-to-number records, i.e. that the single-argument form is current.
z.record(z.number())z.record(z.string(), z.number())Generated code fails type-checking on zod@4. Hard build break.
Wrote z.string().uuid(), z.string().email(...), z.string().url() and asserted ".email() method is the current recommended way to validate email."
z.string().uuid()
z.string().email()
z.string().url()z.uuid()
z.email()
z.url()Deprecated surface, and the "current recommended way" claim is false for current Zod.
Wrote z.string().email('Please enter a valid email') and { message: ... } as the way to customise errors.
z.string().email('Please enter a valid email')z.email({ error: 'Please enter a valid email' })Zod 4 unifies error customization under the error param; message is deprecated and invalid_type_error / required_error were dropped entirely.
Wrote result.error.flatten() to obtain fieldErrors / formErrors for a form UI.
result.error.flatten()z.treeifyError(result.error)Deprecated in Zod 4 alongside .format().
Asserted Zod 3 is the latest major, "released September 2024," and described a fictional v2-to-v3 changelog. Zod 3 in fact shipped in 2021.
A user asking this model "should I upgrade?" gets confidently wrong advice, with fabricated supporting detail and no hedge.
Recorded as the contrast case for calibration: every other subject hedged its version claim. This one did not.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | — | safeParse discrimination, z.infer, .partial(), .pick({...}), basic z.coerce.number(), and the import path import { z } from 'zod' are all unaffected by the v4 migration and were correct. |
Battery specification: prompts/zod.md in the studio repo.
Every finding above also carries its own citation.