What AI coding models get wrong about zod

33 current findings across 16 models, verified against zod release notes. Latest zod at last verification: 4.5.4.

Where each model’s zod version attribution stops

ModelStated cutoffAttribution stops atLag
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 5.12026-06 4.1.0 · 2025-08-23 ~10 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Sonnet 52026-01 4.0.0 · 2025-07-10 ~6 months
Claude Sonnet 52026-01 4.0.0 · 2025-07-10 ~6 months

Measured by asking each model what shipped in a given zod release, then checking the answer against the release timeline. Not inferred from the vendor’s cutoff date. This is the newest release the model can correctly place, not the newest zod feature it knows — a model past this point often uses a newer API correctly while naming the wrong release for it. Why that distinction matters.

Findings

Sorted by severity. “Chargeable” means the change was published before that model’s own stated cutoff, so the model had the opportunity to know it.

SeverityBeliefModelChanged inChargeableProof
S1breaks-build single-argument z.record()
z.record()
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S1breaks-build States that deriving a sub-schema from a refined object schema loads without error and silently drops the refinement - the exact behaviour 4.3.0 replaced with a throw
.pick() / .omit() on a schema with refinements
Claude Fable 5 4.3.0
2025-12-31
yes source
S1breaks-build .pick() on a refined object schema throws
.pick() / .omit()
Claude Opus 5 4.3.0
2025-12-31
yes source
S1breaks-build .extend() overwrite on a refined schema throws; .safeExtend() unknown
.extend() / .safeExtend()
Claude Opus 5 4.3.0
2025-12-31
yes source
S1breaks-build States that .pick() and .omit() on a refined object schema succeed at load and silently drop the refinement, where 4.3.0 made them throw
.pick() / .omit() on a schema with refinements
Claude Opus 5 4.3.0
2025-12-31
yes source
S1breaks-build .pick() on a refined object schema throws
.pick() / .omit()
Claude Sonnet 5 4.3.0
2025-12-31
yes source
S1breaks-build .extend() overwrite on a refined schema throws
.extend() / .safeExtend()
Claude Sonnet 5 4.3.0
2025-12-31
yes source
S2silently-wrong asserts Zod cannot consume a JSON Schema at runtime
z.fromJSONSchema()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong hand-rolls exclusive-or
z.xor()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong Denies that the library can consume a JSON Schema document at runtime and prescribes adding Ajv as a second validator
z.fromJSONSchema()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong Denies that the library has an exclusive union, calling the exactly-one-of contract something you must build yourself
z.xor()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong declares an unnecessary dependency for JSON Schema input
z.fromJSONSchema()
Claude Opus 5 4.2.0
2025-12-15
yes source
S2silently-wrong hand-rolls exclusive-or, states Zod cannot express it
z.xor()
Claude Opus 5 4.2.0
2025-12-15
yes source
S2silently-wrong asserts z.fromJSONSchema() does not exist, with high confidence
z.fromJSONSchema()
Claude Sonnet 5 4.2.0
2025-12-15
yes source
S2silently-wrong hand-rolls exclusive-or, then advises abandoning schema-based validation
z.xor()
Claude Sonnet 5 4.2.0
2025-12-15
yes source
S2silently-wrong Denies that the library can consume a JSON Schema document at runtime and prescribes Ajv as a new dependency
z.fromJSONSchema()
Claude Sonnet 5 4.2.0
2025-12-15
no source
S2silently-wrong Denies that the library has an exclusive union primitive
z.xor()
Claude Sonnet 5 4.2.0
2025-12-15
no source
S2silently-wrong Denies that the library has any way to make an object key omittable while rejecting an explicit undefined value
.exactOptional()
Claude Fable 5 4.3.0
2025-12-31
yes source
S2silently-wrong States that an intersection of two strict object schemas is unsatisfiable, where 4.3.0 made it parse to the merged object
intersections involving z.strictObject()
Claude Fable 5 4.3.0
2025-12-31
yes source
S2silently-wrong Does not reach .exactOptional() for strict optionality, and offers a constructor it cannot confirm exists plus a type-system cast instead
.exactOptional()
Claude Opus 5 4.3.0
2025-12-31
yes source
S2silently-wrong Denies that the library has any way to make an object key omittable while rejecting an explicit undefined value, and argues the gap is structural
.exactOptional()
Claude Sonnet 5 4.3.0
2025-12-31
no source
S2silently-wrong States that an intersection of two strict object schemas cannot parse an object carrying keys from both sides
intersections involving z.strictObject()
Claude Sonnet 5 4.3.0
2025-12-31
no source
S2silently-wrong claims Zod cannot enforce key presence with an undefined value
z.undefined() object properties
Claude Fable 5 4.4.0
2026-04-29
no source
S2silently-wrong misses z.httpUrl(), prescribes a manual normalization workaround
z.httpUrl()
Claude Opus 5 4.4.0
2026-04-29
yes source
S2silently-wrong States that "https:/example.com" passes http/https URL validation, and prescribes the hand-rolled normalization 4.4.0 removed the need for
z.httpUrl()
Claude Opus 5 4.4.0
2026-04-29
yes source
S2silently-wrong claims Zod cannot distinguish a missing key from an explicit undefined
z.undefined() object properties
Claude Sonnet 5 4.4.0
2026-04-29
no source
S2silently-wrong wrong runtime prediction for tuple defaults
z.tuple() defaults
Claude Sonnet 5 4.4.0
2026-04-29
no source
S2silently-wrong misses z.httpUrl(), frames the leniency as a security problem to hand-roll around
z.httpUrl()
Claude Sonnet 5 4.4.0
2026-04-29
yes source
S3deprecated method-style string format validators
z.string().uuid() / .email() / .url()
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S3deprecated message-style custom errors
message / invalid_type_error / required_error
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S3deprecated .flatten() for form errors
ZodError.flatten() / .format()
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S4wrong-metadata asserts Zod 3 is the latest major and invents a changelog to support it Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S4wrong-metadata correct code, disbelieved
z.undefined() object properties
Claude Opus 5 4.4.0
2026-04-29
yes source

Runs

Superseded runs (kept, never deleted)