What AI coding models get wrong about zod

37 current findings across 22 models, verified against zod release notes. Latest zod at last verification: 4.6.5.

Where each model’s zod version attribution stops

ModelStated cutoffAttribution stops atLag
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 52026-01 4.1.0 · 2025-08-23 ~5 months
Claude Fable 5.12026-06 4.1.0 · 2025-08-23 ~10 months
Claude Fable 5.12026-06 4.2.0 · 2025-12-15 ~6 months
Claude Fable 5.12026-06 4.2.0 · 2025-12-15 ~6 months
Claude Fable 5.12026-06 4.2.0 · 2025-12-15 ~6 months
Claude Fable 5.12026-06 4.2.0 · 2025-12-15 ~6 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 52026-05 4.1.0 · 2025-08-23 ~9 months
Claude Opus 5.52026-06 4.1.0 · 2025-08-23 ~10 months
Claude Sonnet 52026-01 4.0.0 · 2025-07-09 ~6 months
Claude Sonnet 52026-01 4.0.0 · 2025-07-09 ~6 months

Measured by asking each model what shipped in a given zod release, then checking the answer against the release timeline. Not inferred from the vendor’s cutoff date. This is the newest release the model can correctly place, not the newest zod feature it knows — a model past this point often uses a newer API correctly while naming the wrong release for it. Why that distinction matters.

One subject at a time

The same data, split the way the question usually gets asked — one page per subject, with its boundary measurements, its findings on zod, and how repeatable the boundary was.

And the comparison, ranked by measured boundary and by nothing else: Which Claude model knows zod 4 best?

Findings

Sorted by severity. “Chargeable” means the change was published before that model’s own stated cutoff, so the model had the opportunity to know it.

SeverityBeliefModelChanged inChargeableProof
S1breaks-build single-argument z.record()
z.record()
Claude Haiku 4.5 4.0.0
2025-07-09
yes source
S1breaks-build .extend() overwrite on a refined schema throws; .safeExtend() unknown
.extend() / .safeExtend()
Claude Opus 5 4.1.0
2025-08-23
yes source
S1breaks-build .extend() overwrite on a refined schema throws
.extend() / .safeExtend()
Claude Sonnet 5 4.1.0
2025-08-23
yes source
S1breaks-build States that deriving a sub-schema from a refined object schema loads without error and silently drops the refinement - the exact behaviour 4.3.0 replaced with a throw
.pick() / .omit() on a schema with refinements
Claude Fable 5.1 4.3.0
2025-12-31
yes source
S1breaks-build States that deriving a sub-schema from a refined object schema loads without error and silently drops the refinement - the exact behaviour 4.3.0 replaced with a throw
.pick() / .omit() on a schema with refinements
Claude Fable 5 4.3.0
2025-12-31
yes source
S1breaks-build .pick() on a refined object schema throws
.pick() / .omit()
Claude Opus 5 4.3.0
2025-12-31
yes source
S1breaks-build States that .pick() and .omit() on a refined object schema succeed at load and silently drop the refinement, where 4.3.0 made them throw
.pick() / .omit() on a schema with refinements
Claude Opus 5 4.3.0
2025-12-31
yes source
S1breaks-build .pick() on a refined object schema throws
.pick() / .omit()
Claude Sonnet 5 4.3.0
2025-12-31
yes source
S2silently-wrong asserts Zod cannot consume a JSON Schema at runtime
z.fromJSONSchema()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong hand-rolls exclusive-or
z.xor()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong Denies that the library can consume a JSON Schema document at runtime and prescribes adding Ajv as a second validator
z.fromJSONSchema()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong Denies that the library has an exclusive union, calling the exactly-one-of contract something you must build yourself
z.xor()
Claude Fable 5 4.2.0
2025-12-15
yes source
S2silently-wrong declares an unnecessary dependency for JSON Schema input
z.fromJSONSchema()
Claude Opus 5 4.2.0
2025-12-15
yes source
S2silently-wrong hand-rolls exclusive-or, states Zod cannot express it
z.xor()
Claude Opus 5 4.2.0
2025-12-15
yes source
S2silently-wrong asserts z.fromJSONSchema() does not exist, with high confidence
z.fromJSONSchema()
Claude Sonnet 5 4.2.0
2025-12-15
yes source
S2silently-wrong hand-rolls exclusive-or, then advises abandoning schema-based validation
z.xor()
Claude Sonnet 5 4.2.0
2025-12-15
yes source
S2silently-wrong Denies that the library can consume a JSON Schema document at runtime and prescribes Ajv as a new dependency
z.fromJSONSchema()
Claude Sonnet 5 4.2.0
2025-12-15
no source
S2silently-wrong Denies that the library has an exclusive union primitive
z.xor()
Claude Sonnet 5 4.2.0
2025-12-15
no source
S2silently-wrong Denies that the library has any way to make an object key omittable while rejecting an explicit undefined value
.exactOptional()
Claude Fable 5 4.3.0
2025-12-31
yes source
S2silently-wrong Does not reach .exactOptional() for strict optionality, and offers a constructor it cannot confirm exists plus a type-system cast instead
.exactOptional()
Claude Opus 5 4.3.0
2025-12-31
yes source
S2silently-wrong Denies that the library has any way to make an object key omittable while rejecting an explicit undefined value, and argues the gap is structural
.exactOptional()
Claude Sonnet 5 4.3.0
2025-12-31
no source
S2silently-wrong States that an intersection of two strict object schemas can never be satisfied, and advises against the construction on that basis - true of the library for the first 49 minutes of 4.3.x and false ever since
intersections involving z.strictObject()
Claude Fable 5.1 4.3.2
2025-12-31
yes source
S2silently-wrong States that an intersection of two strict object schemas is unsatisfiable, where 4.3.0 made it parse to the merged object
intersections involving z.strictObject()
Claude Fable 5 4.3.2
2025-12-31
yes source
S2silently-wrong States that an intersection of two strict object schemas cannot parse an object carrying keys from both sides
intersections involving z.strictObject()
Claude Sonnet 5 4.3.2
2025-12-31
no source
S2silently-wrong States that an object schema with a coerced string field accepts a missing key and parses it to the string "undefined" - true for eleven releases and false since 4.4.0
z.coerce.*
Claude Fable 5.1 4.4.0
2026-04-29
yes source
S2silently-wrong claims Zod cannot enforce key presence with an undefined value
z.undefined() object properties
Claude Fable 5 4.4.0
2026-04-29
no source
S2silently-wrong misses z.httpUrl(), prescribes a manual normalization workaround
z.httpUrl()
Claude Opus 5 4.4.0
2026-04-29
yes source
S2silently-wrong States that "https:/example.com" passes http/https URL validation, and prescribes the hand-rolled normalization 4.4.0 removed the need for
z.httpUrl()
Claude Opus 5 4.4.0
2026-04-29
yes source
S2silently-wrong claims Zod cannot distinguish a missing key from an explicit undefined
z.undefined() object properties
Claude Sonnet 5 4.4.0
2026-04-29
no source
S2silently-wrong wrong runtime prediction for tuple defaults
z.tuple() defaults
Claude Sonnet 5 4.4.0
2026-04-29
no source
S2silently-wrong misses z.httpUrl(), frames the leniency as a security problem to hand-roll around
z.httpUrl()
Claude Sonnet 5 4.4.0
2026-04-29
yes source
S2silently-wrong Asserts that converting the partner's document throws on its uniqueItems and minProperties keywords, so no payload is ever checked - the converter accepts that document without a word on every release it has ever existed in
z.fromJSONSchema(): which JSON Schema keywords throw and which convert silently
Claude Fable 5.1 —
yes source
S3deprecated method-style string format validators
z.string().uuid() / .email() / .url()
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S3deprecated message-style custom errors
message / invalid_type_error / required_error
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S3deprecated .flatten() for form errors
ZodError.flatten() / .format()
Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S4wrong-metadata asserts Zod 3 is the latest major and invents a changelog to support it Claude Haiku 4.5 4.0.0
2025-07-10
yes source
S4wrong-metadata correct code, disbelieved
z.undefined() object properties
Claude Opus 5 4.4.0
2026-04-29
yes source

Runs

Superseded runs (kept, never deleted)