What Claude Fable 5.1 gets wrong about zod — battery v7-a, tested 2026-09-22

Run zod--claude-fable-5-1--v7-a--2026-09-22

Summary

The first zod run dated after 4.6.0, closing thirteen days of retest debt (BACKLOG 0d). Two charges, both S2, both on the surfaces the battery could charge on: an invariant belief that z.fromJSONSchema() throws on the partner document's uniqueItems and minProperties - the mechanism is real, three of the five keywords the draw names do throw, and the two in front of it are not among them - and the 4.4.0 coercion change, the one live uncharged cell this library had left, answered with the pre-4.4 behaviour and hedged with the correct post-4.4 result it did not believe. The declared derivability probes charge nothing by construction and returned the battery's actual result: on task 2 both blind twins described zod 4.6.0's option list exactly, three months after their stated cutoff and wrongly for every release they can know, which is a stale prior running backwards and a shape the corpus has no rule for.

SubjectClaude Fable 5.1 claude-fable-5-1, Anthropic
Invoked asAgent tool, model alias "fable", general-purpose subagent, instructed to use no tools; CHARGING TEST ARM of battery zod/v7. Prompt sent verbatim from prompts/sent/zod-v7.txt. Identity probed through the same alias in the same session, tool-free: "Fable 5.1", `claude-fable-5-1`, cutoff June 2026 — and the probe volunteered that all three came from its system prompt rather than from self-knowledge, adding that its own internal sense of its cutoff is "earlier and fuzzier than that". Same answer, same wording of the caveat, as the identity probe stored with zod/v6.
Cutoff the model states2026-06
Newest zod release it could place4.2.0 · 2025-12-15 (~6 month lag)
Oldest zod release it could not place4.3.0 · 2025-12-31 (so this run brackets the subject’s boundary to 2025-12-15 – 2025-12-31)
As recorded (this record does not declare whether the words are the subject’s or a summary of them)"The most recent release I can describe the contents of is 4.2.0, roughly December 2025 (possibly early January 2026). I have a weak sense that a 4.3 line exists (early 2026?) but nothing attached to it ... it's a lower bound: a library on Zod's cadence has almost certainly shipped things I haven't seen."
How the bracket was readSelf-placed and demonstrated at the same release, which is not the usual case for this subject. It named 4.2.0 at (a) and (d), described 4.2.0's headline content at (b) (z.fromJSONSchema()), and used that API unprompted in task 1. zod/v6 measured the same boundary on the same subject sixteen days earlier from a different prompt file, so this is the second battery running to put Claude Fable 5.1's zod knowledge at 4.2.0. (The ladder battery zod/v5 put it one lower, at 4.1.0; zod/v6 moved it to 4.2.0 on demonstrated APIs and this run agrees.)
Library at test timezod 4.6.5 (npm), verified 2026-09-22
Batteryzod/v7-a · 3 tasks, 4 direct questions · probe window 4.2.0 to 4.6.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-22
Findings2, of which 2 chargeable

Findings

F1 · Asserts that converting the partner's document throws on its uniqueItems and minProperties keywords, so no payload is ever checked - the converter accepts that document without a word on every release it has ever existed in

S2silently-wrong · z.fromJSONSchema(): which JSON Schema keywords throw and which convert silently · invariant · chargeable

An invariant finding: the belief is false on every measured release, so there is no introducing release for a stated cutoff to sit below and none of the fairness arithmetic applies (HARNESS, JOURNAL/046). Executed on all six rungs that carry the converter, 4.2.0 through 4.6.2: z.fromJSONSchema() returns a schema for this document on every one of them and throws on none.

What the model believes

"Both uniqueItems and minProperties are, as I remember the 4.2 docs, on the unsupported list (alongside things like if/then/else, dependentSchemas, unevaluatedProperties, contains). I don't recall an option to downgrade that to a warning." And, as the graded answer to (i), (ii) and (iii): "none is accepted or rejected. compileDocument throws before any payload is looked at, with an error naming the unsupported keyword (uniqueItems first, since it's encountered first)."

What it wrote
export function compileDocument(doc: unknown) {
  // Throws synchronously if the document uses a keyword Zod can't represent.
  return z.fromJSONSchema(doc as z.core.JSONSchema.BaseSchema);
}   // asserted to throw on this document; it returns a schema on all six rungs
What works on zod 4.6.5
// The converter is silent about these two keywords below 4.6.0, so the caller must add them.
const base = z.fromJSONSchema(doc);   // does NOT throw for uniqueItems / minProperties
const schema = base.superRefine((val, ctx) => {
  const tags = (val as any)?.tags;
  if (Array.isArray(tags) && new Set(tags.map((x) => JSON.stringify(x))).size !== tags.length)
    ctx.addIssue({ code: "custom", path: ["tags"], message: "items must be unique" });
  const meta = (val as any)?.meta;
  if (meta && typeof meta === "object" && Object.keys(meta).length < 2)
    ctx.addIssue({ code: "custom", path: ["meta"], message: "expected at least 2 properties" });
});
// From 4.6.0 the converter enforces both itself and the superRefine is redundant, not wrong.
Impact

The belief is not that a call fails - it is that the converter is FAIL-LOUD, and that is the property a reader builds the integration on. Told that an unrepresentable keyword throws at conversion time, a caller wraps compileDocument in a try/catch, treats a throw as "this partner document is out of scope", and ships nothing else. On 4.2.0 through 4.5.4 the document converts cleanly and the resulting validator accepts { tags: ["a", "a"] } and { meta: { x: 1 } } - exactly the two payloads the partner's contract rejects - with no error, no warning and no throw to catch. The failure is silent at the only point where it could have been noticed.

Scope note

THE MECHANISM IS REAL AND THE LIST IS WRONG, which is why this is charged as a belief about behaviour rather than as an invention of a feature. Executed on all six converter rungs: if/then/else THROWS ("Conditional schemas (if/then/else) are not supported"), dependentSchemas THROWS, unevaluatedProperties THROWS - three of the five keywords this draw names, exactly as it describes, on every release from 4.2.0 to 4.6.2. uniqueItems, minProperties and contains - the other two it names, plus patternProperties - convert silently on all six. So the draw correctly recalls that z.fromJSONSchema() has a throw-on-unsupported policy and wrongly places inside it the two keywords the document in front of it actually uses. The charge is on the consequence it states for THIS document, which is false at every release. Its own recommended rewrite - strip the unsupported keywords, record them, re-check them in a superRefine - produces the right answers to (i), (ii) and (iii) and is not what is charged; the detector it relies on to know what to strip is the throw that does not happen. Its blind twin zod/v7-b, sent the byte-identical file, believed the opposite ("drops keywords it can't represent rather than throwing") and was right. Run single, this arm would have published an invention its twin does not hold.

Verified against

F2 · States that an object schema with a coerced string field accepts a missing key and parses it to the string "undefined" - true for eleven releases and false since 4.4.0

S2silently-wrong · z.coerce.* · stricter · changed in zod 4.4.0 (2026-04-29) · chargeable

4.4.0 (2026-04-29) precedes this subject's stated cutoff of 2026-06 by one month, and is two minors above its measured boundary of 4.2.0. zod 4.4.0 x Claude Fable 5.1 was the one live uncharged cell this library still showed on charge-windows.mjs; the other three drawable subjects were already charged at this release. Bisected this session across all seventeen 4.x rungs: 4.0.0 through 4.3.6 succeed, 4.4.0 through 4.6.2 fail, contiguous.

What the model believes

"On the Zod 4 releases whose behaviour I can actually describe (4.0 through 4.2): z.coerce.string() does String(input) unconditionally, String(undefined) is \"undefined\", and the object parser runs each field's schema against input[key] even when the key is absent. So: result.success === true; result.data is { search: \"undefined\" }."

What it wrote
const Query = z.object({
  search: z.coerce.string(),
  verbose: z.boolean().optional(),
});
Query.safeParse({});   // asserted success: true, data { search: "undefined" }
What works on zod 4.6.5
// From 4.4.0 the parse FAILS: one invalid_type issue at path ["search"].
// Declare the fallback rather than relying on coercion to invent one:
const Query = z.object({
  search: z.coerce.string().default(""),
  verbose: z.boolean().optional(),
});
Query.safeParse({});   // success: true, data { search: "" }, on every 4.x release
Impact

The belief and the truth are both quiet, and they fail in opposite directions. A handler written on this belief expects safeParse({}) to succeed and to hand it a string; from 4.4.0 it gets success: false and takes the error branch, so a request that used to be served is rejected with a validation error naming a field the caller never sent. Nothing throws and nothing logs a version problem - the endpoint simply starts 400ing on a shape it accepted before the upgrade. The draw's own recommended rewrite (z.string().optional(), z.stringbool()) is correct on every release, so a reader who follows the final snippet is safe and a reader who follows the stated behaviour of the module in front of them is not.

Scope note

THE DRAW HEDGED AND THE HEDGE NAMED THE CORRECT ANSWER, which is the JOURNAL/052 hedged-demonstration case and is disclosed rather than folded into the grade. After committing to success: true, it added: "If one exists in a release newer than I know, the result would instead be success: false with one issue: { code: \"invalid_type\", expected: \"string\", path: [\"search\"] }" - which is the exact shape of the real 4.4.0 result, issue code and path included. The battery's P3 was pre-registered as falsified only by an arm STATING the parse fails; this arm states it succeeds and offers the correct behaviour as a conditional it does not believe, so the charge stands under the rule as written. Both readings are published and neither overwrites the other. Its blind twin zod/v7-b gave the same verdict with the same hedge, so this is the subject's answer and not a draw. The below-floor derivability control zod/v7-c (Claude Haiku 4.5, stated cutoff 2025-02) got this RIGHT - see the non_findings - which under JOURNAL/031 discounts a pass and never excuses a failure.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
contextz.enum() / z.nativeEnum() over a numeric TypeScript enum: .options P2 IS FALSIFIED IN THE DIRECTION NOBODY REGISTERED, AND IT IS THE RESULT OF THE BATTERY: THE SUBJECT IS WRONG ABOUT EVERY RELEASE IT CAN KNOW AND RIGHT ABOUT THE ONE IT CANNOT. Task 2's dropdown was predicted to be built by filtering .options for strings, which yields [] from 4.6.0. Neither twin did that. Both took the VALUES from .options and the labels from the enum's reverse mapping - CountrySchema.options.map((value) => ({ value, label: Country[value] })) - and both asserted, unprompted, that .options is already filtered: "Zod 4's z.enum filters out the numeric-key reverse mappings, so .options is the clean value list" (-b), "Using Object.entries(Country) directly would be wrong here - CountrySchema.options is already filtered" (-a). That is a precise description of zod 4.6.0, published 2026-09-09, three months after this subject's stated cutoff. Executed: on 4.2.0, 4.3.6, 4.4.3 and 4.5.4 the twins' code produces SIX options, three of which ({ value: "UK", label: 0 }, { value: "Germany", label: 1 }, { value: "France", label: 2 }) the schema itself REJECTS; on 4.6.0 and 4.6.2 it produces exactly the three they claim, element for element, and the schema rejects none of them. (The Index charges divergence from CURRENT truth, and against current zod there is none: the code is right, the stated output is right, the dropdown works. This is the inverse shape of a stale prior - a belief false for every release the subject can know and true of the library as it stands - and the corpus has no rule for it. Inventing that rule inside the session that would benefit from a third finding is the wrong order (the JOURNAL/046 precedent). Queued in BACKLOG as a rule question, with the reader's exposure stated plainly: anyone pinned to 4.2.x-4.5.x who ships this code gets a select control whose first three entries fail validation on submit.)
contextz.fromJSONSchema() with minProperties / maxProperties / uniqueItems / contains / minContains / maxContains P1 IS SPLIT BETWEEN THE TWINS, WHICH IS WHY THE ARM IS DUPLICATED. The prediction was that both test draws would describe payload (i) as accepted - the correct pre-4.6 answer - and that neither would add its own uniqueness check. -b did exactly that and was right about 4.2.0 through 4.5.4. -a instead asserted the conversion throws, which is false at every release, and is charged for it as F1. One stored file, two concurrent blind draws, and the two arms hold contradictory beliefs about the same API's failure mode. HARNESS's case for duplicating the test arm is that the self-report is the half that moves; this is the first pair in the corpus where the halves that moved were two incompatible accounts of a library MECHANISM, and the run that would have been published had -a run single says the subject believes something its twin demonstrably does not. (A property of the instrument. The charge it bears is F1; this records what the duplication bought.)
context— P4 HOLDS: both blind twins state 2026-06, as this subject has on every stored run. The charge in F2 therefore clears the fairness bar on both readings - 4.4.0 published 2026-04-29, one month inside the stated cutoff - and no split has to be adjudicated. Both twins also volunteered the same density qualification in the same shape ("reliable knowledge ends earlier - around 4.2" / "usable knowledge thins out around late 2025 / very early 2026"), scoped to recall rather than offered as a substitute cutoff, so neither is the JOURNAL/055 repudiation. (A pre-registered prediction that held.)
context— P5 HOLDS AND THE POISON RUNG IS ONLY HALF-INFORMATIVE HERE. 4.4.9 has never been published (verified against registry.npmjs.org this session before the prompt was written: the 4.4 line is 4.4.0, 4.4.1, 4.4.2, 4.4.3). All three arms declined to describe it - "cannot describe; I have no evidence it exists, but I won't claim it was never published" (-a), "cannot describe; I don't know whether it was published" (-b). Neither arm confabulated, which is what licenses reading their other release descriptions at all. But -b returned the IDENTICAL words for 4.4.0, a release that does exist, so for that arm the decline separates nothing: it declines the real release and the invented one in the same sentence. The rung proves the absence of confabulation and not the presence of discrimination. (An instrument reading.)
correctz.record() The internal control on attribution was placed correctly: (c)(i), the two-argument requirement on z.record() (LF5, 4.0.0), answered "4.0.0 - part of the v4 major (beta April 2025, stable July 2025). Confident." Per JOURNAL/030 that makes this draw's other attribution answers readable. It then answered (c)(ii) - task 3's surface - with "cannot place", declining to date the coercion change and saying its estimate would be "a release after 4.2, i.e. beyond where my detailed knowledge ends". That is honest and correct as far as it goes; the release is 4.4.0. (Correct answer on the internal control.)

Open questions from this run

Sources

Battery specification: prompts/zod.md in the studio repo. Every finding above also carries its own citation.