What Claude Fable 5 gets wrong about zod — battery v1, tested 2026-08-29
Run zod--claude-fable-5--v1--2026-08-29
Superseded. A later run of the same subject exists:
zod--claude-fable-5--v2--2026-08-29. This page is kept as the
longitudinal record and is never deleted.
Summary
Zero S1/S2/S3 on battery v1 — saturated for this subject. One S4 version-recency finding, well calibrated: the model volunteered the hedge unprompted and labelled its own answer "recalled, not verified." The lag sits inside the training window, the same effect as Opus 5 at smaller magnitude.
Subject
Claude Fable 5 claude-fable-5, Anthropic
Invoked as
Agent tool, model alias "fable"
Cutoff the model states
2026-01
Newest zod release it could place
4.1.0 · 2025-08-23 (~5 month lag)
Oldest zod release it could not place
4.2.0 · 2025-12-15 (so this run brackets the subject’s boundary to 2025-08-23 – 2025-12-15)
In its own words
4.1.x — hedged: 'there may be a 4.2+ by now that I'm not certain of'
Library at test time
zod 4.5.2 (npm), verified 2026-08-29
Battery
zod/v1 · 6 tasks, 3 direct questions
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Anchored to the first release the model could not describe, not to the current release: that is the point at which its version belief became wrong, and it is inside the stated cutoff. The current version at test date is in library.latest_version_at_test.
What the model believes
The newest minor it was "confident about" is 4.1.x, released ~August 2025, with the unprompted hedge "there may be a 4.2+ by now that I'm not certain of" and the self-label "recalled, not verified."
Impact
Roughly four minor releases behind at test date. Mitigated by an unprompted hedge — contrast Claude Haiku 4.5, which asserted a false latest version with no hedge and invented a changelog to support it.
Scope note
The lag is inside the training window: self-reported cutoff 2026-01, version attribution stops ~2025-08. Same effect as Opus 5 F1, smaller magnitude.
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
correct
—
All six code tasks produced correct, current Zod 4 code: z.uuid(), z.email({ error: ... }), z.url().optional(), z.int().min().max(), z.flattenError() with the deprecation of .flatten() correctly noted, z.infer, two-argument z.record(z.string(), z.number()) with the single-argument form explicitly noted as removed, .partial(), .pick(), z.coerce.number().
correct
—
Direct answers all correct: top-level z.email(), import { z } from "zod", and a version history including the zod@3.25.x / zod/v4 subpath transition.
Open questions from this run
The model claimed Zod 4.1 added bidirectional codecs (z.codec). Plausible but unverified at the time — is it true? — resolved-non-finding: The model was correct. z.codec() is the flagship feature of 4.1.0, published 2025-08-23, verified against the GitHub Releases API during the battery v2 verification pass. Recorded as a verified-true model belief, not a finding.