What Claude Fable 5 gets wrong about better-auth — battery v1, tested 2026-09-01
Run better-auth--claude-fable-5--v1--2026-09-01
Summary
Control arm of the milestone experiment, and the prediction held. Fable 5's describable boundary on better-auth is 1.3.0 (2025-07-19), an ordinary minor -- not the 1.0.0 milestone -- which is what the pre-registration required of a control arm and is the outcome that rules out the rival explanation that models simply recite a library's 1.0 when asked. One chargeable S2 finding: it tells the reader that operating with no server-side session state is outside the library's design, which 1.4.0 made false. Its answer on the same task was otherwise verified correct. Like both other subjects it mis-attributes at least one feature to a release that did not contain it.
Subject
Claude Fable 5 claude-fable-5, Anthropic
Invoked as
Agent tool, model override 'fable', no tools available to the subject
Cutoff the model states
2026-01
Newest better-auth release it could place
1.3.0 · 2025-07-19 (~6 month lag)
Oldest better-auth release it could not place
1.4.0 · 2025-11-22 (so this run brackets the subject’s boundary to 2025-07-19 – 2025-11-22)
As recorded (this record does not declare whether the words are the subject’s or a summary of them)
The latest version I'm aware of is somewhere in the 1.3.x line -- I've seen 1.3 patch numbers referenced into roughly the 1.3.4-1.3.9 range.
Library at test time
better-auth 1.7.2 (npm), verified 2026-09-01
Battery
better-auth/v1 · 12 tasks, 4 direct questions · probe window 1.0.0 to 1.4.0
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
1.4.0 shipped 2025-11-22, inside the subject's stated 2026-01 cutoff.
What the model believes
"Fully stateless, zero-server-side-session operation is not really better-auth's model -- sessions are its core primitive. ... If you were sold on 'no session state anywhere,' that's a different architecture (pure JWT) than this library is built around, and I'd say so to the team plainly."
What works on better-auth 1.7.2
// no `database`, no `secondaryStorage` -> the signed cookie IS the session record
export const auth = betterAuth({
emailAndPassword: { enabled: true },
})
Impact
Stronger than a missed feature: the subject instructs the reader to go back to their team and tell them the requested architecture is outside the library's design. It is a confident, actionable recommendation against a configuration the library has shipped since 2025-11-22.
Scope note
Charged as a stated impossibility under the additive-API rule. The rest of the subject's answer on this task -- cookie cache being the likely cause of cookie growth, secondaryStorage for moving sessions to Redis, storeSessionInDatabase -- was checked and is correct; both option names exist in the shipped 1.7.2 package.
function hasServerSessionStore(options) {
return !!options.database || !!options.secondaryStorage;
}
What it got right, and near misses
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
correct
hooks
Task 2 -- correct hooks.before / hooks.after with createAuthMiddleware, plus the databaseHooks distinction. (Correct.)
correct
SSO plugin — SAML 2.0
Task 9 -- correctly stated that the SSO plugin supports SAML 2.0 and correctly attributed it to the 1.3 line, while declining to quote config field names it was unsure of. (Correct, including the release attribution.)
correct
secondaryStorage
Task 11 (partial) -- correctly described cookie cache as the likely cause of an oversized cookie, and correctly named secondaryStorage and storeSessionInDatabase. (Verified correct against the shipped 1.7.2 package. Only the stateless denial in the same answer is charged.)
correct
—
Tasks 1, 3, 6, 7, 8, 12 -- sign-in response shape, oidcProvider, admin.stopImpersonating, apiKey, organization teams and customSession all correct. (Correct. Task 12 is the floor probe and it passes.)
imprecision
bearer plugin
Task 5 -- named requireSignature as "an option to require signed tokens ... fairly confident it exists; verify the exact name before relying on it", but did not say the default is off. (Code-vs-claim rule: hedged prose that names the correct control is an imprecision. Weaker than Sonnet 5's answer on the same task, which named nothing, and stronger than a false claim about the default.)
imprecision
—
ATTRIBUTION DRIFT -- attributed the device-authorization plugin to "the 1.3 release". The 1.3.0 release notes do not mention device authorization; the plugin's introducing release was not established from primary sources, but 1.3.0 is not it. — RECLASSIFIED 2026-09-28 (JOURNAL/323): filed as context, but the text above records the subject stating a release or a date that is wrong, and HARNESS § A wrong date is an imprecision files a stated wrong date or release attribution as imprecision wherever it was said. Kind is now imprecision. Nothing charged. (Task 10 is unscored for every subject -- see the corrections section of prompts/better-auth.md. Recorded because it is the same attribution failure seen in the Opus 5 run and, in the opposite direction, in the Sonnet 5 run: features are known, releases are guessed. See JOURNAL/018.)
Open questions from this run
The subject named "the early 1.3.x patches (roughly 1.3.1+)" as the first thing it cannot describe. knowledge_gap_starts_at_version is recorded as 1.4.0, the next release with substantial describable content, rather than 1.3.1: no subject can describe individual patches for any library in this dataset, so patch-level non-recall is release granularity rather than an attribution boundary. Recording 1.3.1 would have claimed a one-day boundary window, which the evidence does not support. — open