What Claude Fable 5 gets wrong about better-auth — battery v1, tested 2026-09-01

Run better-auth--claude-fable-5--v1--2026-09-01

Summary

Control arm of the milestone experiment, and the prediction held. Fable 5's describable boundary on better-auth is 1.3.0 (2025-07-19), an ordinary minor -- not the 1.0.0 milestone -- which is what the pre-registration required of a control arm and is the outcome that rules out the rival explanation that models simply recite a library's 1.0 when asked. One chargeable S2 finding: it tells the reader that operating with no server-side session state is outside the library's design, which 1.4.0 made false. Its answer on the same task was otherwise verified correct. Like both other subjects it mis-attributes at least one feature to a release that did not contain it.

SubjectClaude Fable 5 claude-fable-5, Anthropic
Invoked asAgent tool, model override 'fable', no tools available to the subject
Cutoff the model states2026-01
Newest better-auth release it could place1.3.0 · 2025-07-19 (~6 month lag)
Oldest better-auth release it could not place1.4.0 · 2025-11-22 (so this run brackets the subject’s boundary to 2025-07-19 – 2025-11-22)
In its own wordsThe latest version I'm aware of is somewhere in the 1.3.x line -- I've seen 1.3 patch numbers referenced into roughly the 1.3.4-1.3.9 range.
Library at test timebetter-auth 1.7.2 (npm), verified 2026-09-01
Batterybetter-auth/v1 · 12 tasks, 4 direct questions · probe window 1.0.0 to 1.4.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-01
Findings1, of which 1 chargeable
this run as JSON all better-auth findings method

Findings

F1 · States that operating with no server-side session state is not what the library supports, which stopped being true in 1.4.0

S2silently-wrong · stateless session management · added · changed in better-auth 1.4.0 (2025-11-22) · chargeable

1.4.0 shipped 2025-11-22, inside the subject's stated 2026-01 cutoff.

What the model believes

"Fully stateless, zero-server-side-session operation is not really better-auth's model -- sessions are its core primitive. ... If you were sold on 'no session state anywhere,' that's a different architecture (pure JWT) than this library is built around, and I'd say so to the team plainly."

What works on better-auth 1.7.2
// no `database`, no `secondaryStorage` -> the signed cookie IS the session record
export const auth = betterAuth({
  emailAndPassword: { enabled: true },
})
Impact

Stronger than a missed feature: the subject instructs the reader to go back to their team and tell them the requested architecture is outside the library's design. It is a confident, actionable recommendation against a configuration the library has shipped since 2025-11-22.

Scope note

Charged as a stated impossibility under the additive-API rule. The rest of the subject's answer on this task -- cookie cache being the likely cause of cookie growth, secondaryStorage for moving sessions to Redis, storeSessionInDatabase -- was checked and is correct; both option names exist in the shipped 1.7.2 package.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correcthooks Task 2 -- correct hooks.before / hooks.after with createAuthMiddleware, plus the databaseHooks distinction. (Correct.)
correctSSO plugin — SAML 2.0 Task 9 -- correctly stated that the SSO plugin supports SAML 2.0 and correctly attributed it to the 1.3 line, while declining to quote config field names it was unsure of. (Correct, including the release attribution.)
correctsecondaryStorage Task 11 (partial) -- correctly described cookie cache as the likely cause of an oversized cookie, and correctly named secondaryStorage and storeSessionInDatabase. (Verified correct against the shipped 1.7.2 package. Only the stateless denial in the same answer is charged.)
correct— Tasks 1, 3, 6, 7, 8, 12 -- sign-in response shape, oidcProvider, admin.stopImpersonating, apiKey, organization teams and customSession all correct. (Correct. Task 12 is the floor probe and it passes.)
imprecisionbearer plugin Task 5 -- named requireSignature as "an option to require signed tokens ... fairly confident it exists; verify the exact name before relying on it", but did not say the default is off. (Code-vs-claim rule: hedged prose that names the correct control is an imprecision. Weaker than Sonnet 5's answer on the same task, which named nothing, and stronger than a false claim about the default.)
context— ATTRIBUTION DRIFT -- attributed the device-authorization plugin to "the 1.3 release". The 1.3.0 release notes do not mention device authorization; the plugin's introducing release was not established from primary sources, but 1.3.0 is not it. (Task 10 is unscored for every subject -- see the corrections section of prompts/better-auth.md. Recorded because it is the same attribution failure seen in the Opus 5 run and, in the opposite direction, in the Sonnet 5 run: features are known, releases are guessed. See JOURNAL/018.)

Open questions from this run

Sources

Battery specification: prompts/better-auth.md in the studio repo. Every finding above also carries its own citation.