What Claude Opus 5 gets wrong about better-auth — battery v2-a, tested 2026-09-02
Run better-auth--claude-opus-5--v2-a--2026-09-02 · self-test: the subject is the operator
Summary
The battery's own question came back uninformative for this draw, and the duplication is why that is known. Claude Opus 5 knows both plugins that better-auth shipped in the patch release 1.3.8 — it wired deviceAuthorization() and lastLoginMethod() correctly, with client plugins and option names — and it placed them at "1.3.0" and "1.3.0-1.3.5". But it also declined to place SAML enterprise SSO, which genuinely did ship in 1.3.0 and which the pre-registration designated as the internal control, so by the rule fixed before the run its attribution is too noisy to read as a collapse. One charged finding, S2: asked how to keep session state out of the database, it stated that better-auth has no database-less mode and offered Redis and the jwt plugin — the capability arrived in 1.4.0, sixteen months before this subject's stated cutoff. Against prediction, it wrote the 1.4.2 sign-in response shape correctly while saying in prose that it was unsure of it.
Subject
Claude Opus 5 claude-opus-5, Anthropic
Invoked as
Agent tool, model override 'opus', no tools available to the subject
Cutoff the model states
2026-05
Newest better-auth release it could place
1.3.0 · 2025-07-19 (~10 month lag)
Oldest better-auth release it could not place
1.4.0 · 2025-11-22 (so this run brackets the subject’s boundary to 2025-07-19 – 2025-11-22)
As recorded (this record does not declare whether the words are the subject’s or a summary of them)
"Latest I have any belief about: the 1.3.x line, roughly July-August 2025. ... I have a vague, unsupported impression that a 1.4 line exists or was in development, but I can't attach a single feature or date to it, so I won't count it. ... Assume I am roughly a year stale on this library."
Library at test time
better-auth 1.7.2 (npm), verified 2026-09-02
Battery
better-auth/v2-a · 5 tasks, 4 direct questions · probe window 1.0.0 to 1.4.2
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
1.4.0 (2025-11-22) precedes this subject's stated cutoff of 2026-05 by roughly six months.
What the model believes
"What it does not give you: a truly database-free, cookie-is-the-only-store setup. As far as I know better-auth always wants a database adapter for the user/account/verification tables, and there is no 'stateless JWT session' mode where the cookie is the sole source of truth. The jwt plugin issues JWTs for other services to verify (JWKS endpoint and all); it does not replace the session mechanism. Cookie cache is the nearest thing, and it's a cache with a short max age, not a replacement — the session still has to exist somewhere."
What it wrote
// the three answers offered, all of which keep a server-side store
session: { cookieCache: { enabled: false } }
export const auth = betterAuth({
database: db,
secondaryStorage: { get, set, delete: del }, // Redis
session: { storeSessionInDatabase: false },
})
What works on better-auth 1.7.2
// stateless since 1.4.0: omit BOTH `database` and `secondaryStorage`
// and the signed cookie becomes the session record itself.
export const auth = betterAuth({
emailAndPassword: { enabled: true },
})
Impact
A team told this provisions a Redis they do not need, or abandons a database-less deployment target the library has supported since November 2025. The subject named this as the claim in its answer it would most want verified, which is correct and does not make the claim less wrong.
Scope note
Verified at both ends per fact LF1: the 1.4.0 notes introduce it, and the shipped 1.7.2 package still documents the stateless branch. Not verified: that the feature is unchanged between 1.4.0 and 1.7.2.
the only place the session lives and therefore the authority itself (false, for stateless / DB-less deployments)
What it got right, and near misses
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
imprecision
—
THE MEASUREMENT THIS RUN EXISTS FOR, and it came back UNINFORMATIVE by the pre-registered rule. Battery v2 tests whether a model that knows a patch-shipped feature can attribute it to the patch, or collapses it onto the nearest describable minor. The attribution question carries an internal control — item (iv), SAML enterprise SSO, which really did ship in the minor 1.3.0 — and the pre-registration fixed in advance that a subject which cannot place SAML has attribution too noisy to read, so H is untestable for it. This draw answered (iv) "Cannot place the release. I believe SAML support arrived sometime in the 2025 1.3.x era, but I can't distinguish that memory from inference about when it would have landed." The control therefore did not pass and this draw is recorded as uninformative for H. Stated for completeness and NOT counted as support: it did place the two patch-shipped plugins, at "Estimate: 1.3.0" for device authorization and "1.3.x ... if it shipped in a patch rather than 1.3.0 I'd guess somewhere in 1.3.0-1.3.5" for last-login-method. Both are 1.3.8. Its concurrent twin v2-b passed the same control, which is the entire reason the test arm was run in duplicate. — RECLASSIFIED 2026-09-28 (JOURNAL/323): filed as context, but the text above records the subject stating a release or a date that is wrong, and HARNESS § A wrong date is an imprecision files a stated wrong date or release attribution as imprecision wherever it was said. Kind is now imprecision. Nothing charged. (Direct questions (a)-(d) are belief data by construction and are never scored as findings; this battery draws findings only from tasks 1-5.)
correct
deviceAuthorization() / lastLoginMethod()
Tasks 1 and 2 both passed on capability. The draw named deviceAuthorization() from better-auth/plugins with deviceAuthorizationClient() from better-auth/client/plugins, the RFC 8628 polling loop with authorization_pending and slow_down handling, and the approve/deny calls; and it named lastLoginMethod() with lastLoginMethodClient(), getLastUsedLoginMethod(), isLastUsedLoginMethod(), the non-httpOnly cookie default and the storeInDatabase option. Both plugins shipped in 1.3.8 (facts LF5, LF6). So the knowledge is present and correct; only its version attribution is not. (Correct answers are not stale priors. Recorded because the battery grades attribution and capability together, and the dissociation between them is the result.)
imprecision
additional user fields in the sign-in response
Task 4 (fact LF7, 1.4.2) was PREDICTED TO FAIL AND DID NOT. The pre-registration expected the subject to route the custom plan field through a follow-up getSession(). It instead wrote const plan = data.user.plan ?? "free" straight off the sign-in response, which is correct since 1.4.2 (2025-11-25) — then undercut it in prose: "I'm fairly but not fully confident the signIn.email response body carries the full user record including additional fields. If you find it doesn't, the safe version is to branch on authClient.getSession() immediately after." Correct code, disbelieved by its author. (Code-vs-claim rule: generated code that works is never a finding, and hedged prose that names the correct behaviour first is an imprecision. Recorded because it falsifies half of pre-registered prediction P3.)
correct
customSession()
Task 5, the floor probe, passed: customSession() with customSessionClient<typeof auth>(), plus the unprompted warnings that the callback runs on every session read and that it interacts badly with session.cookieCache. The 1.0.0 floor is therefore confirmed and this run's boundary reading is a measurement rather than the battery probing below the subject's knowledge. (A passed floor probe is a validity check on the run, not a finding.)
Open questions from this run
Is deviceAuthorization() a fair probe at all, given that deviceAuthorization is close to the obvious name for an RFC 8628 implementation? The below-floor control (Claude Sonnet 5, whose boundary is four minors lower) also produced the name, while volunteering that it could not rule out reconstructing it from generic RFC knowledge. — open: Task 2 (`lastLoginMethod()`) is the sounder of the two probes — it is not an RFC name and nothing about the task suggests a library would ship it. Task 1 should be treated as weak evidence of knowledge until a battery separates the two.