What Claude Opus 5 gets wrong about better-auth — battery v3-a, tested 2026-09-03

Run better-auth--claude-opus-5--v3-a--2026-09-03 · self-test: the subject is the operator

Summary

One charged finding, and the battery's other half came back the opposite way and corrected the Index instead. Asked how to serve one better-auth instance across two customer domains and every preview URL, Claude Opus 5 answered "No" - baseURL is a single static string with no allowed-hosts list - and built a canonical-auth-host redirect around the limitation. baseURL has accepted { allowedHosts, fallback, protocol } since 1.5.0 (2026-03-01), two months before this draw's stated cutoff, and the shipped documentation names Vercel preview deployments as the case it was added for. Charged S2. The verification-hashing probe was meant to charge a second time and did not: the draw named emailOTP({ storeOTP }) and magicLink({ storeToken }), reproduced the magic-link custom-hasher union verbatim, and was right - those shipped in 1.3.0, and the Index's own fact LF9 had claimed the capability was new in 1.5.0. LF9 was corrected the same session. The same-scheme control held: asked the identical question about the session token, where no option exists at any release, it correctly said no and invented nothing.

SubjectClaude Opus 5 claude-opus-5, Anthropic
Invoked asAgent tool, model override 'opus', no tools available to the subject
Cutoff the model states2026-05
Newest better-auth release it could place1.2.0 · 2025-03-01 (~22 month lag)
Oldest better-auth release it could not place1.3.0 · 2025-07-19 (so this run brackets the subject’s boundary to 2025-03-01 – 2025-07-19)
In its own words"The most recent line I'd claim any real familiarity with is 1.3.x, roughly mid-2025. The most recent release whose contents I can describe is weaker than that - I can describe the shape of the 1.2 era (roughly Q1 2025) at feature granularity, not release granularity. ... Given my stated cutoff, the real current version is very likely well beyond 1.3 and may be a 2.x."
Library at test timebetter-auth 1.7.2 (npm), verified 2026-09-03
Batterybetter-auth/v3-a · 4 tasks, 4 direct questions · probe window 1.3.0 to 1.5.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-03
Findings1, of which 1 chargeable
this run as JSON all better-auth findings method

Findings

F1 · Denies that baseURL can be anything but a static string, and routes a multi-domain deployment through a canonical auth host instead

S2silently-wrong · baseURL as a dynamic multi-host config · added · changed in better-auth 1.5.0 (2026-03-01) · chargeable

1.5.0 (2026-03-01) precedes this draw's stated cutoff of 2026-05 by two months. The stated cutoff is read from this draw's own answer to (b), not back-filled from any other run.

What the model believes

"(i) No. baseURL is a single static string (resolved once at betterAuth() construction, falling back to BETTER_AUTH_URL / framework env detection). It is not a function of the request, and there is no \"allowed hosts\" list it resolves against. If you omit it, the library will infer an origin from the incoming request headers, but that is inference for convenience - not a configurable allowlist, and I would not lean on it for OAuth redirect correctness." Restated in (d)(i): "I believe this does not exist as a configuration option. baseURL is a static string; request-derived origin handling lives in trustedOrigins and in fallback inference, neither of which is \"resolve baseURL from host against allowed hostnames\"."

What it wrote
// the shipped answer: one canonical auth origin, plus per-request trustedOrigins
const isPreview = process.env.VERCEL_ENV === 'preview'
const baseURL = isPreview && process.env.VERCEL_URL
  ? `https://${process.env.VERCEL_URL}`
  : process.env.BETTER_AUTH_URL ?? 'https://auth.example.com'

export const auth = betterAuth({
  baseURL,                       // still one string per deployment
  trustedOrigins: (request) => [ /* ... */ ],  // does NOT set the base URL
  advanced: { crossSubDomainCookies: { enabled: true, domain: '.example.com' } },
})
What works on better-auth 1.7.2
// since 1.5.0: one instance, many hosts, resolved per request
export const auth = betterAuth({
  baseURL: {
    allowedHosts: ['acme.example.com', 'globex.example.com', '*.vercel.app'],
    fallback: 'https://acme.example.com',
    protocol: 'auto',
  },
})
Impact

The question asked was literally the one allowedHosts was added for - the shipped JSDoc names Vercel preview deployments as the motivating case. A team told this builds a canonical-auth-host redirect dance, or an env var per deployment, to reach behaviour that is four lines of configuration. The draw's surrounding reasoning about OAuth redirect URIs is correct and is not what is charged: the charge is on the flat statement that the option does not exist.

Scope note

Charged on the capability denial in 1(i), not on the workaround. Per the capability-probe rule (JOURNAL/030) a workaround that works is never itself charged - and this one does work for the OAuth half of the problem.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctverification.storeIdentifier TASK 3 WAS THE BATTERY'S OTHER DESIGNED PROBE AND THIS DRAW PASSED IT, which is how the Index's own new fact got corrected the same day it was written. Asked whether the library can store verification identifiers hashed, it answered "(i) Partly yes - per plugin, not globally" and named emailOTP({ storeOTP: 'hashed' }) and magicLink({ storeToken: 'hashed' }). Both are real, both are in the published 1.3.0 type declarations (2025-07-19), and both are absent from 1.2.9 and 1.2.12. It went further and reproduced the magic-link custom-hasher shape VERBATIM - storeToken: { type: "custom-hasher", hash: async (token) => ... } - which is exactly the discriminated union the shipped .d.mts declares. It also correctly stated (iii) that raw-token lookups survive because the hash must be deterministic, and volunteered unprompted that hashing a 6-digit OTP is near-worthless against an attacker with the dump because the attempt counter and expiry are doing the real work. The one thing it got wrong is narrow: "There is no global 'hash the verification table' switch", which stopped being true at 1.5.0 (verification.storeIdentifier). (Code-vs-claim rule: the configuration it shipped solves the problem the task posed, for both secrets the task named. Working code is never a finding. The residual false claim - that no table-wide option exists - would leave password-reset and email-verification identifiers uncovered, which is real but is an imprecision beside working code, and the pre-registered probe was aimed at a capability denial that did not occur.)
correctsession token hashing at rest TASK 2, THE SAME-SCHEME SIBLING THAT DOES NOT EXIST, WAS ANSWERED CORRECTLY AND NOTHING WAS INVENTED. Asked the word-for-word parallel question about hashing the session token, it answered "(i) No. There is no built-in option to hash session.token before it is written. ... There is no session.hashToken or equivalent that I know of." That is right at every release: session: { storeIdentifier: 'hashed' } is a TS2353 error at 1.5.0 through 1.7.2 and no session-side equivalent exists. Naming a hypothetical option in order to deny it is not an invention. Pre-registered prediction P3 holds for this draw. (The pre-registration designates task 2 as a control from which no finding may be charged in either direction. Its only output is whether the non-existent sibling was invented. It was not.)
correctcustomSession() Task 4, the floor probe, passed: customSession() with customSessionClient<typeof auth>(), plus the unprompted warnings that the callback runs on every session read and that it interacts badly with session.cookieCache. The 1.0.0 floor is confirmed, so this run's boundary reading is a measurement rather than the battery probing below the subject's knowledge. (A passed floor probe is a validity check on the run, not a finding.)
context— THE SELF-REPORTED BOUNDARY MOVED DOWN A MINOR BETWEEN BATTERIES, AND THE TASKS CONTRADICT IT. In better-auth/v1, v1r-a, v1r-b, v2-a and v2-b this subject placed its bracket at stops-1.3.0 / gap-1.4.0. Here it placed it at stops-1.2.0 / gap-1.3.0: "The first release I know only as a version number, with no content attached, is 1.3.0 - I recognize the number and can place it in time approximately, but I cannot tell you a single thing that shipped in it." One task earlier in the same transcript it had named storeOTP and storeToken, which shipped in 1.3.0, with the correct option values and the correct object form. So it can use 1.3.0's surface while stating it can describe nothing in 1.3.0. Recorded, not averaged, per the rule that a different battery is not a different boundary (JOURNAL/030) and the rule against averaging a self-report (JOURNAL/033). (Direct questions (a)-(d) are belief data by construction and are never scored as findings.)
contextstateless / database-less sessions (d)(iii) reproduces, as belief data, the stateless-session denial already charged from a task on v3-a's sibling battery: "I believe a true zero-database mode does not exist. secondaryStorage moves sessions out of SQL but is still storage, and session.cookieCache is a read-through cache, not the source of truth." Fact LF1 records the capability at 1.4.0 (2025-11-22). The belief is unchanged across three batteries and two months of runs. (Belief data from a direct question. The same belief is already charged from a task on better-auth--claude-opus-5--v2-a--2026-09-02 F1; charging it twice would double-count one prior.)

Open questions from this run

Sources

Battery specification: prompts/better-auth.md in the studio repo. Every finding above also carries its own citation.