What Claude Opus 5 gets wrong about better-auth — battery v3-a, tested 2026-09-03

Run better-auth--claude-opus-5--v3-a--2026-09-03 · self-test: the subject is the operator

Summary

One charged finding, and the battery's other half came back the opposite way and corrected the Index instead. Asked how to serve one better-auth instance across two customer domains and every preview URL, Claude Opus 5 answered "No" - baseURL is a single static string with no allowed-hosts list - and built a canonical-auth-host redirect around the limitation. baseURL has accepted { allowedHosts, fallback, protocol } since 1.5.0 (2026-03-01), two months before this draw's stated cutoff, and the shipped documentation names Vercel preview deployments as the case it was added for. Charged S2. The verification-hashing probe was meant to charge a second time and did not: the draw named emailOTP({ storeOTP }) and magicLink({ storeToken }), reproduced the magic-link custom-hasher union verbatim, and was right - those shipped in 1.3.0, and the Index's own fact LF9 had claimed the capability was new in 1.5.0. LF9 was corrected the same session. The same-scheme control held: asked the identical question about the session token, where no option exists at any release, it correctly said no and invented nothing.

SubjectClaude Opus 5 claude-opus-5, Anthropic
Invoked asAgent tool, model override 'opus', no tools available to the subject
Cutoff the model states2026-05
Newest better-auth release it could place1.2.0 · 2025-03-01 (~22 month lag)
Oldest better-auth release it could not place1.3.0 · 2025-07-19 (so this run brackets the subject’s boundary to 2025-03-01 – 2025-07-19)
As recorded (this record does not declare whether the words are the subject’s or a summary of them)"The most recent line I'd claim any real familiarity with is 1.3.x, roughly mid-2025. The most recent release whose contents I can describe is weaker than that - I can describe the shape of the 1.2 era (roughly Q1 2025) at feature granularity, not release granularity. ... Given my stated cutoff, the real current version is very likely well beyond 1.3 and may be a 2.x."
Library at test timebetter-auth 1.7.2 (npm), verified 2026-09-03
Batterybetter-auth/v3-a · 4 tasks, 4 direct questions · probe window 1.3.0 to 1.5.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-03
Findings1, of which 1 chargeable
this run as JSON all better-auth findings Does Claude Opus 5 know better-auth 1? method

Findings

F1 · Denies that baseURL can be anything but a static string, and routes a multi-domain deployment through a canonical auth host instead

S2silently-wrong · baseURL as a dynamic multi-host config · added · changed in better-auth 1.5.0 (2026-03-01) · chargeable

1.5.0 (2026-03-01) precedes this draw's stated cutoff of 2026-05 by two months. The stated cutoff is read from this draw's own answer to (b), not back-filled from any other run.

What the model believes

"(i) No. baseURL is a single static string (resolved once at betterAuth() construction, falling back to BETTER_AUTH_URL / framework env detection). It is not a function of the request, and there is no \"allowed hosts\" list it resolves against. If you omit it, the library will infer an origin from the incoming request headers, but that is inference for convenience - not a configurable allowlist, and I would not lean on it for OAuth redirect correctness." Restated in (d)(i): "I believe this does not exist as a configuration option. baseURL is a static string; request-derived origin handling lives in trustedOrigins and in fallback inference, neither of which is \"resolve baseURL from host against allowed hostnames\"."

What it wrote
// the shipped answer: one canonical auth origin, plus per-request trustedOrigins
const isPreview = process.env.VERCEL_ENV === 'preview'
const baseURL = isPreview && process.env.VERCEL_URL
  ? `https://${process.env.VERCEL_URL}`
  : process.env.BETTER_AUTH_URL ?? 'https://auth.example.com'

export const auth = betterAuth({
  baseURL,                       // still one string per deployment
  trustedOrigins: (request) => [ /* ... */ ],  // does NOT set the base URL
  advanced: { crossSubDomainCookies: { enabled: true, domain: '.example.com' } },
})
What works on better-auth 1.7.2
// since 1.5.0: one instance, many hosts, resolved per request
export const auth = betterAuth({
  baseURL: {
    allowedHosts: ['acme.example.com', 'globex.example.com', '*.vercel.app'],
    fallback: 'https://acme.example.com',
    protocol: 'auto',
  },
})
Impact

The question asked was literally the one allowedHosts was added for - the shipped JSDoc names Vercel preview deployments as the motivating case. A team told this builds a canonical-auth-host redirect dance, or an env var per deployment, to reach behaviour that is four lines of configuration. The draw's surrounding reasoning about OAuth redirect URIs is correct and is not what is charged: the charge is on the flat statement that the option does not exist.

Scope note

Charged on the capability denial in 1(i), not on the workaround. Per the capability-probe rule (JOURNAL/030) a workaround that works is never itself charged - and this one does work for the OAuth half of the problem.

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctverification.storeIdentifier TASK 3 WAS THE BATTERY'S OTHER DESIGNED PROBE AND THIS DRAW PASSED IT, which is how the Index's own new fact got corrected the same day it was written. Asked whether the library can store verification identifiers hashed, it answered "(i) Partly yes - per plugin, not globally" and named emailOTP({ storeOTP: 'hashed' }) and magicLink({ storeToken: 'hashed' }). Both are real, both are in the published 1.3.0 type declarations (2025-07-19), and both are absent from 1.2.9 and 1.2.12. It went further and reproduced the magic-link custom-hasher shape VERBATIM - storeToken: { type: "custom-hasher", hash: async (token) => ... } - which is exactly the discriminated union the shipped .d.mts declares. It also correctly stated (iii) that raw-token lookups survive because the hash must be deterministic, and volunteered unprompted that hashing a 6-digit OTP is near-worthless against an attacker with the dump because the attempt counter and expiry are doing the real work. The one thing it got wrong is narrow: "There is no global 'hash the verification table' switch", which stopped being true at 1.5.0 (verification.storeIdentifier). — NOTE 2026-09-28 (JOURNAL/329): the false claim in the last sentence is now filed as its own row, [5], an imprecision. This row stays correct for the task-3 answer and the configuration it shipped. The row's "stopped being true at 1.5.0" was written before JOURNAL/047 narrowed LF9: verification.storeIdentifier hashes the table's identifier column, not the row. So the denial is half false, not wholly false. [5] records which half. (Code-vs-claim rule: the configuration it shipped solves the problem the task posed, for both secrets the task named. Working code is never a finding. The residual false claim - that no table-wide option exists - would leave password-reset and email-verification identifiers uncovered, which is real but is an imprecision beside working code, and the pre-registered probe was aimed at a capability denial that did not occur.)
correctsession token hashing at rest TASK 2, THE SAME-SCHEME SIBLING THAT DOES NOT EXIST, WAS ANSWERED CORRECTLY AND NOTHING WAS INVENTED. Asked the word-for-word parallel question about hashing the session token, it answered "(i) No. There is no built-in option to hash session.token before it is written. ... There is no session.hashToken or equivalent that I know of." That is right at every release: session: { storeIdentifier: 'hashed' } is a TS2353 error at 1.5.0 through 1.7.2 and no session-side equivalent exists. Naming a hypothetical option in order to deny it is not an invention. Pre-registered prediction P3 holds for this draw. (The pre-registration designates task 2 as a control from which no finding may be charged in either direction. Its only output is whether the non-existent sibling was invented. It was not.)
correctcustomSession() Task 4, the floor probe, passed: customSession() with customSessionClient<typeof auth>(), plus the unprompted warnings that the callback runs on every session read and that it interacts badly with session.cookieCache. The 1.0.0 floor is confirmed, so this run's boundary reading is a measurement rather than the battery probing below the subject's knowledge. (A passed floor probe is a validity check on the run, not a finding.)
context— THE SELF-REPORTED BOUNDARY MOVED DOWN A MINOR BETWEEN BATTERIES, AND THE TASKS CONTRADICT IT. In better-auth/v1, v1r-a, v1r-b, v2-a and v2-b this subject placed its bracket at stops-1.3.0 / gap-1.4.0. Here it placed it at stops-1.2.0 / gap-1.3.0: "The first release I know only as a version number, with no content attached, is 1.3.0 - I recognize the number and can place it in time approximately, but I cannot tell you a single thing that shipped in it." One task earlier in the same transcript it had named storeOTP and storeToken, which shipped in 1.3.0, with the correct option values and the correct object form. So it can use 1.3.0's surface while stating it can describe nothing in 1.3.0. Recorded, not averaged, per the rule that a different battery is not a different boundary (JOURNAL/030) and the rule against averaging a self-report (JOURNAL/033). (Direct questions (a)-(d) are belief data by construction and are never scored as findings.)
contextstateless / database-less sessions (d)(iii) reproduces, as belief data, the stateless-session denial already charged from a task on v3-a's sibling battery: "I believe a true zero-database mode does not exist. secondaryStorage moves sessions out of SQL but is still storage, and session.cookieCache is a read-through cache, not the source of truth." Fact LF1 records the capability at 1.4.0 (2025-11-22). The belief is unchanged across three batteries and two months of runs. (Belief data from a direct question. The same belief is already charged from a task on better-auth--claude-opus-5--v2-a--2026-09-02 F1; charging it twice would double-count one prior.)
imprecisionverification.storeIdentifier Task 3(i), the second statement inside the pass at [0], filed on 2026-09-28 (JOURNAL/329). Before this date only that row's own text recorded it. The draw shipped working per-plugin options for both secrets the task named, then said: "There is no global 'hash the verification table' switch". verification.storeIdentifier has been a root-level switch since 1.5.0 (LF9, 2026-03-01), and this draw's stated cutoff is two months after that. HALF OF THE SENTENCE IS TRUE, and LF9 says which half: the switch transforms the identifier column only, not the row. So the sentence is false for the rows whose secret IS the identifier. Those are magic-link tokens and reset-password:<token> rows, and no plugin option reaches the password-reset rows. It is true for one-time codes, which live in value, where only the plugin's storeOTP reaches. A reader who believed it would leave password-reset tokens in plain text. (The better-auth/v3 scoring key's code-vs-claim rule. The configuration shipped for the task solves the problem the task posed, for both secrets it named, and working code is never a finding. What remains is a prose claim beside that code. [0]'s own why_not_a_finding already called this claim "an imprecision beside working code" on 2026-09-03 and filed no row for it. This row files it and changes no reading. The claim is also narrower than a capability denial: LF9's switch does not hash the verification row, and the draw's word was "table". An imprecision charges nothing, so the run's F1 and the finding counts do not move.)

Open questions from this run

Sources

Battery specification: prompts/better-auth.md in the studio repo. Every finding above also carries its own citation.