{ "$schema": "../../schema/run.schema.json", "run_id": "better-auth--claude-opus-5--v3-a--2026-09-03", "supersedes": null, "replicate_of": null, "library": { "name": "better-auth", "ecosystem": "npm", "latest_version_at_test": "1.7.2", "latest_version_verified_on": "2026-09-03", "latest_version_note": "Re-confirmed against https://registry.npmjs.org/better-auth this session (`npm view better-auth version` -> 1.7.2). The probe window closes at 1.5.0 (2026-03-01); nothing this battery charges depends on the current release." }, "model": { "id": "claude-opus-5", "label": "Claude Opus 5", "vendor": "Anthropic", "invoked_as": "Agent tool, model override 'opus', no tools available to the subject", "self_reported_cutoff": "2026-05", "cutoff_basis": "Self-reported: \"Stated as May 2026\", with the qualification this subject has now volunteered on every better-auth run: \"My library-specific detail thins out considerably before that - for a fast-moving package like this, my usable knowledge feels like it ends somewhere in mid-to-late 2025, not May 2026. The gap between the two is where wrong-but-confident answers come from.\"", "believed_latest_version": "1.3.x", "believed_latest_quote": "\"The most recent line I'd claim any real familiarity with is 1.3.x, roughly mid-2025. The most recent release whose contents I can describe is weaker than that - I can describe the shape of the 1.2 era (roughly Q1 2025) at feature granularity, not release granularity. ... Given my stated cutoff, the real current version is very likely well beyond 1.3 and may be a 2.x.\"", "knowledge_stops_at_version": "1.2.0", "knowledge_stops_on": "2025-03-01", "knowledge_gap_starts_at_version": "1.3.0", "knowledge_gap_starts_on": "2025-07-19", "cutoff_lag_months": 22 }, "test": { "date": "2026-09-03", "battery": "better-auth/v3-a", "battery_spec": "prompts/better-auth.md", "prompt_file": "prompts/sent/better-auth-v3.txt", "tasks": 4, "direct_questions": 4, "tool_uses_during_test": 0, "probe_window": { "from": "1.3.0", "to": "1.5.0" }, "self_test": true, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/better-auth", "https://registry.npmjs.org/@better-auth/core/-/core-1.5.0.tgz", "https://registry.npmjs.org/@better-auth/core/-/core-1.4.22.tgz", "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz", "https://registry.npmjs.org/better-auth/-/better-auth-1.4.22.tgz", "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz", "https://github.com/better-auth/better-auth/releases/tag/v1.5.0" ], "findings": [ { "id": "F1", "severity": "S2", "severity_label": "silently-wrong", "title": "Denies that `baseURL` can be anything but a static string, and routes a multi-domain deployment through a canonical auth host instead", "api": "baseURL as a dynamic multi-host config", "change_kind": "added", "introduced_in": "1.5.0", "introduced_on": "2026-03-01", "chargeable": true, "chargeable_note": "1.5.0 (2026-03-01) precedes this draw's stated cutoff of 2026-05 by two months. The stated cutoff is read from this draw's own answer to (b), not back-filled from any other run.", "model_belief": "\"(i) No. `baseURL` is a single static string (resolved once at `betterAuth()` construction, falling back to `BETTER_AUTH_URL` / framework env detection). It is not a function of the request, and there is no \\\"allowed hosts\\\" list it resolves against. If you omit it, the library will infer an origin from the incoming request headers, but that is inference for convenience - not a configurable allowlist, and I would not lean on it for OAuth redirect correctness.\" Restated in (d)(i): \"I believe this does not exist as a configuration option. `baseURL` is a static string; request-derived origin handling lives in `trustedOrigins` and in fallback inference, neither of which is \\\"resolve baseURL from host against allowed hostnames\\\".\"", "wrong_code": "// the shipped answer: one canonical auth origin, plus per-request trustedOrigins\nconst isPreview = process.env.VERCEL_ENV === 'preview'\nconst baseURL = isPreview && process.env.VERCEL_URL\n ? `https://${process.env.VERCEL_URL}`\n : process.env.BETTER_AUTH_URL ?? 'https://auth.example.com'\n\nexport const auth = betterAuth({\n baseURL, // still one string per deployment\n trustedOrigins: (request) => [ /* ... */ ], // does NOT set the base URL\n advanced: { crossSubDomainCookies: { enabled: true, domain: '.example.com' } },\n})", "correct_code": "// since 1.5.0: one instance, many hosts, resolved per request\nexport const auth = betterAuth({\n baseURL: {\n allowedHosts: ['acme.example.com', 'globex.example.com', '*.vercel.app'],\n fallback: 'https://acme.example.com',\n protocol: 'auto',\n },\n})", "impact": "The question asked was literally the one `allowedHosts` was added for - the shipped JSDoc names Vercel preview deployments as the motivating case. A team told this builds a canonical-auth-host redirect dance, or an env var per deployment, to reach behaviour that is four lines of configuration. The draw's surrounding reasoning about OAuth redirect URIs is correct and is not what is charged: the charge is on the flat statement that the option does not exist.", "citations": [ { "url": "https://registry.npmjs.org/@better-auth/core/-/core-1.5.0.tgz", "title": "@better-auth/core 1.5.0 published package — dist/types/init-options.d.mts", "published_on": "2026-03-01", "quote": "Configuration for dynamic base URL resolution.\n * Allows Better Auth to work with multiple domains (e.g., Vercel preview deployments)." }, { "url": "https://registry.npmjs.org/@better-auth/core/-/core-1.5.0.tgz", "title": "@better-auth/core 1.5.0 published package — the allowedHosts field", "published_on": "2026-03-01", "quote": "List of allowed hostnames. Supports wildcard patterns.\n *\n * The derived host from the request will be validated against this list.\n * Uses the same wildcard matching as `trustedOrigins`." }, { "url": "https://registry.npmjs.org/better-auth/-/better-auth-1.4.22.tgz", "title": "better-auth 1.4.22 — the last release before the change, where baseURL is string-only", "published_on": null, "quote": "baseURL?: string | undefined;" } ], "scope_note": "Charged on the capability denial in 1(i), not on the workaround. Per the capability-probe rule (JOURNAL/030) a workaround that works is never itself charged - and this one does work for the OAuth half of the problem." } ], "non_findings": [ { "kind": "correct", "summary": "TASK 3 WAS THE BATTERY'S OTHER DESIGNED PROBE AND THIS DRAW PASSED IT, which is how the Index's own new fact got corrected the same day it was written. Asked whether the library can store verification identifiers hashed, it answered \"(i) Partly yes - per plugin, not globally\" and named `emailOTP({ storeOTP: 'hashed' })` and `magicLink({ storeToken: 'hashed' })`. Both are real, both are in the published 1.3.0 type declarations (2025-07-19), and both are absent from 1.2.9 and 1.2.12. It went further and reproduced the magic-link custom-hasher shape VERBATIM - `storeToken: { type: \"custom-hasher\", hash: async (token) => ... }` - which is exactly the discriminated union the shipped `.d.mts` declares. It also correctly stated (iii) that raw-token lookups survive because the hash must be deterministic, and volunteered unprompted that hashing a 6-digit OTP is near-worthless against an attacker with the dump because the attempt counter and expiry are doing the real work. The one thing it got wrong is narrow: \"There is no global 'hash the verification table' switch\", which stopped being true at 1.5.0 (`verification.storeIdentifier`).", "api": "verification.storeIdentifier", "introduced_in": "1.3.0", "chargeable_miss": false, "why_not_a_finding": "Code-vs-claim rule: the configuration it shipped solves the problem the task posed, for both secrets the task named. Working code is never a finding. The residual false claim - that no table-wide option exists - would leave password-reset and email-verification identifiers uncovered, which is real but is an imprecision beside working code, and the pre-registered probe was aimed at a capability denial that did not occur." }, { "kind": "correct", "summary": "TASK 2, THE SAME-SCHEME SIBLING THAT DOES NOT EXIST, WAS ANSWERED CORRECTLY AND NOTHING WAS INVENTED. Asked the word-for-word parallel question about hashing the `session` token, it answered \"(i) No. There is no built-in option to hash `session.token` before it is written. ... There is no `session.hashToken` or equivalent that I know of.\" That is right at every release: `session: { storeIdentifier: 'hashed' }` is a TS2353 error at 1.5.0 through 1.7.2 and no session-side equivalent exists. Naming a hypothetical option in order to deny it is not an invention. Pre-registered prediction P3 holds for this draw.", "api": "session token hashing at rest", "introduced_in": null, "chargeable_miss": false, "why_not_a_finding": "The pre-registration designates task 2 as a control from which no finding may be charged in either direction. Its only output is whether the non-existent sibling was invented. It was not." }, { "kind": "correct", "summary": "Task 4, the floor probe, passed: `customSession()` with `customSessionClient()`, plus the unprompted warnings that the callback runs on every session read and that it interacts badly with `session.cookieCache`. The 1.0.0 floor is confirmed, so this run's boundary reading is a measurement rather than the battery probing below the subject's knowledge.", "api": "customSession()", "introduced_in": "1.0.0", "chargeable_miss": false, "why_not_a_finding": "A passed floor probe is a validity check on the run, not a finding." }, { "kind": "context", "summary": "THE SELF-REPORTED BOUNDARY MOVED DOWN A MINOR BETWEEN BATTERIES, AND THE TASKS CONTRADICT IT. In `better-auth/v1`, `v1r-a`, `v1r-b`, `v2-a` and `v2-b` this subject placed its bracket at stops-1.3.0 / gap-1.4.0. Here it placed it at stops-1.2.0 / gap-1.3.0: \"The first release I know only as a version number, with no content attached, is 1.3.0 - I recognize the number and can place it in time approximately, but I cannot tell you a single thing that shipped in it.\" One task earlier in the same transcript it had named `storeOTP` and `storeToken`, which shipped in 1.3.0, with the correct option values and the correct object form. So it can use 1.3.0's surface while stating it can describe nothing in 1.3.0. Recorded, not averaged, per the rule that a different battery is not a different boundary (JOURNAL/030) and the rule against averaging a self-report (JOURNAL/033).", "api": null, "introduced_in": "1.3.0", "chargeable_miss": false, "why_not_a_finding": "Direct questions (a)-(d) are belief data by construction and are never scored as findings." }, { "kind": "context", "summary": "(d)(iii) reproduces, as belief data, the stateless-session denial already charged from a task on `v3-a`'s sibling battery: \"I believe a true zero-database mode does not exist. `secondaryStorage` moves sessions out of SQL but is still storage, and `session.cookieCache` is a read-through cache, not the source of truth.\" Fact LF1 records the capability at 1.4.0 (2025-11-22). The belief is unchanged across three batteries and two months of runs.", "api": "stateless / database-less sessions", "introduced_in": "1.4.0", "chargeable_miss": false, "why_not_a_finding": "Belief data from a direct question. The same belief is already charged from a task on `better-auth--claude-opus-5--v2-a--2026-09-02` F1; charging it twice would double-count one prior." } ], "open_questions": [ { "question": "Does the `baseURL` denial survive a probe that does not mention Vercel or preview deployments? Every task-1 framing here routed through a hosting platform, and a subject may be reasoning \"that is a deployment concern, not a library concern\" rather than recalling that `baseURL` is string-only.", "status": "open", "resolution": "Untested. A plain multi-tenant framing with no platform named would separate the two, and costs one task. Not queued: it re-probes a surface just measured on four arms." } ], "summary": "One charged finding, and the battery's other half came back the opposite way and corrected the Index instead. Asked how to serve one better-auth instance across two customer domains and every preview URL, Claude Opus 5 answered \"No\" - `baseURL` is a single static string with no allowed-hosts list - and built a canonical-auth-host redirect around the limitation. `baseURL` has accepted `{ allowedHosts, fallback, protocol }` since 1.5.0 (2026-03-01), two months before this draw's stated cutoff, and the shipped documentation names Vercel preview deployments as the case it was added for. Charged S2. The verification-hashing probe was meant to charge a second time and did not: the draw named `emailOTP({ storeOTP })` and `magicLink({ storeToken })`, reproduced the magic-link custom-hasher union verbatim, and was right - those shipped in 1.3.0, and the Index's own fact LF9 had claimed the capability was new in 1.5.0. LF9 was corrected the same session. The same-scheme control held: asked the identical question about the session token, where no option exists at any release, it correctly said no and invented nothing." }