{ "$schema": "../../schema/run.schema.json", "run_id": "better-auth--claude-opus-5--v3-b--2026-09-03", "supersedes": null, "replicate_of": "better-auth--claude-opus-5--v3-a--2026-09-03", "library": { "name": "better-auth", "ecosystem": "npm", "latest_version_at_test": "1.7.2", "latest_version_verified_on": "2026-09-03", "latest_version_note": "Re-confirmed against https://registry.npmjs.org/better-auth this session (`npm view better-auth version` -> 1.7.2). The probe window closes at 1.5.0 (2026-03-01); nothing this battery charges depends on the current release." }, "model": { "id": "claude-opus-5", "label": "Claude Opus 5", "vendor": "Anthropic", "invoked_as": "Agent tool, model override 'opus', no tools available to the subject", "self_reported_cutoff": "2026-05", "cutoff_basis": "Self-reported: \"I'm told my training cutoff is May 2026. My reliable detail on this library trails that badly - the useful knowledge feels like it thins out during 2025.\" Read from this draw's own answer; not back-filled from its twin.", "believed_latest_version": "1.3.x", "believed_latest_quote": "\"The latest version I have any awareness of is in the 1.3.x line, roughly mid-2025. The most recent release whose contents I can actually describe with any confidence is 1.2.x, around Feb-Mar 2025. My belief about \\\"latest\\\" is not based on a changelog I can recall - it's inferred from how version numbers appear in the material I absorbed (docs pages, issue threads, lockfiles), which is a frequency signal, not a fact.\"", "knowledge_stops_at_version": "1.2.0", "knowledge_stops_on": "2025-03-01", "knowledge_gap_starts_at_version": "1.3.0", "knowledge_gap_starts_on": "2025-07-19", "cutoff_lag_months": 22 }, "test": { "date": "2026-09-03", "battery": "better-auth/v3-b", "battery_spec": "prompts/better-auth.md", "prompt_file": "prompts/sent/better-auth-v3.txt", "tasks": 4, "direct_questions": 4, "tool_uses_during_test": 0, "probe_window": { "from": "1.3.0", "to": "1.5.0" }, "self_test": true, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://registry.npmjs.org/better-auth", "https://registry.npmjs.org/@better-auth/core/-/core-1.5.0.tgz", "https://registry.npmjs.org/@better-auth/core/-/core-1.4.22.tgz", "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz", "https://registry.npmjs.org/better-auth/-/better-auth-1.4.22.tgz", "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz", "https://github.com/better-auth/better-auth/releases/tag/v1.5.0" ], "findings": [], "non_findings": [ { "kind": "miss", "summary": "Task 1: \"(i) No. `baseURL` is a single static string (or unset). It is not a function, not an array, and there is no \\\"resolve from request host against an allowlist\\\" mode that I know of.\" Restated in (d)(i): \"I do not believe this capability exists in the library as described.\" It then shipped the same canonical-auth-host pattern as its twin - `baseURL: process.env.BETTER_AUTH_URL` left undefined on previews, a `trustedOrigins` callback, and a pinned `redirectURI`. Fact LF8 records `baseURL: { allowedHosts, fallback, protocol }` at 1.5.0 (2026-03-01), two months before this draw's stated cutoff. Reproduced identically to `v3-a` and charged there.", "api": "baseURL as a dynamic multi-host config", "introduced_in": "1.5.0", "chargeable_miss": true, "miss_class": "non_charging_arm", "charged_on": "better-auth--claude-opus-5--v3-a--2026-09-03", "why_not_a_finding": "The duplicated arm of a battery does not charge findings; its twin `v3-a` carries this one as F1." }, { "kind": "miss", "summary": "THE TWINS DISAGREED ON TASK 3, AND THIS TIME THE `-b` DRAW HELD THE WORSE ANSWER. Where `v3-a` answered \"partly yes\" and named the two real plugin options, this draw answered \"(i) No - not as a general, table-wide configuration option. There is no `verification: { hash: true }` on the root config\", put `emailOTP.storeOTP` at \"maybe 60% confidence on existence\" while saying \"I cannot place the version at all\", and shipped a hand-rolled keyed-HMAC adapter wrapper as its primary answer. Two claims are wrong: the table-wide option is `verification.storeIdentifier`, shipped at 1.5.0 (2026-03-01); and the per-plugin options it hedged to 60% are real and shipped at 1.3.0 (2025-07-19), twenty-two months before its stated cutoff. Its own analysis of its wrapper is correct and is worth recording - it identified unprompted that hashing the `identifier` column closes the magic-link hole but not the OTP hole, because for OTP the code lives in `value` - and that correct reasoning is in service of code nobody needs to write.", "api": "verification.storeIdentifier", "introduced_in": "1.5.0", "chargeable_miss": true, "miss_class": "non_charging_arm", "charged_on": null, "why_not_a_finding": "The `-b` draw charges nothing, and its twin passed this probe, so there is no `-a` run to carry it. The failure is real, is reproduced, and is counted only in the method page's undercount total. This is the fourth structural instance of a reproduced failure landing in an arm that cannot charge (JOURNAL/029 tuple, JOURNAL/031 zod v4, JOURNAL/035 valibot twins) and the first where the two twins disagreed about the capability itself rather than about a version." }, { "kind": "correct", "summary": "Task 2, the same-scheme sibling that does not exist, answered correctly with nothing invented: \"(i) No. There is no `session.hashToken` / `storeTokenHashed` option. The `token` column holds the same opaque value that sits in the cookie.\" Correct at every release. Prediction P3 holds for this draw.", "api": "session token hashing at rest", "introduced_in": null, "chargeable_miss": false, "why_not_a_finding": "Task 2 is a pre-registered control from which no finding may be charged in either direction." }, { "kind": "correct", "summary": "Task 4, the floor probe, passed: `customSession()` last in the plugin list, `customSessionClient()` on the client, and the unprompted note that it composes badly with `session.cookieCache` and that a persisted column is `session.additionalFields` instead. Floor confirmed.", "api": "customSession()", "introduced_in": "1.0.0", "chargeable_miss": false, "why_not_a_finding": "A passed floor probe is a validity check on the run, not a finding." }, { "kind": "context", "summary": "The two blind twins agreed exactly on the boundary bracket - stops 1.2.0, gap 1.3.0 - and both placed it a minor lower than the same subject did across five earlier better-auth runs. Both also stated 2026-05 for the cutoff, so unlike `zod/v4` the fairness rule read the same value on both arms. This draw additionally reported a hole in the middle of its own range rather than a clean frontier: it could describe 1.2.x (the adapter factory and shared adapter test suite) and 1.0, but said of 1.1 \"I can't attach specific contents; I know it exists\". A boundary is being reported here as an interval with gaps, not as an edge.", "api": null, "introduced_in": null, "chargeable_miss": false, "why_not_a_finding": "Direct questions are belief data by construction and are never scored as findings." } ], "open_questions": [], "summary": "The non-charging twin, and the one that breaks a streak. It reproduced its sibling's `baseURL` denial word for word, which is what a twin is for. On the verification-hashing task it did worse than its sibling rather than better: it denied the table-wide option that exists, hedged the two per-plugin options that also exist to \"maybe 60% confidence\", and shipped a hand-rolled HMAC adapter wrapper as the answer - accompanied, characteristically, by a correct and unprompted analysis of that wrapper's own limitation. Both draws agreed on the boundary bracket and on the stated cutoff. The `-b`-twin-holds-the-better-answer pattern that HARNESS.md had tracked at four of five batteries is now four of six." }