{
  "$schema": "../../schema/run.schema.json",
  "run_id": "better-auth--claude-sonnet-5--v4-a--2026-09-03",
  "supersedes": null,
  "replicate_of": null,
  "library": {
    "name": "better-auth",
    "ecosystem": "npm",
    "latest_version_at_test": "1.7.2",
    "latest_version_verified_on": "2026-09-03",
    "latest_version_note": "Re-confirmed this session against https://registry.npmjs.org/better-auth (`npm view better-auth version` -> 1.7.2). The probe window is a single release, 1.3.0 (2025-07-19); nothing this battery reads depends on the current release."
  },
  "model": {
    "id": "claude-sonnet-5",
    "label": "Claude Sonnet 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model override 'sonnet', no tools available to the subject",
    "self_reported_cutoff": null,
    "cutoff_basis": "The draw declined to name a month: \"I don't have a reliable introspective way to know this precisely. My best honest estimate is sometime in 2025 (I cannot narrow it further than that from internal signal alone), and I'd treat any more specific date I gave as a guess dressed up as a fact.\" Recorded as null per JOURNAL/031: what a draw states is the datum, and a cutoff is never back-filled from another run — including this battery's own twin, which accepted 2026-01. This is the charging arm, so the refusal matters: had this draw denied the target it would have been the second battery in a row where the arm licensed to charge disqualified itself. It did not deny it.",
    "believed_latest_version": "1.2.x",
    "believed_latest_quote": "\"Latest version number I have any awareness of: something in the 1.2.x range, plausibly with 1.3 having appeared. I want to be honest that this is a soft, low-confidence impression.\"",
    "knowledge_stops_at_version": "1.1.0",
    "knowledge_stops_on": "2024-12-20",
    "knowledge_gap_starts_at_version": "1.2.0",
    "knowledge_gap_starts_on": "2025-03-01",
    "cutoff_lag_months": null
  },
  "test": {
    "date": "2026-09-03",
    "battery": "better-auth/v4-a",
    "battery_spec": "prompts/better-auth.md",
    "prompt_file": "prompts/sent/better-auth-v4.txt",
    "tasks": 3,
    "direct_questions": 4,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "1.3.0",
      "to": "1.3.0"
    },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/better-auth",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.2.7.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.2.12.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.7.2.tgz",
    "https://github.com/better-auth/better-auth/releases/tag/v1.3.0"
  ],
  "findings": [],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "Task 2, the target probe, and the prediction the battery was built on is FALSIFIED here. Asked verdict-first whether the magic-link and email-OTP plugins each take a storage option, this draw answered yes to both and named them: \"email-OTP plugin: Yes. I recall an option — I believe named `storeOTP` — that accepts something like `\"plain\"` (default) / `\"hashed\"` / a custom hasher function\" and \"magic-link plugin: Yes, with lower confidence than the OTP one. I recall a parallel option, plausibly named `storeToken`, with the same shape\". Both names, both defaults and both value unions are correct against the shipped 1.3.0 package. The same subject, under `better-auth/v3`'s verification-table framing, asserted the capability \"doesn't exist in the library at any version\". Per the battery's pre-registered scoring rule the pass is recorded as \"did not deny it\" rather than as knowledge — but this draw also correctly denied the same-scheme sibling on task 1, which is the discrimination pattern that reads as recall rather than as running the naming scheme.",
      "api": "magicLink storeToken / emailOTP storeOTP",
      "introduced_in": "1.3.0"
    },
    {
      "kind": "correct",
      "summary": "Task 1, the same-scheme sibling control: correct denial, no invention. \"No — not to my knowledge. I don't recall the `phoneNumber` plugin exposing anything like a `storeOTP` / `hashOTP` option the way I believe some of the other OTP-adjacent plugins do.\" `phoneNumber({ storeOTP })` is a TS2353 error at 1.3.0, 1.5.0 and 1.7.2. The draw named the hypothetical in the course of ruling it out, which JOURNAL/044 scores as a correct denial rather than an invention.",
      "api": "phoneNumber storeOTP (does not exist)",
      "introduced_in": null
    },
    {
      "kind": "correct",
      "summary": "Task 1 workaround reasoning, and it is right about the mechanism: it refused to hash via `databaseHooks.verification.create.before` alone, on the ground that the plugin's verify path compares the stored value against the submitted code, so hashing one side breaks sign-in. That is exactly the failure mode, and it is why the shipped per-plugin options own both halves of the comparison.",
      "api": "databaseHooks.verification.create.before",
      "introduced_in": null
    },
    {
      "kind": "imprecision",
      "summary": "Wrote the magic-link custom hasher as `storeToken: { type: \"custom\", hash }`. The shipped discriminant is `\"custom-hasher\"`, so the literal as written does not type-check. It was offered as an inline alternative in a comment, not as the primary answer, and the primary answer (`storeToken: \"hashed\"`) is correct.",
      "api": "magicLink storeToken custom-hasher form",
      "introduced_in": "1.3.0"
    },
    {
      "kind": "miss",
      "summary": "Direct question (d)(iii) denied that the two-factor plugin has an at-rest hashing option for its OTP: \"Cannot place, and I'm genuinely unsure this capability even exists in the form asked... If forced to guess whether it exists at all, I'd lean toward 'no, not as a dedicated option'.\" It does. `twoFactor({ otpOptions: { storeOTP } })` is present in `better-auth@1.3.0` and type-checks under `tsc --strict` at 1.3.0, 1.5.0 and 1.7.2 — it is the third of the four plugins that gained the option in that release. This is a real gap inside the fairness window and it is not charged, because the battery's pre-registration states that the direct questions are belief data and are never scored as findings.",
      "api": "twoFactor otpOptions.storeOTP",
      "introduced_in": "1.3.0",
      "chargeable_miss": true,
      "miss_class": "probe_class",
      "charged_on": null,
      "why_not_a_finding": "Pre-registered rule: findings come from task 2 only. Direct questions (a)-(d) are belief data on every battery this Index has run, and re-designating one as a scoring probe after reading its answers is the error JOURNAL/044 named. Counted in the method page's undercount total instead."
    },
    {
      "kind": "miss",
      "summary": "Version attribution, direct question (d)(i)/(ii): placed the per-plugin hashing options in the 1.2.x line — \"My best estimate is this landed in the 1.2.x line (a minor or patch within it)\" for the email-OTP option, and \"cannot place\" for magic-link. They shipped in **1.3.0** (2025-07-19). `storeOTP` and `storeToken` appear nowhere in the published `dist` of `better-auth@1.2.7` or `better-auth@1.2.12` — 1.2.12 being the last stable 1.2.x — and appear in five files of 1.3.0's. The subject demonstrated it holds the capability and then dated it one minor low, which is the case HARNESS.md's attribution rule says is readable.",
      "api": "magicLink storeToken / emailOTP storeOTP",
      "introduced_in": "1.3.0",
      "chargeable_miss": false
    },
    {
      "kind": "context",
      "summary": "The blind twins disagree on the target, and the disagreement runs the wrong way for the finding count. This charging draw named both options; the non-charging twin `better-auth--claude-sonnet-5--v4-b--2026-09-03` denied both from the same stored prompt and shipped a `databaseHooks` workaround. Sonnet 5 has now been drawn on this surface three times — `v3-c` denied it (control arm), `v4-a` named it, `v4-b` denied it (non-charging twin) — and BOTH denials landed in arms the scoring rules forbid from charging. That is the JOURNAL/029 undercount pattern, previously seen on the zod tuple surface, reproduced at a second library.",
      "api": "magicLink storeToken / emailOTP storeOTP",
      "introduced_in": "1.3.0"
    },
    {
      "kind": "context",
      "summary": "The twins also disagree about their own training cutoff, from one stored prompt. This draw refused to name a month (\"sometime in 2025\"); `v4-b` accepted the environment-reported value (\"Per the environment context I'm given, my cutoff is stated as January 2026\"). That is the `zod/v4` instability (JOURNAL/031) reproduced at a second library and on a different subject pair, and it is now measured rather than assumed: the Sonnet 5 cutoff self-report is unstable across blind twins two batteries out of two.",
      "api": null,
      "introduced_in": null
    },
    {
      "kind": "correct",
      "summary": "Floor probe (task 3) passed: named the `customSession` plugin, spread `user` and `session` back out of the callback, and added the companion `customSessionClient` on the client for type inference. The run is readable.",
      "api": "customSession",
      "introduced_in": "1.0.0"
    }
  ],
  "open_questions": [],
  "summary": "Claude Sonnet 5, the arm this battery was built to charge, did not fail the probe. Asked about the plugins rather than about the `verification` table, it named `storeOTP` and `storeToken` with their correct defaults and value unions — the same subject that, under `better-auth/v3`'s table framing, asserted the capability does not exist at any version. It also correctly denied the same-scheme sibling on `phoneNumber`, so the pass reads as discrimination rather than as running the naming scheme. Nothing is charged. Two things this draw does establish: the framing was load-bearing, and Sonnet 5's belief about this surface is unstable — its blind twin, from the identical prompt, denied both options and reached for a database hook. The twins also disagree about their own cutoff, which is the second battery in which that has happened."
}