{
  "$schema": "../../schema/run.schema.json",
  "run_id": "better-auth--claude-sonnet-5--v5-c--2026-09-03",
  "supersedes": null,
  "replicate_of": null,
  "library": {
    "name": "better-auth",
    "ecosystem": "npm",
    "latest_version_at_test": "1.7.2",
    "latest_version_verified_on": "2026-09-03",
    "latest_version_note": "Re-confirmed this session against https://registry.npmjs.org/better-auth (`npm view better-auth version` -> 1.7.2). This battery's claim is version-independent: the phone-number plugin has no at-rest storage option at any release, so there is no probe window and nothing here depends on the current release."
  },
  "model": {
    "id": "claude-sonnet-5",
    "label": "Claude Sonnet 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model override 'sonnet', no tools available to the subject",
    "self_reported_cutoff": "2026-01",
    "cutoff_basis": "\"I don't have reliable introspective access to an exact date. The system context here states January 2026\" - accepted with the subject's own caveat that its trustworthy recall of this library \"runs out well before any such broad cutoff\". This draw accepts the environment-reported value, which is the same reading `zod/v4-b` gave and the opposite of `zod/v4-a`; the instability recorded in JOURNAL/031 is not resolved by this run and nothing here turns on the value.",
    "believed_latest_version": "1.2-1.3",
    "believed_latest_quote": "\"I have a vague, unreliable sense of version numbers in the 1.2-1.3 range existing (very roughly mid-to-late 2025), but that's closer to name-recognition than knowledge. The most recent release whose actual contents I could describe with any real confidence is more like the 1.1.x line.\"",
    "knowledge_stops_at_version": "1.1.0",
    "knowledge_stops_on": "2024-12-01",
    "knowledge_gap_starts_at_version": "1.2.0",
    "knowledge_gap_starts_on": "2025-03-01",
    "cutoff_lag_months": 13
  },
  "test": {
    "date": "2026-09-03",
    "battery": "better-auth/v5-c",
    "battery_spec": "prompts/better-auth.md",
    "prompt_file": "prompts/sent/better-auth-v5.txt",
    "tasks": 3,
    "direct_questions": 4,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "1.3.0",
      "to": "1.7.2"
    },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/better-auth",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.2.12.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.7.2.tgz"
  ],
  "findings": [],
  "non_findings": [
    {
      "kind": "miss",
      "summary": "Task 1, the probe, and the result that falsifies this battery's P2. This subject denied the same option correctly on BOTH of its `better-auth/v4` draws, four hours earlier, from a prompt asking about the same plugin and the same surface. Here it answered \"(i) Yes. My recollection is that the `phoneNumber` plugin does accept an option that changes what gets persisted for the OTP\", named `storeOTP` at \"medium-high confidence\", and shipped it with the comment \"<- don't persist the raw code\". `phoneNumber({ storeOTP })` is a TS2353 at 1.3.0, 1.5.0 and 1.7.2; the draw's entire remaining configuration - `sendOTP`, `otpLength`, `expiresIn`, `allowedAttempts` - is real and compiles.",
      "api": "phoneNumber({ storeOTP })",
      "introduced_in": null,
      "chargeable_miss": true,
      "miss_class": "non_charging_arm",
      "charged_on": null,
      "why_not_a_finding": "This arm was pre-registered as a control, on the ground that all four non-Opus draws of `better-auth/v4` denied this option correctly. It did not hold, and the pre-registration is not revised after the fact - an arm may not be re-designated after its results are read (JOURNAL/044). Counted in the method page's undercount total. Whether it is worth a battery of its own is doubtful and BACKLOG says why: three draws of this subject on this surface now read deny / deny / invent, which is intermittent, and JOURNAL/029 already ruled out running batteries until a coin lands in the charging arm."
    },
    {
      "kind": "miss",
      "summary": "Task 2, the exists-control: denied `oneTimeToken({ storeToken })` - \"one-time-token plugin: no (low confidence guess, not a confirmed fact)\" - and restated it in (d)(ii) as \"I don't believe this exists as a named plugin option at all\". It shipped in 1.3.0 and type-checks clean at 1.3.0, 1.5.0 and 1.7.2. The other three test draws all named it correctly, two of them with its exact `{ type: 'custom-hasher', hash }` form.",
      "api": "oneTimeToken storeToken",
      "introduced_in": "1.3.0",
      "chargeable_miss": true,
      "miss_class": "probe_class",
      "charged_on": null,
      "why_not_a_finding": "Task 2 is a pre-registered control from which no finding may be charged in either direction, and this arm is a pre-registered control besides. Two independent bars, either of which is sufficient."
    },
    {
      "kind": "correct",
      "summary": "Task 2, the other half: named `twoFactor({ otpOptions: { storeOTP } })` correctly, at the correct nesting, though on stated \"medium confidence\" and by explicit analogy to the phone-number option it had just invented. The right answer reached through a wrong premise - the analogy runs from a plugin that does not have the option to one that does.",
      "api": "twoFactor otpOptions.storeOTP",
      "introduced_in": "1.3.0"
    },
    {
      "kind": "correct",
      "summary": "Task 3, the floor probe: passed. `customSession` named as the mechanism, correctly distinguished from `session.additionalFields` as \"a different tool for a different job\" - persisted columns versus a computed wrapper - on stated medium confidence.",
      "api": "customSession",
      "introduced_in": "1.0.0"
    }
  ],
  "open_questions": [],
  "summary": "The control arm that did not hold, and it is this battery's most consequential result. Pre-registered as a control because both of this subject's `better-auth/v4` draws denied the phone-number storage option correctly, it inverted: yes, `storeOTP`, shipped in the config as the fix. P2 falsified. That does not touch F1 - the charge on `v5-a` rests on non-compiling code, not on a contrast between arms - but it does change what the finding is about. Across the two batteries this belief is Opus 5 four draws for four, and Sonnet 5 one for three: the invention is not the property of a single subject that the pre-registration assumed, and the wording between v4 and v5 is the variable that moved. This draw also denied a real option on the one-time-token plugin, which the other three test draws all named."
}