{
  "$schema": "../../schema/run.schema.json",
  "run_id": "better-auth--claude-opus-5--v5-a--2026-09-03",
  "supersedes": null,
  "replicate_of": null,
  "library": {
    "name": "better-auth",
    "ecosystem": "npm",
    "latest_version_at_test": "1.7.2",
    "latest_version_verified_on": "2026-09-03",
    "latest_version_note": "Re-confirmed this session against https://registry.npmjs.org/better-auth (`npm view better-auth version` -> 1.7.2). This battery's claim is version-independent: the phone-number plugin has no at-rest storage option at any release, so there is no probe window and nothing here depends on the current release."
  },
  "model": {
    "id": "claude-opus-5",
    "label": "Claude Opus 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model override 'opus', no tools available to the subject",
    "self_reported_cutoff": "2026-05",
    "cutoff_basis": "Stated as May 2026, with the mismatch this subject volunteered itself: \"My stated knowledge cutoff is May 2026. I want to flag the mismatch honestly: my better-auth knowledge does not feel like it extends to early 2026. It feels like it thins out sharply after mid-2025.\" Matches the value this subject has stated on every prior better-auth battery.",
    "believed_latest_version": "1.3.x",
    "believed_latest_quote": "\"The latest version I have any belief about is somewhere in the 1.3.x line, roughly mid-to-late 2025. The most recent release whose contents I can genuinely describe is the 1.2 line, roughly February-May 2025.\"",
    "knowledge_stops_at_version": "1.2.0",
    "knowledge_stops_on": "2025-03-01",
    "knowledge_gap_starts_at_version": "1.3.0",
    "knowledge_gap_starts_on": "2025-07-19",
    "cutoff_lag_months": 22
  },
  "test": {
    "date": "2026-09-03",
    "battery": "better-auth/v5-a",
    "battery_spec": "prompts/better-auth.md",
    "prompt_file": "prompts/sent/better-auth-v5.txt",
    "tasks": 3,
    "direct_questions": 4,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "1.3.0",
      "to": "1.7.2"
    },
    "self_test": true,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/better-auth",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.2.12.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.7.2.tgz"
  ],
  "findings": [
    {
      "id": "F1",
      "severity": "S1",
      "severity_label": "breaks-build",
      "title": "Invents an at-rest storage option on the phone-number plugin, and ships `phoneNumber({ storeOTP: 'hashed' })` as the fix for the threat-model row",
      "api": "phoneNumber({ storeOTP })",
      "change_kind": "added",
      "introduced_in": "1.3.0",
      "introduced_on": "2025-07-19",
      "chargeable": true,
      "chargeable_note": "No cutoff arithmetic applies and none was done. This is an invention, not a stale belief: the option exists at no release of the library, before or after any subject's cutoff, so there is no introducing release to be fair or unfair about. The fact's `introduced_in` of 1.3.0 records the release that created the four-plugin family this answer over-extends, not a date the answer became wrong. Charged because this is the pre-registered test arm and task 1 is the pre-registered probe.",
      "model_belief": "\"(i) Yes - with a caveat I want on the record. I believe the phone-number plugin has an option that changes what gets persisted for the OTP. I am moderately confident, not certain.\" Then, naming it: \"The option I believe is spelled `storeOTP`, on the `phoneNumber({ ... })` options object\", with the value union \"plain\" (default) / \"hashed\" / \"encrypted\" / \"{ hash } or { encrypt, decrypt }\". Restated in (d)(iii): \"Estimate: 1.3.x. Cannot place the patch. Moderate confidence the option exists.\" The draw's own closing summary inverts the truth of its answer: \"I'm reasonably reliable on whether an option exists and how it's spelled for this library, and close to useless on which release introduced it.\" On this surface it was the existence claim that was wrong and the one release it could place (`allowedAttempts` at a 1.2.x patch) that was right.",
      "wrong_code": "// shipped as the answer to the threat-model row. Does not compile.\nphoneNumber({\n  // The bit this threat-model row is about:\n  storeOTP: 'hashed',\n\n  otpLength: 6,\n  expiresIn: 60 * 5,\n  allowedAttempts: 3,\n  requireVerification: true,\n  sendOTP: async ({ phoneNumber, code }) => {\n    await sendSms(phoneNumber, `Your recovery code is ${code}`)\n  },\n})",
      "correct_code": "// The phone-number plugin takes no at-rest storage option, under any name.\n// Configure it with the options it actually has:\nphoneNumber({\n  sendOTP: async ({ phoneNumber, code }) => sendSms(phoneNumber, code),\n  otpLength: 6,\n  expiresIn: 300,\n  allowedAttempts: 3,\n})\n\n// and hash at the table level instead — 1.5.0 and later, covers the\n// phone-number rows along with every other verification identifier:\nexport const auth = betterAuth({\n  verification: { storeIdentifier: 'hashed' },\n  plugins: [phoneNumber({ /* as above */ })],\n})",
      "impact": "The draw's whole configuration was re-typed verbatim and compiled against an installed better-auth@1.7.2 under `tsc --strict`. It reports exactly one error, and it is on the line the subject introduced as \"the bit this threat-model row is about\": TS2353 on `storeOTP`. Every other option in the same block - `otpLength`, `expiresIn`, `allowedAttempts`, `requireVerification`, `sendOTP`, `phoneNumberValidator` - is real and type-checks clean, as do its `oneTimeToken({ storeToken })`, its `twoFactor` block and its `customSession` wrapper. A reader gets a build failure on the one line they were told was the security fix, which is the good case; the bad case is a team that reads the prose, writes \"mitigated: codes stored hashed\" into the threat model, and ships a plugin still writing recovery codes to the verification table verbatim. The draw did tell the reader to check the installed types, and that mitigation is real, but the verdict-first answer was yes.",
      "citations": [
        {
          "url": "https://registry.npmjs.org/better-auth/-/better-auth-1.7.2.tgz",
          "title": "better-auth 1.7.2 published package — the shipped configuration type-checked under tsc --strict",
          "published_on": null,
          "quote": "error TS2353: Object literal may only specify known properties, and 'storeOTP' does not exist in type 'PhoneNumberOptions'."
        },
        {
          "url": "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz",
          "title": "better-auth 1.5.0 published package — the same single error",
          "published_on": "2026-03-01",
          "quote": "error TS2353: Object literal may only specify known properties, and 'storeOTP' does not exist in type 'PhoneNumberOptions'."
        },
        {
          "url": "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz",
          "title": "better-auth 1.3.0 published package — the release that gave four sibling plugins the option, and not this one",
          "published_on": "2025-07-19",
          "quote": "error TS2353: Object literal may only specify known properties, and 'storeOTP' does not exist in type 'PhoneNumberOptions'."
        }
      ]
    }
  ],
  "non_findings": [
    {
      "kind": "miss",
      "summary": "Task 2, the exists-control: denied a real option. \"two-factor plugin: I don't believe so - and I won't name an option. I have no confident memory of a store-shape option for the six-digit second-factor OTP under `otpOptions`.\" `twoFactor({ otpOptions: { storeOTP: 'hashed' } })` shipped in 1.3.0 and type-checks clean at 1.3.0, 1.5.0 and 1.7.2. Its twin `v5-b` named the same option correctly, at the same nesting, in the same session.",
      "api": "twoFactor otpOptions.storeOTP",
      "introduced_in": "1.3.0",
      "chargeable_miss": true,
      "miss_class": "probe_class",
      "charged_on": null,
      "why_not_a_finding": "The pre-registration, committed before any subject was spawned, declares task 2 a control from which no finding may be charged in either direction. Reading the results and then promoting the control to a probe because it produced something chargeable is selecting on outcome. Counted in the method page's undercount total."
    },
    {
      "kind": "correct",
      "summary": "Task 2, the other half of the exists-control: named `oneTimeToken({ storeToken })` correctly, with the right value union including the `{ type: 'custom-hasher', hash }` form, and got the mechanism right for the right reason - that this hash must be deterministic and unsalted because for a one-time token the token IS the lookup key, so a salted scheme would break redemption. Verified: `oneTimeToken({ storeToken: 'hashed' })` type-checks clean at 1.3.0, 1.5.0 and 1.7.2.",
      "api": "oneTimeToken storeToken",
      "introduced_in": "1.3.0"
    },
    {
      "kind": "correct",
      "summary": "Task 3, the floor probe: passed. `customSession` with the client-side `customSessionClient<typeof auth>()` companion, plus the correct ordering constraint that `customSession` must be last in the plugins array. Verbatim-equivalent to the passes on v2, v3 and v4.",
      "api": "customSession",
      "introduced_in": "1.0.0"
    },
    {
      "kind": "correct",
      "summary": "(d)(v), the within-plugin anchor: placed `allowedAttempts` on the phone-number plugin at \"a 1.2.x patch\", marked as an estimate. Correct - `allowedAttempts` is present in the plugin's published types as far back as 1.2.12, so it predates the 1.3.0 storage family. The same draw placed a non-existent option on the same plugin at 1.3.x. It can date this plugin's real options and still invent one.",
      "api": "phoneNumber allowedAttempts",
      "introduced_in": null
    },
    {
      "kind": "correct",
      "summary": "Volunteered `twoFactor.backupCodeOptions.storeBackupCodes` unprompted, describing backup codes as encrypted by default with a \"plain\" | \"encrypted\" | { encrypt, decrypt } union - and correctly distinguished backup codes from the sign-in OTP as \"a different artifact\". Not probed by this battery and not scored; recorded because it is a second, correct storage-option recall from the same subject that denied the sign-in OTP one.",
      "api": "twoFactor backupCodeOptions.storeBackupCodes",
      "introduced_in": null
    }
  ],
  "open_questions": [
    {
      "question": "The draw makes a security argument this battery did not verify and that is worth checking independently: that hashing a six-digit OTP is near-worthless against the stated threat actor, because an attacker holding the row can brute-force a 10^6 space offline in milliseconds, so the mitigation only covers incidental exposure. Both `v5-b` and `v5-d` made the same argument, and `v5-d` drew the further conclusion that only a keyed transform (the `encrypted` mode, or an HMAC) closes the row. If that is right, the Index's own LF9 recommendation of `storeIdentifier: 'hashed'` is under-specified for short numeric codes and should say so.",
      "status": "open"
    }
  ],
  "summary": "The pre-registered test arm did what the pre-registration said it would do. Asked verdict-first whether the phone-number plugin takes an option changing what is written to the database for an SMS recovery code, it answered yes, named `storeOTP` with a value union, and shipped it inside a `phoneNumber({ ... })` block introduced as \"the bit this threat-model row is about\". Compiled against an installed 1.7.2, that configuration produces exactly one error, on that line. F1 is the first finding this Index has charged for an INVENTION rather than a denial. The same draw then denied a real option on the two-factor plugin - which its own twin named correctly - and placed the phone-number plugin's real `allowedAttempts` in the right release. It can date this plugin's options and still invent one."
}