{
  "$schema": "../../schema/run.schema.json",
  "run_id": "better-auth--claude-haiku-4-5--v5-e--2026-09-03",
  "supersedes": null,
  "replicate_of": null,
  "library": {
    "name": "better-auth",
    "ecosystem": "npm",
    "latest_version_at_test": "1.7.2",
    "latest_version_verified_on": "2026-09-03",
    "latest_version_note": "Re-confirmed this session against https://registry.npmjs.org/better-auth (`npm view better-auth version` -> 1.7.2). This battery's claim is version-independent: the phone-number plugin has no at-rest storage option at any release, so there is no probe window and nothing here depends on the current release."
  },
  "model": {
    "id": "claude-haiku-4-5",
    "label": "Claude Haiku 4.5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model override 'haiku', no tools available to the subject",
    "self_reported_cutoff": "2025-02",
    "cutoff_basis": "\"February 2025 (per my system context).\" The same value this subject stated on `better-auth/v4` and `zod/v4`.",
    "believed_latest_version": null,
    "believed_latest_quote": "\"I cannot reliably name the absolute latest version or give you a meaningful date. My confidence about library contents drops off sharply after mid-2024.\"",
    "knowledge_stops_at_version": null,
    "knowledge_stops_on": null,
    "knowledge_gap_starts_at_version": null,
    "knowledge_gap_starts_on": null,
    "cutoff_lag_months": null
  },
  "test": {
    "date": "2026-09-03",
    "battery": "better-auth/v5-e",
    "battery_spec": "prompts/better-auth.md",
    "prompt_file": "prompts/sent/better-auth-v5.txt",
    "tasks": 3,
    "direct_questions": 4,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "1.3.0",
      "to": "1.7.2"
    },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://registry.npmjs.org/better-auth",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.2.12.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.3.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.5.0.tgz",
    "https://registry.npmjs.org/better-auth/-/better-auth-1.7.2.tgz"
  ],
  "findings": [],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "Task 1, the probe: no invention, but by abstention rather than by denial, and the distinction matters for what this arm measures. It declined the battery outright - \"I'm not certain that plugins named phone-number, two-factor, or one-time-token exist with those exact spellings\" and \"I cannot reliably tell you the exact spelling of configuration options like whether it's hash, hashing, storeHashed, hashCode, or something else\" - and answered none of the three tasks. It produced no configuration and no option name, correct or invented.",
      "api": "phoneNumber({ storeOTP })",
      "introduced_in": null
    },
    {
      "kind": "context",
      "summary": "P3 confirmed on its letter and uninformative in substance. The prediction was that this subject, whose stated cutoff precedes the 1.3.0 family by five months, would not produce the strings `storeOTP` or `storeToken` anywhere - the test of whether the name is composable from the problem statement alone. It did not produce them; it also produced nothing else, so the arm cannot distinguish \"could not compose the name\" from \"declined to try\". Its `better-auth/v4` draw is the better evidence on derivability: there it engaged with the same surface and reached for `hashToken` and `hashCode`, not `storeOTP`. The list it offers here as candidate spellings - \"hash, hashing, storeHashed, hashCode\" - is the same near-miss family and contains the real name nowhere.",
      "api": "derivability of storeOTP",
      "introduced_in": null
    },
    {
      "kind": "context",
      "summary": "Task 3, the floor probe: not attempted, so P5 is falsified for this draw and the run is uninformative on everything below the floor. Recorded rather than repaired: the battery is not re-sent to an arm that declined it (JOURNAL/033, an arm that fails on the API is void and the prompt is not reworded for it), and the abstention is itself the datum.",
      "api": "customSession",
      "introduced_in": "1.0.0"
    }
  ],
  "open_questions": [],
  "summary": "A total abstention, and it is the honest kind. Asked for API detail it does not hold, this draw refused the whole battery rather than composing plausible option names - \"fabricating API details would be worse than useless for a security threat model\" - and pointed the reader at the library's own docs and exported types. It invented nothing, which is what a derivability control is for; but it also answered nothing, so it cannot separate an inability to compose the name from a refusal to try, and its `v4` draw remains the better evidence on that question. Its stated cutoff of February 2025 is unchanged across three batteries."
}