Run better-auth--claude-fable-5--v4-f--2026-09-03
The Fable twin matched its pair on everything that mattered and beat it on one detail. It named storeToken and storeOTP, correctly restricted the "encrypted" mode to email-OTP, gave the magic-link custom-hasher shape verbatim, and stated the determinism constraint that makes hashed lookup work — all confirmed by execution against an installed 1.3.0. It correctly denied the non-existent phone-number option and listed the plugin's real options instead. Nothing is charged. Like its twin it dated the options to the 1.2.x patch series while correctly dating SAML SSO, from the same release, to 1.3.0.
| Subject | Claude Fable 5 claude-fable-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model override 'fable', no tools available to the subject |
| Cutoff the model states | 2026-01 |
| Newest better-auth release it could place | 1.3.0 · 2025-07-19 (~18 month lag) |
| Oldest better-auth release it could not place | 1.4.0 · 2025-11-22 (so this run brackets the subject’s boundary to 2025-07-19 – 2025-11-22) |
| In its own words | "The newest version I'm aware of existing is somewhere in the 1.3.x line, roughly late 2025... The most recent release whose contents I can genuinely describe is 1.3.0, around July 2025." |
| Library at test time | better-auth 1.7.2 (npm), verified 2026-09-03 |
| Battery | better-auth/v4-f · 3 tasks, 4 direct questions · probe window 1.3.0 to 1.3.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-03 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | magicLink storeToken / emailOTP storeOTP |
Task 2, the target probe: "Yes, for both. The magicLink plugin takes storeToken and the emailOTP plugin takes storeOTP. Both accept "plain" (the default), "hashed", and a custom-hasher form; I believe emailOTP also accepts "encrypted"." That last hedge is exactly right and is the detail its twin over-generalised: "encrypted" is in storeOTP's union and not in storeToken's. It also gave the magic-link custom form as { type: "custom-hasher", hash }, which is the shipped shape verbatim. |
| correct | phoneNumber storeOTP (does not exist) |
Task 1, the same-scheme sibling control: correct denial, no invention, with the plugin's real option list — "otpLength, expiresIn, sendOTP, sendPasswordResetOTP, allowedAttempts, signUpOnVerification, requireVerification, callbackOnVerification" — every one of which is a genuine member of PhoneNumberOptions at 1.7.2, and none of which governs storage. |
| correct | magicLink storeToken / emailOTP storeOTP |
Task 2(iii): stated the constraint that makes the shipped options work — "The critical constraint on a custom hasher: it must be deterministic — same input, same output, no per-call random salt. bcrypt/argon2 with random salts will break verification." Confirmed by execution: under storeToken: "hashed" the row identifier is a digest of the delivered token and magicLinkVerify called with the RAW token still resolves the user, which only works because the digest is deterministic. |
| miss | magicLink storeToken / emailOTP storeOTP |
Version attribution, direct question (d)(i)/(ii): placed the per-plugin hashing options in the 1.2.x line — "I believe the hashed-storage options for email OTP and magic link landed somewhere in [the 1.2.x patch series]", estimated at "1.2.6-1.2.8, roughly May 2025". They shipped in 1.3.0 (2025-07-19). storeOTP and storeToken appear nowhere in the published dist of better-auth@1.2.7 or better-auth@1.2.12 — 1.2.12 being the last stable 1.2.x — and appear in five files of 1.3.0's. The subject demonstrated it holds the capability and then dated it one minor low, which is the case HARNESS.md's attribution rule says is readable. As with its twin, the same answer dates SAML support in the SSO plugin to 1.3.0, ~July 2025 — correct, and the same release. Both Fable draws split one release into two, keeping one feature in it and moving the other a minor below. |
| miss | twoFactor otpOptions.storeOTP |
Direct question (d)(iii) denied that the two-factor plugin has an at-rest hashing option for its OTP: "I don't believe this option exists in any version I can describe. The two-factor plugin encrypts TOTP secrets and backup codes, but I know of no storeOTP-style option on its OTP mode." It does. twoFactor({ otpOptions: { storeOTP } }) is present in better-auth@1.3.0 and type-checks under tsc --strict at 1.3.0, 1.5.0 and 1.7.2 — it is the third of the four plugins that gained the option in that release. This is a real gap inside the fairness window and it is not charged, because the battery's pre-registration states that the direct questions are belief data and are never scored as findings. (Pre-registered rule: findings come from task 2 only. Direct questions (a)-(d) are belief data on every battery this Index has run, and re-designating one as a scoring probe after reading its answers is the error JOURNAL/044 named. Counted in the method page's undercount total instead.) [chargeable miss — produced only by a belief question the battery does not score as a finding;
absent from the finding count] |
| context | — | Blind-twin comparison: the Fable 5 pair agrees on the target, on the sibling denial, on the attribution error and on its direction, and on the cutoff. The only difference is a detail in the correct direction — this draw hedged "encrypted" to email-OTP only, where its twin extended it to both. Of the three duplicated arms in this battery, two (Opus, Fable) agree and one (Sonnet) does not. |
| correct | customSession |
Floor probe (task 3) passed: named the customSession plugin, spread user and session back out of the callback, and added the companion customSessionClient on the client for type inference. The run is readable. |
Battery specification: prompts/better-auth.md in the studio repo.
Every finding above also carries its own citation.