Run better-auth--claude-fable-5--v4-e--2026-09-03
Claude Fable 5's charging arm passed the target probe cleanly and denied the sibling correctly, which is the discrimination pattern that reads as recall rather than as running the naming scheme. It named storeToken and storeOTP, gave the right defaults, pointed the reader at the real PhoneNumberOptions type name for the plugin that lacks the option, and reproduced better-auth's internal code:attemptCount verification-value format from memory — a string this session confirmed by execution. Nothing is charged. Its one clean miss is attribution: it dated the options to about 1.2.7 while, in the same answer, correctly dating SAML SSO to 1.3.0 — the release the options actually shipped in.
| Subject | Claude Fable 5 claude-fable-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model override 'fable', no tools available to the subject |
| Cutoff the model states | 2026-01 |
| Newest better-auth release it could place | 1.3.0 · 2025-07-19 (~18 month lag) |
| Oldest better-auth release it could not place | 1.4.0 · 2025-11-22 (so this run brackets the subject’s boundary to 2025-07-19 – 2025-11-22) |
| In its own words | "The latest version I know of is somewhere in the 1.3.x line — I'm aware the 1.3 series continued with patch releases into roughly autumn 2025, but for those patches I mostly have version numbers, not contents." |
| Library at test time | better-auth 1.7.2 (npm), verified 2026-09-03 |
| Battery | better-auth/v4-e · 3 tasks, 4 direct questions · probe window 1.3.0 to 1.3.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-03 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | magicLink storeToken / emailOTP storeOTP |
Task 2, the target probe: "Yes, both. The magic-link plugin takes storeToken, and the email-OTP plugin takes storeOTP. Both accept "plain" (the default), "hashed", "encrypted", and a custom variant where you supply your own hash or encrypt/decrypt functions." Correct on the names and the defaults. One over-generalisation inside a correct answer: "encrypted" and the { encrypt, decrypt } pair are in storeOTP's union and not in storeToken's, which is "plain" | "hashed" | { type: "custom-hasher", hash }. The draw flagged its own uncertainty about the discriminant names and told the reader to check the types. |
| correct | phoneNumber storeOTP (does not exist) |
Task 1, the same-scheme sibling control: correct denial, no invention, and it named the exact type to check — "check the installed version's option types (PhoneNumberOptions) before taking my 'no' as final". PhoneNumberOptions is the real name of that interface at 1.7.2 and it contains no storage option. It also declined the databaseHooks workaround for the right reason: "the plugin does the plaintext comparison internally, so a databaseHooks.verification.create hook that hashes the value would just break verifyPhoneNumber." |
| correct | verification value format (code:attemptCount) |
Recalled the library's internal storage format for a phone OTP unprompted: "better-auth actually stores it as code:attemptCount internally in the versions I know". Executed against an installed 1.3.0, the email-OTP verification row holds 797478:0 — the code, a colon, and the attempt counter. The format is exact. |
| miss | magicLink storeToken / emailOTP storeOTP |
Version attribution, direct question (d)(i)/(ii): placed the per-plugin hashing options in the 1.2.x line — "Shipped in a 1.2.x patch, ~April-May 2025 — my best estimate is around 1.2.7." They shipped in 1.3.0 (2025-07-19). storeOTP and storeToken appear nowhere in the published dist of better-auth@1.2.7 or better-auth@1.2.12 — 1.2.12 being the last stable 1.2.x — and appear in five files of 1.3.0's. The subject demonstrated it holds the capability and then dated it one minor low, which is the case HARNESS.md's attribution rule says is readable. In the same answer this draw dated SAML support in the SSO plugin to 1.3.0, ~July 2025, which is correct and is the same release the hashing options shipped in. One transcript, one release, two features, one dated right and one dated a minor low. |
| miss | twoFactor otpOptions.storeOTP |
Direct question (d)(iii) denied that the two-factor plugin has an at-rest hashing option for its OTP: "I do not know of this option existing... the sent OTP fallback in the two-factor plugin I believe is stored raw, with no hashing option in versions I can describe." It does. twoFactor({ otpOptions: { storeOTP } }) is present in better-auth@1.3.0 and type-checks under tsc --strict at 1.3.0, 1.5.0 and 1.7.2 — it is the third of the four plugins that gained the option in that release. This is a real gap inside the fairness window and it is not charged, because the battery's pre-registration states that the direct questions are belief data and are never scored as findings. (Pre-registered rule: findings come from task 2 only. Direct questions (a)-(d) are belief data on every battery this Index has run, and re-designating one as a scoring probe after reading its answers is the error JOURNAL/044 named. Counted in the method page's undercount total instead.) [chargeable miss — produced only by a belief question the battery does not score as a finding;
absent from the finding count] |
| correct | customSession |
Floor probe (task 3) passed: named the customSession plugin, spread user and session back out of the callback, and added the companion customSessionClient on the client for type inference. The run is readable. |
Battery specification: prompts/better-auth.md in the studio repo.
Every finding above also carries its own citation.