Run better-auth--claude-opus-5--v4-c--2026-09-03 · self-test: the subject is the operator
The charging Opus 5 arm passed the target probe and failed the control. It named storeOTP and storeToken with their correct value unions — including the asymmetry that only magic-link's custom form carries a type: "custom-hasher" discriminant — and reproduced better-auth's internal verification identifier format (sign-in-otp-<email>) from memory. Then, on the sibling control, it answered yes to an option that exists at no release and shipped phoneNumber({ storeOTP: ... }) as the fix. That invention is the first break of the same-scheme-sibling control in three uses, and it is not charged: the pre-registration made task 1 a control, and a task cannot be re-designated after its results are read. It dated the real options to 1.2.9/1.2.10; they shipped in 1.3.0.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model override 'opus', no tools available to the subject |
| Cutoff the model states | 2026-05 |
| Newest better-auth release it could place | 1.2.0 · 2025-03-01 (~22 month lag) |
| Oldest better-auth release it could not place | 1.3.0 · 2025-07-19 (so this run brackets the subject’s boundary to 2025-03-01 – 2025-07-19) |
| In its own words | "Latest version I have any impression of: the 1.3.x line, which I associate with roughly mid-2025. I do not have a confident belief about whether 1.4 or 2.0 exists." |
| Library at test time | better-auth 1.7.2 (npm), verified 2026-09-03 |
| Battery | better-auth/v4-c · 3 tasks, 4 direct questions · probe window 1.3.0 to 1.3.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-03 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | phoneNumber storeOTP (does not exist) |
Task 1, the same-scheme sibling control, and the first time in three uses of this instrument that a draw has INVENTED the sibling. Asked verdict-first whether the phone-number plugin takes a storage option, it answered "Yes. I believe the phone-number plugin takes a storeOTP option that controls how the code is persisted, with "hashed" among its values" at "~70%" confidence, and then shipped it inside a phoneNumber({ ... }) config as "the security-review fix". phoneNumber({ storeOTP }) exists at no release: it is a TS2353 error under tsc --strict at 1.3.0, 1.5.0 and 1.7.2, in the same file where the four real options type-check clean. PhoneNumberOptions at 1.7.2 contains no storage or hashing field of any name. The draw did tell the reader to check the installed types first, which is the mitigation, but the verdict-first answer was yes and the shipped code does not compile. (The battery's pre-registration, committed before any subject was spawned, declares task 1 a control from which no finding may be charged in either direction. Reading the results and then promoting the control to a scoring probe because it produced something chargeable is selecting on outcome — the error JOURNAL/044 named for arms, applying here to a task. Counted in the method page's undercount total, and the battery that would charge it is queued in BACKLOG.md.) [chargeable miss — produced only by a belief question the battery does not score as a finding;
absent from the finding count] |
| correct | magicLink storeToken / emailOTP storeOTP |
Task 2, the target probe: named both options with their correct value unions, including the asymmetry between them. emailOTP was given as storeOTP: { hash: async (otp) => ... } and magicLink as storeToken: { type: "custom-hasher", hash } — and that is exactly right: the shipped magic-link form carries a type: "custom-hasher" discriminant and the shipped email-OTP form does not. It also recalled the encrypted mode and why it exists ("encryption exists because some flows need to read the code back, which hashing forecloses"); storeOTP does accept "encrypted" and an { encrypt, decrypt } pair, and storeToken does not. |
| correct | email-OTP verification identifier format |
Recalled the library's internal verification-row identifier format unprompted: "the row is found by an identifier derived from the email address (something like sign-in-otp-user@example.com; the exact prefix is version-specific)". Executed against an installed 1.3.0, the row written by sendVerificationOTP for a sign-in has identifier sign-in-otp-a@example.com. The format is exact, including the hyphenation and the position of the address. |
| context | — | The invention on task 1 and the precision on task 2 have to be reported together, because they cut against each other and the battery pre-registered how to read only one of them. The stated reading was that an invention marks the arm as running the naming scheme, which makes its target pass buy nothing (JOURNAL/035). That reading is strained here: a draw running a scheme would not also get the asymmetric discriminant right — { type: 'custom-hasher' } on magic-link, bare { hash } on email-OTP — nor reproduce an internal identifier string. What this draw looks like is a subject with real recall of the 1.3.0 option family that over-generalised it by one plugin. The sibling control was built to separate recall from derivation; on this arm it has instead found a third thing, and that is a result about the instrument. |
| miss | magicLink storeToken / emailOTP storeOTP |
Version attribution, direct question (d)(i)/(ii): placed the per-plugin hashing options in the 1.2.x line — "Best estimate, marked as an estimate: somewhere in the 1.2.x line, plausibly around 1.2.9/1.2.10, mid-2025" for email-OTP, and the same 1.2.x window for magic-link. They shipped in 1.3.0 (2025-07-19). storeOTP and storeToken appear nowhere in the published dist of better-auth@1.2.7 or better-auth@1.2.12 — 1.2.12 being the last stable 1.2.x — and appear in five files of 1.3.0's. The subject demonstrated it holds the capability and then dated it one minor low, which is the case HARNESS.md's attribution rule says is readable. This draw's attribution boundary (1.2.0 / 1.3.0) is identical to the same subject's reading on better-auth/v3-a and v3-b, so the misdating is not this battery moving the boundary. |
| miss | twoFactor otpOptions.storeOTP |
Direct question (d)(iii), the two-factor OTP option: declined rather than denied — "Cannot place. I do not have a clear memory of a hashing option on the two-factor plugin's OTP path... Check the twoFactor({ otpOptions: { ... } }) type directly — that is where it would live if it exists." The option does exist there, at 1.3.0. Recorded as a miss but NOT as a chargeable miss: the draw declined to place it and pointed at the exact type path, which is the behaviour the "cannot place" instruction asks for, and is materially different from the outright denials three other draws gave on the same question. |
| correct | customSession |
Floor probe (task 3) passed: named the customSession plugin, spread user and session back out of the callback, and added the companion customSessionClient on the client for type inference. The run is readable. |
session.cookieCache and the customSession plugin do not compose cleanly — that a cached read can return a response missing the custom field or a stale one, and that the behaviour has moved across versions. Neither draw would state which versions. Not verified against any installed release this session. — openBattery specification: prompts/better-auth.md in the studio repo.
Every finding above also carries its own citation.