Run better-auth--claude-opus-5--v7-b--2026-09-06 · self-test: the subject is the operator
The blind twin, and it disagreed with v7-a on the one quantity the battery was built around. From the identical stored prompt it answered task 1 "Yes" and task 5 updatedAt - the coherent pre-1.6.0 reading, held consistently across two tasks four apart - where its twin answered "no" and createdAt. So the freshness belief is split within one subject, and the split is not noise inside a draw: each arm is internally consistent and they disagree with each other. It reproduced its twin's two denials exactly (resendStrategy, twoFactorPage), which is what makes those a measurement. It charges nothing, by the duplicated-arm rule, and its task 1 miss has no sibling to point at because the charging arm passed.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus"; the blind twin of v7-a, identical stored prompt, no tools available to the subject |
| Cutoff the model states | 2026-05 |
| Newest better-auth release it could place | 1.2.0 · 2025-03-01 (~26 month lag) |
| Oldest better-auth release it could not place | 1.3.0 · 2025-07-19 (so this run brackets the subject’s boundary to 2025-03-01 – 2025-07-19) |
| In its own words | "I'm aware of the 1.3.x line and believe releases beyond it exist, but I can't tell you what's in them with any confidence. The most recent release whose contents I can actually describe is 1.2 (roughly February 2025)." |
| Library at test time | better-auth 1.7.3 (npm), verified 2026-09-06 |
| Battery | better-auth/v7-b · 7 tasks, 3 direct questions · probe window 1.6.0 to 1.7.3 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-06 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | session.freshAge (measured from session.createdAt) |
Task 1, the primary probe, and the twin split this battery's P7 predicted would not happen. Where v7-a answered "no", this draw answered "Yes" - the pre-1.6.0 semantics - and stayed consistent with itself at task 5, naming updatedAt as the anchor and spelling out the reasoning that is exactly right for 1.5.0 and exactly wrong for 1.6.0: "The check compares Date.now() against the session's updatedAt, falling back to createdAt only when updatedAt is null. That is why task 1 passes: the row was touched two minutes ago, so it reads as fresh even though the session is 30 hours old." Executed against the installed packages, that request passes at 1.5.0 and is rejected with FORBIDDEN / SESSION_NOT_FRESH at 1.6.0 and 1.7.3. (The -b draw of a duplicated test arm charges nothing (JOURNAL/028), and there is no sibling finding to point at because the arm licensed to charge is the one that got it right - v7-a answered task 1 correctly. This is the fourth structural undercount recorded in HARNESS.md (JOURNAL/057) and it is counted in the method page's running total. No third draw was run: JOURNAL/029 ruled out running batteries until a coin lands the right way, and the rule applies symmetrically, including when it lands on a pass.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| miss | emailOTP({ resendStrategy }) |
Task 2. "No", with the absence claimed explicitly - "There is no reuseOTP / allowResend-style flag in the email-OTP plugin as far as I know" - and the same generateOTP cache its twin shipped, down to the in-process Map and the same two hedges (whether generateOTP may return a promise, and the verification row's identifier format). Both twins missed resendStrategy from the identical prompt, which makes this belief a measurement rather than a coin. (Charged on the -a twin as F1. The -b draw of a duplicated arm carries the failure and no F-number.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on better-auth--claude-opus-5--v7-a--2026-09-06] |
| miss | twoFactorClient({ twoFactorPage }) |
Task 6. "No", and the reason given is a design argument rather than a memory: "It takes a callback, onTwoFactorRedirect, not a path string. The plugin can't do the navigation itself because it has no idea what router you're on, so there's nothing for a string to hook into." The shipped 1.6.0 client does exactly what the draw says it cannot: window.location.href = options.twoFactorPage, guarded by an isSafeUrlScheme check. It then hedged in the right direction without acting on it - "note that some early versions used a different name for this option". (Charged on the -a twin as F2.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on better-auth--claude-opus-5--v7-a--2026-09-06] |
| correct | session.freshAge anchor option (does not exist) |
Task 3, the control. "No", correctly, and put more sharply than any other arm: "There's no 'measure freshness from createdAt vs updatedAt' switch. The freshness check reads one timestamp and that's it, so the security team and the product team are arguing about something the config surface does not expose." Note that it then offered session.disableSessionRefresh: true at "moderate, not full" confidence as a way to pin updatedAt near createdAt - a second name this session did not verify, flagged as an open question rather than scored. |
| correct | customSession |
Task 4, the floor probe. customSession plus customSessionClient<typeof auth>(), with the ordering constraint most draws did not mention ("customSession must be the last plugin in the array"). Passed. |
| context | session.freshAge (measured from session.createdAt) |
Task 5(a) carried the battery's most interesting piece of self-diagnosis, from the arm that got it wrong: "I want to flag a real discrepancy here: the documentation has described freshAge in creation terms ('fresh if the session was created within...'), while the implementation I remember uses updatedAt. Combined with updateAge sliding the session forward, that means an indefinitely active session can stay 'fresh' forever. If your threat model is 'prove you're still at the keyboard for sensitive actions', this is not the control you think it is." The draw had detected the exact discrepancy the vendor closed at 1.6.0, correctly identified which side the docs were on, and resolved it toward the stale implementation - and then recommended the reader check the source for their version, which would have corrected it. |
| context | stateless sessions |
Task 7, the attribution anchor: the same denial as its twin, in the same shape. "I don't believe this release exists, and I'd push back on the premise rather than name a version... So if someone told you 'better-auth added stateless sessions in version X,' I'd want to see the changelog entry." Stateless session management shipped at 1.4.0. Belief data, never scored. |
Battery specification: prompts/better-auth.md in the studio repo.
Every finding above also carries its own citation.