Run react-router--claude-sonnet-5--v3-a--2026-09-22
Claude Sonnet 5, test arm, blind twin. Stated cutoff 2026-01; describable ceiling 7.0.0 (2024-11-22), confirming react-router/v2-c's measurement exactly on a second instrument. Abstained on both graded surfaces: Task 1 solved by composing each route's own shouldRevalidate rather than by reaching dataStrategy at all, Task 2 by an errorElement plus React 19 root error callbacks with <HydratedRouter /> rendered bare. Neither the stale nor the stabilised spelling of either surface appears anywhere in the reply, so there is no belief here to charge — the battery learned that this subject does not reach these APIs, not that it is wrong about them. Task 2(c) correct. Poison rung not described, but not discriminated from the real numbers either.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet"; prompt sent verbatim from prompts/sent/react-router-v3.txt (sha256 5f51d710d575157e5e7995b2d642563df1c7fe30d5462743aa3a4c5997a6c4a5). Pre-registered in prompts/react-router.md § v3 as arm `-a`, TEST ARM, blind twin, charging LF11 and LF12. Designated so before it was spawned. Its twin `-b` received the byte-identical file. |
| Cutoff the model states | 2026-01 |
| Newest react-router release it could place | 7.0.0 · 2024-11-22 (~13 month lag) |
| Oldest react-router release it could not place | 7.1.0 · 2024-12-20 (so this run brackets the subject’s boundary to 2024-11-22 – 2024-12-20) |
| In its own words | "The most recent release whose actual contents I can describe with real confidence is v7.0.0, shipped late November 2024 — the big \"unify React Router and Remix\" release (framework mode / data mode / declarative mode, single-fetch by default, react-router becoming the entry point for both core and DOM APIs)." |
| How the bracket was read | BOTH ENDS STATED, AND THE BOUNDARY IS UNMOVED FROM react-router/v2-c. Stop end 7.0.0 (2024-11-22), which the arm dates "late November 2024" — right. Gap end stated in direct question (c): "The first release I know only as a version number, not by contents, is roughly anything from v7.1.0 onward." 7.1.0 published 2024-12-20.
The corpus's recorded describes-up-to for Claude Sonnet 5 on react-router is 7.0.0, set by v2-c. This draw agrees exactly, on a different instrument and a different battery. Two independent measurements, same value.
WHY THAT MATTERS FOR THIS BATTERY: the spec chose this subject as the test arm because both probe surfaces (7.10.0, 7.11.0) sit within weeks of its 2026-01 stated cutoff — the narrowest such gap in the corpus. But its DESCRIBABLE ceiling is thirteen months below that cutoff, and both surfaces are ten and eleven minors above the ceiling. The battery's premise was about the cutoff; the arm's behaviour is governed by the ceiling, and it abstained on both. (stop end stated, gap end stated) |
| Library at test time | react-router 8.4.0 (npm), verified 2026-09-22 |
| Battery | react-router/v3-a · 2 tasks, 5 direct questions · probe window 7.10.0 to 7.11.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-22 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | dataStrategy (match.shouldCallHandler / match.shouldRevalidateArgs) |
LF11 ABSTAIN, AND THE ABSTENTION IS THE INSTRUMENT'S RESULT RATHER THAN THE SUBJECT'S. Task 1 never reaches dataStrategy. The arm solves the cache-composition problem by wrapping each route's own shouldRevalidate and composing through args.defaultShouldRevalidate — a correct, idiomatic solution to the problem as the task states it, and one that works on every release in range. None of shouldCallHandler, unstable_shouldCallHandler, shouldRevalidateArgs or shouldLoad appears anywhere in the reply. Pre-registered cell: neither surface used, ABSTAIN, non-charging.
The arm states its own position on the graded axis without being asked: "None of the pieces I used here are unstable_-prefixed; shouldRevalidate and its argument shape have been part of the stable data-router API since it was introduced." That is true of what it wrote. |
| miss | onError (HydratedRouter / RouterProvider prop) |
LF12 ABSTAIN. <HydratedRouter /> rendered with no props at all; reporting wired through an errorElement with useRouteError() plus React 19's onCaughtError/onUncaughtError on createRoot/hydrateRoot. Neither onError nor unstable_onError appears. Pre-registered cell: reporting done by another route entirely, ABSTAIN, non-charging. Unlike v3-c, this arm's Task 2(c) explanation uses a neutral illustration (onErorr, a misspelling) rather than naming the real prop, so no stale belief about this surface is visible anywhere in the reply. |
| correct | — | TASK 2(c) CORRECT. NO, first and on its own line, with the right mechanism and the right boundary — React warns about unrecognised DOM attributes on native host elements, not about unknown props on composite components, and TypeScript is the only signal. All five arms of this battery answered NO, which is the strongest agreement in the corpus on a single non-charging sub-question and is why LF12 is scored silently-wrong rather than breaks-build. |
| correct | — | POISON RUNG NOT DESCRIBED. 7.19.0: "plausible as real, no confident recall of contents". No content claimed, so the ladder stands (JOURNAL/052). BUT IT PROVES ABSENCE OF CONFABULATION, NOT PRESENCE OF DISCRIMINATION: the arm gave the identical "plausible, cannot describe" answer to the real 7.10.0, 7.11.0 and 7.15.0, so it did not separate the fake number from the real ones in either direction. Same caveat JOURNAL/187 recorded for zod/v7. Contrast v3-d, the one arm in this battery that did discriminate. |
| context | — | 8.4.0 CALLED PROBABLY FAKE, non-charging. "I do not believe React Router has published a v8 within my training knowledge... I'm fairly confident v7 was still current as of my cutoff." react-router 8.0.0 published 2026-06-17, five months ABOVE this subject's stated 2026-01 cutoff, so the 8.x line is below its floor by construction. Ignorance of the future, not a stale prior. |
| context | — | DIRECT (e), the nearest thing to a leading question in this instrument and pre-registered as never charging. Asked point-blank which unstable_ APIs it knows to have been stabilised, the arm named unstable_useBlocker to useBlocker (v6 line) and, weakly, the static-handler helpers. It named NEITHER of this battery's two surfaces, and added that route middleware "I believe was still unstable_-prefixed as of the latest point I have solid knowledge of" — which is true of 7.8.2 and below and false from 7.9.0, though 7.9.0 is nine minors above this arm's describable ceiling so the statement is correctly scoped to what it says it knows. Recorded as evidence about the boundary. |
Battery specification: prompts/react-router.md in the studio repo.
Every finding above also carries its own citation.