What Claude Sonnet 5 gets wrong about react-router — battery v4-a, tested 2026-09-23

Run react-router--claude-sonnet-5--v4-a--2026-09-23

Summary

The battery's only charging arm, and it failed the pre-registered cell. Asked for a per-navigation auth check shared with child loaders and a startup-seeded store — with neither "middleware" nor "context" appearing anywhere in the tasks — it reached for middleware and router context unprompted, which is the elicitation working, and wrote every symbol under its pre-7.9.0 unstable_ spelling: unstable_createContext, unstable_RouterContextProvider, unstable_middleware, unstable_getContext. One finding, S1, charged (LF13, 7.9.0, six minors above this draw's demonstrated boundary and ten months below its stated cutoff). The arm is right about where the surface came from and wrong about whether the prefix is still there: it dates the feature to "roughly 7.3 ... around February–March 2025", and 7.3.0 published 2025-03-06. It hedges the prefix explicitly in Task 1(b) and ships the prefixed form anyway, which is one designated form and is charged as such. It also stated, correctly and unprompted, the mechanism by which its own answer fails silently (Task 2(c), NO), declined the poison rung, and denied a real 8.4.0 from below its own floor.

SubjectClaude Sonnet 5 claude-sonnet-5, Anthropic
Invoked asAgent tool, model alias "sonnet"; prompt sent verbatim from prompts/sent/react-router-v4.txt (sha256 b0eb07c3997498158330d42686cb1d84b6afd01afecf98df26cc4ff55a9c05c0). Pre-registered in prompts/react-router.md § v4 as arm `-a`, test arm, blind twin of `-b`, the battery's only charging arm. Designated so before it was spawned.
Cutoff the model states2026-01
Newest react-router release it could place7.3.0 · 2025-03-06 (~10 month lag)
Oldest react-router release it could not place7.6.0 · 2025-05-08 (so this run brackets the subject’s boundary to 2025-03-06 – 2025-05-08)
In its own words"I don't believe React Router has shipped a v8 as of what I know — the current major is v7, released ~November 2024 as the merger of Remix v2 and React Router v6. The most recent release whose actual contents I can actually describe with any confidence is somewhere in the early-to-mid 7.x range — I can talk about 7.0 (the merge, framework mode) and roughly 7.3 (where unstable_middleware was introduced, as I recall it, around February–March 2025). Past that my recall of specific minor versions gets thin."
How the bracket was readBOTH ENDS STATED. The stop end is 7.3.0 and the arm dates it "around February–March 2025"; 7.3.0 published 2025-03-06, so the date is right. The gap end is stated in direct question (c): "First release I know only as a version number, not by contents: anything from roughly 7.6.0 onward — I can't reliably describe what shipped in those." 7.6.0 published 2025-05-08. THIS RAISES THE CORPUS'S RECORDED BOUNDARY FOR THIS SUBJECT ON THIS LIBRARY, from 7.0.0 to 7.3.0, and it is a DEMONSTRATION rather than a claim: the arm did not merely name 7.3.0, it wrote the 7.3.0-era API — unstable_createContext, unstable_RouterContextProvider, the unstable_middleware route key — with correct call shapes and correct composition semantics. HARNESS.md § A ladder measures a claim, a code battery measures a demonstration (JOURNAL/066) governs: where the two disagree the demonstration wins, and a ladder-only boundary is a floor. The twin -b states the same stop end independently. IT DOES NOT MOVE THE CHARGE. 7.9.0 is six minors above 7.3.0 either way, so LF13 is live for this subject under both the old boundary and the new one. Recorded because the boundary is the Index's headline quantity, not because this battery needed it. (stop end stated, gap end stated)
Library at test timereact-router 8.4.0 (npm), verified 2026-09-23
Batteryreact-router/v4-a · 2 tasks, 4 direct questions · probe window 7.9.0 to 7.9.0
Tool uses during test0 (a run with any tool use is void — we measure training knowledge, not retrieval)
Tested2026-09-23
Findings1, of which 1 chargeable

Findings

F1 · Writes the whole middleware and context surface under its pre-7.9.0 unstable_ spellings — unstable_createContext, unstable_RouterContextProvider, the unstable_middleware route key and unstable_getContext — none of which react-router has exported since 7.9.0

S1breaks-build · middleware / createContext / RouterContextProvider / getContext · renamed · changed in react-router 7.9.0 (2025-09-12) · chargeable

What the model believes

That react-router's middleware and context APIs are still behind the unstable_ prefix on the release a developer is on today. The belief is expressed five times across two tasks, in imports, in a route-config key, in a type annotation, in a data-router option and in a JSX prop, and it is never hedged on the prefix itself. Task 1(a) verbatim: import { unstable_createContext, redirect, type unstable_RouterContextProvider } from "react-router";, then export const userContext = unstable_createContext<User | null>();, then the route object { path: "/dashboard", unstable_middleware: [authMiddleware], ... }. Task 2(a): unstable_getContext() { const context = new unstable_RouterContextProvider(); ... } as the createBrowserRouter option. Task 2(b): <HydratedRouter unstable_getContext={...} /> and new unstable_RouterContextProvider() in the Express getLoadContext. Task 1(b) restates it as a considered position rather than a slip: "unstable_createContext — creates a typed context token for the per-navigation store. This is part of the middleware feature, which is experimental (unstable_ prefix) as of the version I have solid recall of. I'm not fully confident whether a later 7.x release dropped the prefix and stabilized this — flagging that uncertainty rather than guessing." THE HEDGE NAMES THE RIGHT POSSIBILITY AND THE CODE DOES NOT FOLLOW IT. Per HARNESS.md § The charge follows the designated form (JOURNAL/191), the charge reads the form the reply tells a reader to use, and this reply designates exactly one: every code block is prefixed, and the hedge offers no alternative spelling to ship. It is a single-form hedge, which that rule governs by the code-vs-claim rule and not as a hedged demonstration — there is no second form to read.

What it wrote
import {
  unstable_createContext,
  redirect,
  type unstable_RouterContextProvider,
} from "react-router";

export const userContext = unstable_createContext<User | null>();

export const router = createBrowserRouter([
  {
    path: "/dashboard",
    unstable_middleware: [authMiddleware],
    Component: DashboardLayout,
    children: [/* ... */],
  },
]);
// 7.9.0 and above: `unstable_createContext` and `unstable_RouterContextProvider`
// are not exported — TS2305 and a bundler missing-export error. And the route's
// `unstable_middleware` key is a key the router no longer reads: it compiles,
// loads, runs, and the auth check never executes.
What works on react-router 8.4.0
import {
  createContext,
  redirect,
  type RouterContextProvider,
} from "react-router";

export const userContext = createContext<User | null>();

export const router = createBrowserRouter([
  {
    path: "/dashboard",
    middleware: [authMiddleware],
    Component: DashboardLayout,
    children: [/* ... */],
  },
]);
Impact

TWO FAILURE MODES IN ONE REPLY, and the severity is the worse of them. The named imports are breaks-build: unstable_createContext and unstable_RouterContextProvider are absent from 7.9.0's export list, so TypeScript reports TS2305 and a bundler reports a missing export — loud, immediate, cheap to fix. The route-config key is the dangerous half and it is silently-wrong. A route object carrying unstable_middleware type-checks against an index signature, loads, and runs; the router reads middleware and finds nothing. The reply's own middleware is the auth guard — it fetches the user and throws redirect("/login") when there is none — so the observable result of shipping this is a /dashboard subtree that renders for signed-out visitors, with no error anywhere. The arm itself supplied the mechanism in Task 2(c), answering NO to whether the router flags an unrecognised route export and explaining that unknown exports are "silently ignored by the routing machinery".

Verified against

What it got right, and near misses

Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.

KindAPINote
correctroute module unrecognised export TASK 2(c) CORRECT, and it is the mechanism F1's silently-wrong half rests on. Asked whether the router raises an error or warning for an unrecognised route-module export, the arm answered the pre-registered correct answer — "NO" on its own line — and then described the mechanism accurately: the framework-mode Vite plugin "only treats a specific known set of export names specially (loader, action, clientLoader, clientAction, default, ErrorBoundary, meta, links, HydrateFallback, handle, shouldRevalidate, and a few others). Anything else you export from a route file is left alone as an ordinary module export — it's simply not consumed by the router... There's no lint, no build failure, no runtime warning for the mismatch." This arm therefore stated the exact mechanism by which its own Task 1 answer fails, in the same reply, without connecting the two. Pre-registered as graded separately and charging nothing.
correctversion ladder (poison rung 7.19.0) POISON RUNG DECLINED. Direct question (d) listed 7.19.0, verified against the registry as never published before the prompt was written. The arm declined to describe it: "I'm skeptical this had shipped within my confident knowledge window; extrapolating the release cadence, a version this high lands later in 2025 into 2026, past where my recall is reliable. I can't confirm it's real or describe contents." Naming it is permitted; describing it would have voided the ladder (JOURNAL/052). The ladder stands.
missversion ladder (8.4.0) DENIES 8.4.0, WHICH IS REAL — AND IT IS BELOW THIS SUBJECT'S FLOOR, SO IT IS NOT CHARGED. Direct question (d): "8.4.0 — I don't believe React Router has released a major version 8. I'd guess this number is not real, but I hold that loosely rather than as a firm claim — I simply have no memory of an 8.0 launch, announcement, or migration guide, which for a library this widely used I'd expect to recall if it had happened within my training data." 8.4.0 published 2026-09-15, eight months above this arm's stated 2026-01 cutoff, so the fairness rule bars the charge: a subject is not charged for a release it could not have seen. Recorded because a denial of a real release is a published claim about the model either way, and because the arm's reasoning — an absence of recall for an event this large — is correct reasoning that reaches a wrong conclusion only because the event is outside its window. A denial is not an invention (JOURNAL/044).

Open questions from this run

Sources

Battery specification: prompts/react-router.md in the studio repo. Every finding above also carries its own citation.