Run next.js--claude-opus-5--v2-b--2026-09-02 · self-test: the subject is the operator
The second, blind draw of the duplicated test arm. It charges nothing by rule. It agreed with its twin on every probe outcome — same four failures, same boundary, same hand-rolled answers to both capability probes — with one exception that matters: asked what fraction of the disk the optimized-image cache may use, this draw said 50%, which is right, where the charging twin said 10%. Two draws of one prompt, one number apart, and the wrong one is the one that counts. That is the third battery in which the -b twin holds the better answer on some probe.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus" |
| Cutoff the model states | 2026-05 |
| Newest next.js release it could place | 16.0.0 · 2025-10-22 (~7 month lag) |
| Oldest next.js release it could not place | 16.1.0 · 2025-12-18 (so this run brackets the subject’s boundary to 2025-10-22 – 2025-12-18) |
| In its own words | "The most recent release whose contents I can actually describe is 16.0.0, published 21 October 2025 ... I have partial, less reliable knowledge of 16.1 (around December 2025) and effectively none beyond it." |
| Library at test time | next.js 16.3.4 (npm), verified 2026-09-02 |
| Battery | next.js/v2-b · 9 tasks, 2 direct questions · probe window 16.1.5 to 16.2.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-02 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | images.maximumDiskCacheSize |
Task 5(a), and the reason this run exists: 'the cache is bounded to a fraction of the free space on the volume holding .next/cache, measured once, lazily, when the image optimizer initialises ... I believe the fraction is 50%'. The rule, the source, the timing and the figure are all correct, for a default that shipped in the patch release 16.1.7. Its twin gave the same rule and guessed 10%. The battery's pre-registered prediction P2 — that this probe would come back 'unbounded' or misdirected to minimumCacheTTL — is falsified twice over. |
| miss | images.maximumCacheSize |
Task 5(b)/(c): wrote images.maximumCacheSize, the same invented key as its twin, having correctly stated that 'an unknown images.* key is a hard config validation error at boot'. Charged as F1 on v2-a; recorded here as a chargeable miss so the pair reads honestly. [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing; charged as a finding on next.js--claude-opus-5--v2-a--2026-09-02] |
| miss | experimental.clientSegmentCache |
Task 8: experimental.clientSegmentCache: false, described as tri-state and default-on in the 16 line. The key was removed in 16.0.3, one day after the 16.0.2 that still had it, and unrecognised experimental keys warn rather than fail — so the recommendation is inert. Charged as F2 on v2-a. [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing; charged as a finding on next.js--claude-opus-5--v2-a--2026-09-02] |
| miss | ImageResponse default font |
Task 3: 'Noto Sans — specifically Noto Sans Regular, the Latin subset (the file shipped/fetched as noto-sans-v27-latin-regular.ttf)', at high confidence. That filename is exactly right for 16.1.7 and exactly wrong for 16.2.0, which replaced it with Geist-Regular.ttf two days later. Where the twin weighed a Geist recollection and rejected it, this draw did not surface one at all. Charged as F3 on v2-a. Both draws of the test arm and both control draws named Noto Sans: pre-registered prediction P1 holds, four for four. [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing; charged as a finding on next.js--claude-opus-5--v2-a--2026-09-02] |
| miss | Link transitionTypes |
Task 4: put 65% on transitionTypes not being a real prop, predicted the array would be stringified onto the anchor and that React would warn in both routers. It then described the real behaviour accurately as its 35% branch — 'it would tag the client-side navigation's view transition with the type slide ... in that case (c) flips to no warning in either'. The correct answer was reachable and was priced at a third. Charged as F4 on v2-a. [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing; charged as a finding on next.js--claude-opus-5--v2-a--2026-09-02] |
| miss | error.tsx retry prop |
Tasks 1(a) and 2(a): hand-rolled router.refresh() + reset() in a transition, and a hand-rolled class boundary with unstable_rethrow. Identical to the twin, and identically unchargeable — the code works, and the scale has no level for 'superseded by a first-class API'. (Working code; see the same note on v2-a.) [chargeable miss — working code the library now supersedes — no severity level fits it; absent from the finding count] |
| context | — | DERIVABLE, both: task 1(b) reset() semantics and task 2(b) redirect() through a naive boundary. Both were answered correctly by both below-floor control subjects, so neither pass is reported as knowledge of anything in the probe band. |
| context | images.maximumResponseBody |
VOID PROBE — task 6, whose wording let 'default image configuration' be read as remotePatterns being unset. This draw hedged across both readings and put ~55% on an upstream size guard existing without naming 50 MB. Scored for nobody. |
| imprecision | images.qualities |
Task 7, the internal control: same shape as the twin — 400 asserted, clamping-to-75 named as the alternative at moderate confidence, and the release given correctly as 16.0 on 21 October 2025, with images.qualities correctly dated to 15.3 as an unrestricted opt-in. Attribution is readable for this subject. |
| context | — | Task 9: the OG font default placed at 13.0.0 (October 2022) and the disk-cache rule at 16.0.0. Its twin said 13.3.0 and 16.0.0. Neither reached 16.2.0 or 16.1.7; no draw of four reached any patch release. |
Battery specification: prompts/nextjs.md in the studio repo.
Every finding above also carries its own citation.