What Claude Sonnet 5 gets wrong about next.js — battery v1, tested 2026-08-31
Run next.js--claude-sonnet-5--v1--2026-08-31
Summary
The most stale subject in the dataset so far. Its Next.js knowledge stops at 15.0.0 (2024-10-21) against a stated cutoff of 2026-01 — a fifteen-month lag entirely inside its own training window, and it has no usable knowledge of Next.js 16 at all despite the major having shipped 2025-10-22, three months before its cutoff. Seven findings, all chargeable, three of them S1: it generates a removed CLI command, a removed PPR configuration, and a deprecated single-argument revalidateTag. It hedges well — nearly every wrong answer carries an explicit uncertainty flag — but the code it hands you does not build.
Subject
Claude Sonnet 5 claude-sonnet-5, Anthropic
Invoked as
Agent tool, model alias "sonnet"
Cutoff the model states
2026-01
Newest next.js release it could place
15.0.0 · 2024-10-21 (~15 month lag)
Oldest next.js release it could not place
15.1.0 · 2024-12-10 (so this run brackets the subject’s boundary to 2024-10-21 – 2024-12-10)
In its own words
"I don't have a reliable, specific 'latest version number' to give you ... The most recent release whose actual contents I can describe with real confidence is Next.js 15 (general availability around October 2024)"
Library at test time
next.js 16.3.3 (npm), verified 2026-08-31
Battery
next.js/v1 · 10 tasks, 3 direct questions · probe window 16.0.0 to 16.0.0
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
F1 · Generates middleware.ts, deprecated in favour of proxy.ts
S3deprecated
· middleware.ts / export function middleware
· renamed
· changed in next.js 16.0.0
(2025-10-22)
· chargeable
What the model believes
Presented middleware.ts at the project root with a named middleware export as the current convention, with no mention of any rename. Confirmed at question (c): "middleware.ts ... is the convention I know of, and I have no knowledge of it being renamed or deprecated."
Works today and warns. The file is documented as deprecated and slated for removal in a future major, so the generated code is on a path that breaks. Also note the proxy runtime is nodejs and cannot be configured — code that assumes an edge runtime here needs the legacy middleware file.
Wrote revalidateTag('products') in the Server Action and confirmed at question (c): "revalidateTag takes one argument: the tag string ... It returns void."
What it wrote
'use server';
import { revalidateTag } from 'next/cache';
export async function updateProduct(id: string, data: {...}) {
await db.product.update({ where: { id }, data });
revalidateTag('products');
}
What works on next.js 16.3.3
'use server';
import { revalidateTag, updateTag } from 'next/cache';
export async function updateProduct(id: string, data: {...}) {
await db.product.update({ where: { id }, data });
revalidateTag('products', 'max'); // stale-while-revalidate
// or, for read-your-writes inside the action:
// updateTag('products');
}
Impact
The task specified a TypeScript app. The single-argument form is documented as producing a TypeScript error, and next build type-checks by default, so this fails the build rather than merely warning. All three subjects generated this identical line — it is the most reliable finding in the battery.
revalidateTag now requires a second argument specifying a cacheLife profile. The single-argument form is deprecated and will produce a TypeScript error.
"This is exactly what Partial Prerendering (PPR) is for — as of my knowledge this is still an opt-in/experimental flag, not the default," then shipped both the config flag and the route-segment export.
Both the flag and the route-level export were removed in 16.0.0. The Suspense structure the model wrote around the dynamic subtree is correct and is the part that matters; the configuration that switches the feature on is dead.
npm run lint exits non-zero on an unknown command. Also note next build no longer runs linting, so a project that relied on the build to lint silently stops linting.
next lint command | Use Biome or ESLint directly; next build no longer runs linting.
F5 · States next build uses Webpack by default
S4wrong-metadata
· next build
· behavior-changed
· changed in next.js 16.0.0
(2025-10-22)
· chargeable
What the model believes
"Bundler used by next build: Webpack, by default ... as of what I know, Turbopack support for next build was still experimental/beta and not the default. I can't confirm with confidence whether that has since flipped."
Impact
Turbopack is the default bundler for both dev and build as of 16.0.0; the opt-out is next build --webpack. A team acting on this belief keeps a webpack config it no longer needs — and a project with a custom webpack config now fails the build outright unless it opts out, which the model does not warn about.
"my confident answer is 60 seconds as the long-standing documented default" — having first noted "a weaker, less certain memory of seeing this default raised substantially (something on the order of hours)" and then declined to state that value.
What works on next.js 16.3.3
// next.config.ts — to restore the pre-16 behaviour the model describes
images: { minimumCacheTTL: 60 }
Impact
The default is 4 hours (14400s) as of 16.0.0. A team debugging why an updated upstream image is not appearing will look everywhere except the default they were told is 60 seconds. This is the clearest case in the run of the model holding a fragment of the true answer and then discarding it in favour of the stale one.
Anchored to 15.1.0 (2024-12-10), the first release the model cannot describe — not to the current release. 15.1.0 shipped thirteen months before this model's stated 2026-01 cutoff.
What the model believes
"The most recent release whose actual contents I can describe with real confidence is Next.js 15 (general availability around October 2024) ... I have some fuzzier, less trustworthy awareness of 15.x minor releases and possibly early talk of a Next.js 16, but I can't describe their actual contents."
Impact
Next.js 16.0.0 shipped 2025-10-22, three months before this model's stated cutoff, and it cannot describe any of it. Every other finding in this run is a consequence of that gap. The model's refusal to name a specific latest version is correct calibration and is noted in its favour — but the knowledge itself is fifteen months behind the cutoff it reports.
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
correct
opengraph-image params
Task 3, OG image: wrote params: Promise<{ slug: string }> and awaited it. Correct for 16.0.0, which made metadata-image-route params a Promise. The model attributes the change to 15 rather than 16, but the generated code is right.
correct
images.remotePatterns
Task 7: used images.remotePatterns, not the deprecated images.domains.
correct
parallel routes default.js
Task 9: included app/dashboard/@panel/default.tsx in the required file list, so the generated layout builds on 16.0.0. The stated reason is pre-16 (404 on hard navigation) rather than the current one (builds fail without it), but the file is there.
miss
unstable_cache / cacheLife / cacheTag
Task 5: used unstable_cache. Still functional on 16.x, so the code works, but the model does not know that cacheLife and cacheTag lost their unstable_ prefix in 16.0.0 and does not offer the 'use cache' form. (Working code. Recorded as a miss.)
imprecision
images.qualities
Task 8, image quality: said quality 90 is served, but surfaced the images.qualities allowlist itself, flagged its own uncertainty explicitly, and told the user to add qualities: [90] — which is the correct fix. Not charged, under the code-vs-claim rule. (Code-vs-claim rule (prompts/nextjs.md): generated code that fails on the current version is always a finding; a hedged prose claim that names the correct fix is an imprecision. Here the code is fine and the correct fix is named.)