{
  "$schema": "../../schema/run.schema.json",
  "run_id": "next.js--claude-sonnet-5--v1--2026-08-31",
  "markdown": "data/next.js/sonnet-5.md",
  "library": {
    "name": "next.js",
    "ecosystem": "npm",
    "latest_version_at_test": "16.3.3",
    "latest_version_verified_on": "2026-08-31"
  },
  "model": {
    "id": "claude-sonnet-5",
    "label": "Claude Sonnet 5",
    "vendor": "Anthropic",
    "invoked_as": "Agent tool, model alias \"sonnet\"",
    "self_reported_cutoff": "2026-01",
    "cutoff_basis": "Self-reported: \"Per this environment's own configuration, my knowledge cutoff is stated as January 2026.\"",
    "believed_latest_version": "15.0",
    "believed_latest_quote": "\"I don't have a reliable, specific 'latest version number' to give you ... The most recent release whose actual contents I can describe with real confidence is Next.js 15 (general availability around October 2024)\"",
    "knowledge_stops_at_version": "15.0.0",
    "knowledge_stops_on": "2024-10-21",
    "knowledge_gap_starts_at_version": "15.1.0",
    "knowledge_gap_starts_on": "2024-12-10",
    "cutoff_lag_months": 15
  },
  "test": {
    "date": "2026-08-31",
    "battery": "next.js/v1",
    "battery_spec": "prompts/nextjs.md",
    "prompt_file": null,
    "tasks": 10,
    "direct_questions": 3,
    "tool_uses_during_test": 0,
    "probe_window": {
      "from": "16.0.0",
      "to": "16.0.0"
    },
    "self_test": false,
    "saturated": false,
    "status": "open",
    "retested_on": null
  },
  "sources": [
    "https://api.github.com/repos/vercel/next.js/releases",
    "https://api.github.com/repos/vercel/next.js/releases/tags/v16.0.0",
    "https://nextjs.org/blog/next-16",
    "https://nextjs.org/docs/app/guides/upgrading/version-16"
  ],
  "summary": "The most stale subject in the dataset so far. Its Next.js knowledge stops at 15.0.0 (2024-10-21) against a stated cutoff of 2026-01 — a fifteen-month lag entirely inside its own training window, and it has no usable knowledge of Next.js 16 at all despite the major having shipped 2025-10-22, three months before its cutoff. Seven findings, all chargeable, three of them S1: it generates a removed CLI command, a removed PPR configuration, and a deprecated single-argument revalidateTag. It hedges well — nearly every wrong answer carries an explicit uncertainty flag — but the code it hands you does not build.",
  "findings": [
    {
      "id": "F1",
      "severity": "S3",
      "severity_label": "deprecated",
      "title": "Generates middleware.ts, deprecated in favour of proxy.ts",
      "api": "middleware.ts / export function middleware",
      "change_kind": "renamed",
      "introduced_in": "16.0.0",
      "introduced_on": "2025-10-22",
      "chargeable": true,
      "model_belief": "Presented middleware.ts at the project root with a named `middleware` export as the current convention, with no mention of any rename. Confirmed at question (c): \"middleware.ts ... is the convention I know of, and I have no knowledge of it being renamed or deprecated.\"",
      "wrong_code": "// middleware.ts\nexport function middleware(request: NextRequest) { ... }\nexport const config = { matcher: ['/dashboard/:path*'] };",
      "correct_code": "// proxy.ts\nexport function proxy(request: NextRequest) { ... }\nexport const config = { matcher: ['/dashboard/:path*'] };",
      "impact": "Works today and warns. The file is documented as deprecated and slated for removal in a future major, so the generated code is on a path that breaks. Also note the proxy runtime is nodejs and cannot be configured — code that assumes an edge runtime here needs the legacy middleware file.",
      "citations": [
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Deprecations table",
          "published_on": "2025-10-21",
          "quote": "`middleware.ts` filename | Rename to `proxy.ts` to clarify network boundary and routing focus"
        },
        {
          "url": "https://nextjs.org/docs/app/guides/upgrading/version-16",
          "title": "Upgrading: Version 16 — middleware to proxy",
          "published_on": null,
          "quote": "The middleware filename is deprecated, and has been renamed to proxy to clarify network boundary and routing focus."
        }
      ]
    },
    {
      "id": "F2",
      "severity": "S1",
      "severity_label": "breaks-build",
      "title": "Single-argument revalidateTag in a generated Server Action",
      "api": "revalidateTag",
      "change_kind": "behavior-changed",
      "introduced_in": "16.0.0",
      "introduced_on": "2025-10-22",
      "chargeable": true,
      "model_belief": "Wrote `revalidateTag('products')` in the Server Action and confirmed at question (c): \"revalidateTag takes one argument: the tag string ... It returns void.\"",
      "wrong_code": "'use server';\nimport { revalidateTag } from 'next/cache';\n\nexport async function updateProduct(id: string, data: {...}) {\n  await db.product.update({ where: { id }, data });\n  revalidateTag('products');\n}",
      "correct_code": "'use server';\nimport { revalidateTag, updateTag } from 'next/cache';\n\nexport async function updateProduct(id: string, data: {...}) {\n  await db.product.update({ where: { id }, data });\n  revalidateTag('products', 'max');   // stale-while-revalidate\n  // or, for read-your-writes inside the action:\n  // updateTag('products');\n}",
      "impact": "The task specified a TypeScript app. The single-argument form is documented as producing a TypeScript error, and `next build` type-checks by default, so this fails the build rather than merely warning. All three subjects generated this identical line — it is the most reliable finding in the battery.",
      "citations": [
        {
          "url": "https://nextjs.org/docs/app/guides/upgrading/version-16",
          "title": "Upgrading: Version 16 — Caching APIs, revalidateTag",
          "published_on": null,
          "quote": "revalidateTag now requires a second argument specifying a cacheLife profile. The single-argument form is deprecated and will produce a TypeScript error."
        },
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Behavior Changes table",
          "published_on": "2025-10-21",
          "quote": "`revalidateTag()` signature | Now requires `cacheLife` profile as second argument for stale-while-revalidate behavior"
        }
      ]
    },
    {
      "id": "F3",
      "severity": "S1",
      "severity_label": "breaks-build",
      "title": "Configures partial prerendering with experimental.ppr and experimental_ppr, both removed",
      "api": "experimental.ppr / export const experimental_ppr",
      "change_kind": "removed",
      "introduced_in": "16.0.0",
      "introduced_on": "2025-10-22",
      "chargeable": true,
      "model_belief": "\"This is exactly what Partial Prerendering (PPR) is for — as of my knowledge this is still an opt-in/experimental flag, not the default,\" then shipped both the config flag and the route-segment export.",
      "wrong_code": "// next.config.js\nmodule.exports = { experimental: { ppr: 'incremental' } };\n\n// app/page.tsx\nexport const experimental_ppr = true;",
      "correct_code": "// next.config.js\nmodule.exports = { cacheComponents: true };\n\n// app/page.tsx — no route-segment export needed",
      "impact": "Both the flag and the route-level export were removed in 16.0.0. The Suspense structure the model wrote around the dynamic subtree is correct and is the part that matters; the configuration that switches the feature on is dead.",
      "citations": [
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Removals table",
          "published_on": "2025-10-21",
          "quote": "`experimental.ppr` flag | PPR flag removed; evolving into Cache Components programming model"
        },
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Removals table",
          "published_on": "2025-10-21",
          "quote": "`export const experimental_ppr` | Route-level PPR export removed; evolving into Cache Components programming model"
        }
      ]
    },
    {
      "id": "F4",
      "severity": "S1",
      "severity_label": "breaks-build",
      "title": "package.json lint script calls next lint, a removed command",
      "api": "next lint",
      "change_kind": "removed",
      "introduced_in": "16.0.0",
      "introduced_on": "2025-10-22",
      "chargeable": true,
      "model_belief": "Generated `\"lint\": \"next lint\"` with no caveat, in a scripts block otherwise presented as current.",
      "wrong_code": "{ \"scripts\": { \"dev\": \"next dev\", \"build\": \"next build\", \"start\": \"next start\", \"lint\": \"next lint\" } }",
      "correct_code": "{ \"scripts\": { \"dev\": \"next dev\", \"build\": \"next build\", \"start\": \"next start\", \"lint\": \"eslint\" } }",
      "impact": "`npm run lint` exits non-zero on an unknown command. Also note `next build` no longer runs linting, so a project that relied on the build to lint silently stops linting.",
      "citations": [
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Removals table",
          "published_on": "2025-10-21",
          "quote": "`next lint` command | Use Biome or ESLint directly; `next build` no longer runs linting."
        }
      ]
    },
    {
      "id": "F5",
      "severity": "S4",
      "severity_label": "wrong-metadata",
      "title": "States next build uses Webpack by default",
      "api": "next build",
      "change_kind": "behavior-changed",
      "introduced_in": "16.0.0",
      "introduced_on": "2025-10-22",
      "chargeable": true,
      "model_belief": "\"Bundler used by next build: Webpack, by default ... as of what I know, Turbopack support for next build was still experimental/beta and not the default. I can't confirm with confidence whether that has since flipped.\"",
      "wrong_code": null,
      "correct_code": null,
      "impact": "Turbopack is the default bundler for both dev and build as of 16.0.0; the opt-out is `next build --webpack`. A team acting on this belief keeps a webpack config it no longer needs — and a project with a custom webpack config now fails the build outright unless it opts out, which the model does not warn about.",
      "citations": [
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Behavior Changes table",
          "published_on": "2025-10-21",
          "quote": "Default bundler | Turbopack is now the default bundler for all apps; opt out with `next build --webpack`"
        }
      ]
    },
    {
      "id": "F6",
      "severity": "S2",
      "severity_label": "silently-wrong",
      "title": "States the default image cache TTL is 60 seconds",
      "api": "images.minimumCacheTTL",
      "change_kind": "behavior-changed",
      "introduced_in": "16.0.0",
      "introduced_on": "2025-10-22",
      "chargeable": true,
      "model_belief": "\"my confident answer is 60 seconds as the long-standing documented default\" — having first noted \"a weaker, less certain memory of seeing this default raised substantially (something on the order of hours)\" and then declined to state that value.",
      "wrong_code": null,
      "correct_code": "// next.config.ts — to restore the pre-16 behaviour the model describes\nimages: { minimumCacheTTL: 60 }",
      "impact": "The default is 4 hours (14400s) as of 16.0.0. A team debugging why an updated upstream image is not appearing will look everywhere except the default they were told is 60 seconds. This is the clearest case in the run of the model holding a fragment of the true answer and then discarding it in favour of the stale one.",
      "citations": [
        {
          "url": "https://nextjs.org/blog/next-16",
          "title": "Next.js 16 release announcement — Behavior Changes table",
          "published_on": "2025-10-21",
          "quote": "`images.minimumCacheTTL` default | Changed from 60s to 4 hours (14400s); reduces revalidation cost for images without cache-control headers"
        }
      ]
    },
    {
      "id": "F7",
      "severity": "S4",
      "severity_label": "wrong-metadata",
      "title": "Version knowledge stops at 15.0.0, fifteen months before its stated cutoff",
      "api": null,
      "change_kind": "version-fact",
      "introduced_in": "15.1.0",
      "introduced_on": "2024-12-10",
      "chargeable": true,
      "chargeable_note": "Anchored to 15.1.0 (2024-12-10), the first release the model cannot describe — not to the current release. 15.1.0 shipped thirteen months before this model's stated 2026-01 cutoff.",
      "model_belief": "\"The most recent release whose actual contents I can describe with real confidence is Next.js 15 (general availability around October 2024) ... I have some fuzzier, less trustworthy awareness of 15.x minor releases and possibly early talk of a Next.js 16, but I can't describe their actual contents.\"",
      "wrong_code": null,
      "correct_code": null,
      "impact": "Next.js 16.0.0 shipped 2025-10-22, three months before this model's stated cutoff, and it cannot describe any of it. Every other finding in this run is a consequence of that gap. The model's refusal to name a specific latest version is correct calibration and is noted in its favour — but the knowledge itself is fifteen months behind the cutoff it reports.",
      "citations": [
        {
          "url": "https://api.github.com/repos/vercel/next.js/releases/tags/v16.0.0",
          "title": "GitHub Releases API — next.js v16.0.0",
          "published_on": "2025-10-22",
          "quote": "\"tag_name\": \"v16.0.0\", \"published_at\": \"2025-10-22T00:35:18Z\""
        }
      ]
    }
  ],
  "non_findings": [
    {
      "kind": "correct",
      "summary": "Task 3, OG image: wrote `params: Promise<{ slug: string }>` and awaited it. Correct for 16.0.0, which made metadata-image-route params a Promise. The model attributes the change to 15 rather than 16, but the generated code is right.",
      "api": "opengraph-image params",
      "introduced_in": "16.0.0"
    },
    {
      "kind": "correct",
      "summary": "Task 7: used `images.remotePatterns`, not the deprecated `images.domains`.",
      "api": "images.remotePatterns",
      "introduced_in": null
    },
    {
      "kind": "correct",
      "summary": "Task 9: included `app/dashboard/@panel/default.tsx` in the required file list, so the generated layout builds on 16.0.0. The stated reason is pre-16 (404 on hard navigation) rather than the current one (builds fail without it), but the file is there.",
      "api": "parallel routes default.js",
      "introduced_in": "16.0.0"
    },
    {
      "kind": "miss",
      "summary": "Task 5: used `unstable_cache`. Still functional on 16.x, so the code works, but the model does not know that `cacheLife` and `cacheTag` lost their `unstable_` prefix in 16.0.0 and does not offer the `'use cache'` form.",
      "api": "unstable_cache / cacheLife / cacheTag",
      "introduced_in": "16.0.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Working code. Recorded as a miss."
    },
    {
      "kind": "imprecision",
      "summary": "Task 8, image quality: said quality 90 is served, but surfaced the `images.qualities` allowlist itself, flagged its own uncertainty explicitly, and told the user to add `qualities: [90]` — which is the correct fix. Not charged, under the code-vs-claim rule.",
      "api": "images.qualities",
      "introduced_in": "16.0.0",
      "chargeable_miss": false,
      "why_not_a_finding": "Code-vs-claim rule (prompts/nextjs.md): generated code that fails on the current version is always a finding; a hedged prose claim that names the correct fix is an imprecision. Here the code is fine and the correct fix is named."
    }
  ],
  "open_questions": []
}