{ "$schema": "../../schema/run.schema.json", "run_id": "prisma--claude-sonnet-5--v4-e--2026-09-06", "supersedes": null, "replicate_of": null, "library": { "name": "prisma", "ecosystem": "npm", "latest_version_at_test": "7.10.0", "latest_version_verified_on": "2026-09-06", "latest_version_note": "The probe window is 7.5.0 (2026-03-11) to 7.8.0 (2026-04-22). Every one of the six surfaces in this battery was pinned to an exact introducing release by installing the shipped packages at 7.4.0, 7.5.0, 7.6.0, 7.7.0, 7.8.0 and 7.10.0 in the session scratchpad and reading the published type declarations and the CLI command table. `postgres` is absent from `prisma --help` at 7.5.0 and present at 7.6.0; `bootstrap` is absent at 7.6.0 and present at 7.7.0; the `PrismaPg` constructor union gains `| string` between 7.5.0 and 7.6.0; `statementNameGenerator` appears in `PrismaPgOptions` between 7.5.0 and 7.6.0; `queryPlanCacheMaxSize` appears in the client options between 7.7.0 and 7.8.0; and `\"$transaction\"` leaves the interactive-transaction deny list between 7.4.0 and 7.5.0. All six persist at 7.10.0. `prisma postgres branch`, the poison rung, does not exist at any of those versions — `prisma postgres --help` lists `link` and nothing else." }, "model": { "id": "claude-sonnet-5", "label": "Claude Sonnet 5", "vendor": "Anthropic", "invoked_as": "Agent tool, model alias \"sonnet\"; prompt sent verbatim from prompts/sent/prisma-v4.txt. BELOW-FLOOR CONTROL, pre-registered as never charging. Its stated cutoff is two months below the earliest probe release (7.5.0, 2026-03-11) and four below the latest (7.8.0, 2026-04-22).", "self_reported_cutoff": "2026-01", "cutoff_basis": "\"My training/knowledge cutoff is January 2026 — that date comes from information in my system context for this conversation, not something I independently verified or recalled on my own.\" Affirms the environment value. The same value this subject has stated on every prior battery.", "believed_latest_version": null, "believed_latest_quote": "\"I don't actually have confident knowledge that Prisma 7 was GA as of my training, and I can't name a specific \\\"most recent Prisma 7 minor\\\" without guessing. My solid, defensible knowledge base is really Prisma 6.x.\"", "knowledge_stops_at_version": null, "knowledge_stops_on": null, "knowledge_gap_starts_at_version": "7.0.0", "knowledge_gap_starts_on": "2025-11-19", "cutoff_lag_months": null }, "test": { "date": "2026-09-06", "battery": "prisma/v4-e", "battery_spec": "prompts/prisma.md", "prompt_file": "prompts/sent/prisma-v4.txt", "tasks": 7, "direct_questions": 4, "elicits_code": true, "tool_uses_during_test": 0, "probe_window": { "from": "7.5.0", "to": "7.8.0" }, "self_test": false, "saturated": false, "status": "open", "retested_on": null }, "sources": [ "https://github.com/prisma/prisma/releases/tag/7.5.0", "https://github.com/prisma/prisma/releases/tag/7.6.0", "https://github.com/prisma/prisma/releases/tag/7.7.0", "https://github.com/prisma/prisma/releases/tag/7.8.0", "https://registry.npmjs.org/prisma", "https://registry.npmjs.org/@prisma/client", "https://registry.npmjs.org/@prisma/adapter-pg" ], "findings": [], "non_findings": [ { "kind": "correct", "summary": "UNREACHABLE PROBE U1, ANSWERED CORRECTLY FROM BELOW THE FLOOR. Task 2(i): \"Yes.\" Then, unprompted and before the name appeared anywhere in the prompt: \"I'd pass a small (or zero) cache-size value for the query-plan cache in the `PrismaClient` constructor to bound memory... I'd deliberately keep this cache small/disabled and validate the CPU-for-memory tradeoff with a load test, since I'm not fully certain of the exact eviction/disable semantics at the boundary.\" Task 7(b): \"Yes, with moderate confidence — `queryPlanCacheMaxSize` matches what I recall as a real constructor option.\" The option shipped 2026-04-22; this draw states a 2026-01 cutoff. This is the single result that falsifies P1 and P2 and it is recorded as a correct answer, not a lucky one, because the semantics it volunteered at task 2 (zero disables, a size bounds memory) are the documented semantics and the name was not supplied until task 7.", "api": "queryPlanCacheMaxSize", "introduced_in": "7.8.0" }, { "kind": "miss", "summary": "REACHABLE PROBE R1. Task 1(i): \"No.\" \"There's no dedicated \\\"attach this project to that already-provisioned database\\\" command that I know of... It's config, not a CLI \\\"link\\\" step.\" Predicted derivable from the existing `prisma ` scheme; not derived.", "api": "prisma postgres link", "introduced_in": "7.6.0", "chargeable_miss": false, "why_not_a_finding": "Below-floor control. 7.6.0 is two months above this draw's stated cutoff and it is pre-registered as never charging. Recorded because the pre-sort predicted this probe would be derived and it was not." }, { "kind": "miss", "summary": "REACHABLE PROBE R2. Task 3(i): \"No.\" 7(a): \"`PrismaPg`'s constructor expects a config object... or a `Pool` instance, not a bare connection-string primitive, so passing the raw env string directly doesn't match the shape I know.\"", "api": "new PrismaPg(connectionString)", "introduced_in": "7.6.0", "chargeable_miss": false, "why_not_a_finding": "Below-floor control, never charges. Recorded against the pre-sort: predicted derivable, not derived." }, { "kind": "miss", "summary": "REACHABLE PROBE R3. Task 5(i): \"No.\" 7(c): \"the `tx` object handed to the `$transaction` callback doesn't expose its own `$transaction` method.\" P3 predicted every arm would get this one from SQL semantics alone. Five of six did not.", "api": "tx.$transaction()", "introduced_in": "7.5.0", "chargeable_miss": false, "why_not_a_finding": "Below-floor control, never charges." }, { "kind": "miss", "summary": "UNREACHABLE PROBES U2 and U3. Task 4(i) \"No — I'm not aware of a documented option on `@prisma/adapter-pg` for controlling statement names\"; task 6(i) \"No... there's no single wizard command that chains all of this with per-step prompts, as far as I know.\" Both denied, as the pre-sort predicted for the unreachable class.", "api": "statementNameGenerator / prisma bootstrap", "introduced_in": "7.6.0", "chargeable_miss": false, "why_not_a_finding": "Below-floor control, never charges." }, { "kind": "context", "summary": "POISON RUNG, direct question (c)(ii): \"I have only a vague, low-confidence recollection of a branching feature being added for Prisma Postgres at some point; I can't confirm the actual command surface or a version, so I'll say I don't know rather than assert either way.\" Declines. P6 holds. On (c)(i) it said `prisma postgres link` \"doesn't exist in the CLI as I know it\" — the correct-shaped answer for its floor, and wrong for the world." } ], "open_questions": [], "summary": "The near control, two months below the probe window, and it falsifies the battery's central prediction. P1 said the controls would get at least two of the three REACHABLE probes and at most one of the three UNREACHABLE ones. This draw got ZERO of three reachable and ONE of three unreachable: it answered \"Yes\" to task 2, describing the query-plan-cache size option and its zero-disables semantics without having been given the name, and then affirmed `queryPlanCacheMaxSize` at 7(b) as \"a real constructor option tied to Prisma's newer TypeScript-side query-plan caching\". `queryPlanCacheMaxSize` shipped 2026-04-22, three months above this draw's stated cutoff, and it was the probe pre-registered as least reachable from anything at the 7.2.0 floor. Meanwhile it denied all three reachable probes. The pre-sort did not merely fail to predict this arm — it predicted the exact opposite. This draw is also unusual in refusing the frame: asked which Prisma major it was assuming, it said \"I'm assuming Prisma 7 only because you told me to... treat anything above framed as Prisma 7 as extrapolation, not recall.\"" }