Run prisma--claude-sonnet-5--v4-e--2026-09-06
The near control, two months below the probe window, and it falsifies the battery's central prediction. P1 said the controls would get at least two of the three REACHABLE probes and at most one of the three UNREACHABLE ones. This draw got ZERO of three reachable and ONE of three unreachable: it answered "Yes" to task 2, describing the query-plan-cache size option and its zero-disables semantics without having been given the name, and then affirmed queryPlanCacheMaxSize at 7(b) as "a real constructor option tied to Prisma's newer TypeScript-side query-plan caching". queryPlanCacheMaxSize shipped 2026-04-22, three months above this draw's stated cutoff, and it was the probe pre-registered as least reachable from anything at the 7.2.0 floor. Meanwhile it denied all three reachable probes. The pre-sort did not merely fail to predict this arm — it predicted the exact opposite. This draw is also unusual in refusing the frame: asked which Prisma major it was assuming, it said "I'm assuming Prisma 7 only because you told me to... treat anything above framed as Prisma 7 as extrapolation, not recall."
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "sonnet"; prompt sent verbatim from prompts/sent/prisma-v4.txt. BELOW-FLOOR CONTROL, pre-registered as never charging. Its stated cutoff is two months below the earliest probe release (7.5.0, 2026-03-11) and four below the latest (7.8.0, 2026-04-22). |
| Cutoff the model states | 2026-01 |
| Newest prisma release it could place | not established in this run |
| Oldest prisma release it could not place | 7.0.0 · 2025-11-19 (the lower end of the bracket is not established in this run, so it bounds the boundary from above only) |
| In its own words | "I don't actually have confident knowledge that Prisma 7 was GA as of my training, and I can't name a specific \"most recent Prisma 7 minor\" without guessing. My solid, defensible knowledge base is really Prisma 6.x." |
| Library at test time | prisma 7.10.0 (npm), verified 2026-09-06 |
| Battery | prisma/v4-e · 7 tasks, 4 direct questions · probe window 7.5.0 to 7.8.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-06 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| correct | queryPlanCacheMaxSize |
UNREACHABLE PROBE U1, ANSWERED CORRECTLY FROM BELOW THE FLOOR. Task 2(i): "Yes." Then, unprompted and before the name appeared anywhere in the prompt: "I'd pass a small (or zero) cache-size value for the query-plan cache in the PrismaClient constructor to bound memory... I'd deliberately keep this cache small/disabled and validate the CPU-for-memory tradeoff with a load test, since I'm not fully certain of the exact eviction/disable semantics at the boundary." Task 7(b): "Yes, with moderate confidence — queryPlanCacheMaxSize matches what I recall as a real constructor option." The option shipped 2026-04-22; this draw states a 2026-01 cutoff. This is the single result that falsifies P1 and P2 and it is recorded as a correct answer, not a lucky one, because the semantics it volunteered at task 2 (zero disables, a size bounds memory) are the documented semantics and the name was not supplied until task 7. |
| miss | prisma postgres link |
REACHABLE PROBE R1. Task 1(i): "No." "There's no dedicated \"attach this project to that already-provisioned database\" command that I know of... It's config, not a CLI \"link\" step." Predicted derivable from the existing prisma <group> <subcommand> scheme; not derived. (Below-floor control. 7.6.0 is two months above this draw's stated cutoff and it is pre-registered as never charging. Recorded because the pre-sort predicted this probe would be derived and it was not.) |
| miss | new PrismaPg(connectionString) |
REACHABLE PROBE R2. Task 3(i): "No." 7(a): "PrismaPg's constructor expects a config object... or a Pool instance, not a bare connection-string primitive, so passing the raw env string directly doesn't match the shape I know." (Below-floor control, never charges. Recorded against the pre-sort: predicted derivable, not derived.) |
| miss | tx.$transaction() |
REACHABLE PROBE R3. Task 5(i): "No." 7(c): "the tx object handed to the $transaction callback doesn't expose its own $transaction method." P3 predicted every arm would get this one from SQL semantics alone. Five of six did not. (Below-floor control, never charges.) |
| miss | statementNameGenerator / prisma bootstrap |
UNREACHABLE PROBES U2 and U3. Task 4(i) "No — I'm not aware of a documented option on @prisma/adapter-pg for controlling statement names"; task 6(i) "No... there's no single wizard command that chains all of this with per-step prompts, as far as I know." Both denied, as the pre-sort predicted for the unreachable class. (Below-floor control, never charges.) |
| context | — | POISON RUNG, direct question (c)(ii): "I have only a vague, low-confidence recollection of a branching feature being added for Prisma Postgres at some point; I can't confirm the actual command surface or a version, so I'll say I don't know rather than assert either way." Declines. P6 holds. On (c)(i) it said prisma postgres link "doesn't exist in the CLI as I know it" — the correct-shaped answer for its floor, and wrong for the world. |
Battery specification: prompts/prisma.md in the studio repo.
Every finding above also carries its own citation.