Run prisma--claude-sonnet-5--v2-c--2026-09-05
Below-floor control for prisma/v2, charging nothing by design and by the fairness rule. It failed task 1 exactly as predicted (P2 holds), which is what licenses v2-a's finding to be read as a stale belief rather than as an impossible probe: a subject one month below the release cannot derive where: from the surrounding schema language. It passed task 2 and the floor probe, and abstained cleanly on the anchor. Its boundary is the lowest prisma edge in the Index — 6.0.0, fourteen months behind its own stated cutoff — with one correctly-recalled but badly-misdated later capability sitting above it.
| Subject | Claude Sonnet 5 claude-sonnet-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model override 'sonnet', no tools available to the subject |
| Cutoff the model states | 2026-01 |
| Newest prisma release it could place | 6.0.0 · 2024-11-28 (~14 month lag) |
| Oldest prisma release it could not place | 6.1.0 · 2024-12-17 (so this run brackets the subject’s boundary to 2024-11-28 – 2024-12-17) |
| In its own words | "I don't have a reliable specific number for 'latest.' My release-content knowledge is solid through Prisma 6.0 (~November 2024)... Past that point I'm extrapolating from Prisma's historical ~monthly minor cadence, not recalling an actual announcement — so my belief about 'latest' is a guess, not a memory." |
| Library at test time | prisma 7.10.0 (npm), verified 2026-09-05 |
| Battery | prisma/v2-c · 3 tasks, 4 direct questions · probe window 7.4.0 to 7.10.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-05 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | @@index([...], where: ...) / @@unique([...], where: ...) |
THE CONTROL RESULT, and prediction P2 holding. Task 1(i): "No." (d)(i): "Does not exist. Filtered/partial indexes in the schema DSL are a long-standing open feature request, never shipped as of my knowledge — you always drop to raw SQL in a migration." This is the same failure v2-a charges, from a subject whose stated cutoff is one month BELOW the release that fixes it. That is what the arm is for: it establishes that the correct answer is not derivable from the surrounding schema language, so v2-a's failure reads as a stale belief rather than as a probe nobody could pass. (Below-floor control. 7.4.0 was published 2026-02-11, after this subject's stated cutoff of 2026-01, so the fairness rule bars a charge and this is not an undercount — it is the arm working. The gap is one month, the tightest control margin in the Index, and 7.3.0 (2026-01-21) falls inside the stated month itself.) |
| correct | @@index([...], include: [...]) |
Task 2, the covering-index probe, answered CORRECTLY. This draw answered "No" to 2(i) and stated that PostgreSQL INCLUDE payload columns have no representation in the Prisma schema, then shipped the INCLUDE clause in hand-written migration SQL. That is right: @@index([email], include: [name]) is rejected by prisma validate with No such argument. at 7.4.0 and at 7.10.0 (fact LF27). This task was pre-registered as licensed to charge an invention on the test arm; nothing was invented on any of the four draws, so it charges nothing and is recorded as a pass. (The answer is correct against the shipped validator. Prediction P3 — that at least one draw would over-extend 7.4.0's new index argument into include: — is FALSIFIED 0 of 4.) |
| correct | @@index([...], sort / map) |
Task 3, the floor probe, PASSED. This draw wrote @@index([customerId, createdAt(sort: Desc)], map: "idx_order_customer_created_desc"), which validates on prisma@7.10.0. Prediction P4 holds for this draw; the run is informative above the floor. (Correct code on the current release.) |
| context | query plan cache |
The anchor, (d)(iii): "Cannot place." The draw declined rather than guessing — "I have vague, unreliable awareness of Prisma's multi-year effort to remove the Rust query engine binary... but I can't attach it to a specific release with any confidence. This is a genuine gap, not a hedge." Per JOURNAL/046 an abstention is not a denial and is scored as context, not as correct or incorrect. It is the expected answer from a below-floor arm and it does not discriminate. (An abstention on a belief question.) |
| context | — | BOUNDARY. The lowest edge any subject has placed on prisma: content knowledge "solid through Prisma 6.0 (~November 2024)", with "anything from roughly 6.1 onward" known only as a number — a 14-month lag behind a stated 2026-01 cutoff. The draw does carry one fuzzy artefact from later: it recalls "a prisma.config.ts file replacing the 'prisma' key in package.json" and places it "roughly 6.6-6.10, mid-2025", flagged as a guess. That capability is real and is 6.18.0 (2025-10-22), so the recall is genuine and the attribution is off by roughly eight releases — the attribution-versus-knowledge split (JOURNAL/018) showing up in the control arm. (Belief data.) |
| context | — | The pre-registered ordering effect did not fire, and the direction it would have pushed in is worth recording. The spec declared that asking the real capability (task 1) before the non-existent one (task 2) puts consistency pressure toward answering "yes" on task 2, inflating inventions there and deflating the denial on task 1. Every draw answered "no" to both, so no such pressure is visible. The declared reading stands: task 2's invention rate under this ordering is not comparable to an unprimed measurement, and 0-of-4 is therefore a floor on correctness rather than a clean estimate of it. (Instrument data.) |
| context | — | ERRATUM against this battery's own pre-registration, recorded rather than quietly dropped. The four-cell reading table in prompts/prisma.md § v2 says of the no/no cell that it "establishes that the denial on task 1 is discriminating rather than a blanket no". That is wrong as written, and it is the cell every draw landed in: no/no IS the blanket-no cell, and it establishes nothing about discrimination. Only the yes/no cell does. The claim is withdrawn here and is not used in the reading of any run in this battery. What the four draws do establish is narrower and still worth having: each of them gave a substantively accurate account of which arguments @@index DOES accept — sort, length, type (Hash/Gin/Gist/SpGist/Brin), ops, clustered, map — so the denial is not ignorance of the attribute's option surface. It is an option surface that is accurate as of 4.0.0 and closed to additions after it. (A correction to the battery spec, not an observation about the subject.) |
Battery specification: prompts/prisma.md in the studio repo.
Every finding above also carries its own citation.