Run prisma--claude-fable-5--v2-d--2026-09-05
Second below-floor control for prisma/v2. It failed task 1 like its sibling control, completing P2 two of two and establishing that where: on @@index is not derivable from the schema language around it. It passed task 2 and the floor probe. Its boundary sits at 6.7.0, eight months behind its stated 2026-01 cutoff, and it produced the battery's clearest statement of the attribution/knowledge split: it knows Prisma 7 was coming and what it was for, and cannot say whether it shipped.
| Subject | Claude Fable 5 claude-fable-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model override 'fable', no tools available to the subject |
| Cutoff the model states | 2026-01 |
| Newest prisma release it could place | 6.7.0 · 2025-04-29 (~8 month lag) |
| Oldest prisma release it could not place | 6.8.0 · 2025-05-15 (so this run brackets the subject’s boundary to 2025-04-29 – 2025-05-15) |
| In its own words | "The latest Prisma version I have any awareness of is somewhere around 6.16-6.17, roughly September-October 2025, and I have vague awareness that a Prisma 7 (making the Rust-free/TypeScript query compiler client the default) was planned for late 2025 — but I can't tell you whether it shipped." |
| Library at test time | prisma 7.10.0 (npm), verified 2026-09-05 |
| Battery | prisma/v2-d · 3 tasks, 4 direct questions · probe window 7.4.0 to 7.10.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-05 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | @@index([...], where: ...) / @@unique([...], where: ...) |
THE SECOND CONTROL RESULT, completing prediction P2. Task 1(i): "No." In prose: "Prisma's schema language has no where clause on @@index or @@unique (it's a long-standing open feature request), so the filter lives in a hand-edited migration." (d)(i): "does not exist in Prisma at all, as of my knowledge. Open feature request for years; raw SQL is the sanctioned workaround." Two of two below-floor controls fail the probe, so it is not derivable. (Below-floor control; 7.4.0 postdates this subject's stated cutoff of 2026-01. Not an undercount.) |
| correct | @@index([...], include: [...]) |
Task 2, the covering-index probe, answered CORRECTLY. This draw answered "No" to 2(i) and stated that PostgreSQL INCLUDE payload columns have no representation in the Prisma schema, then shipped the INCLUDE clause in hand-written migration SQL. That is right: @@index([email], include: [name]) is rejected by prisma validate with No such argument. at 7.4.0 and at 7.10.0 (fact LF27). This task was pre-registered as licensed to charge an invention on the test arm; nothing was invented on any of the four draws, so it charges nothing and is recorded as a pass. (The answer is correct against the shipped validator. Prediction P3 — that at least one draw would over-extend 7.4.0's new index argument into include: — is FALSIFIED 0 of 4.) |
| correct | @@index([...], sort / map) |
Task 3, the floor probe, PASSED. This draw wrote @@index([customerId, createdAt(sort: Desc)], map: "order_customer_created_desc_idx"), which validates on prisma@7.10.0. Prediction P4 holds for this draw; the run is informative above the floor. (Correct code on the current release.) |
| miss | query plan cache |
The anchor, (d)(iii), answered the same way both Opus draws answered it: "my best estimate is that this is part of the queryCompiler work (preview around 6.7, May 2025), which I believe caches compiled plans for repeated query shapes — but that specific caching claim is an estimate". Three of four draws in this battery, across three different models, substituted the query-compiler work for the caching layer built on top of it nine months later. That consistency is the note: it is not one subject's slip. (Direct questions are belief data. Also below-floor: 7.4.0 postdates this subject's stated cutoff.) |
| context | — | BOUNDARY. Describable content stops at 6.6-6.7 (April-May 2025); "first release I know essentially only as a version number: around 6.8-6.9 (mid-2025)"; awareness of version NUMBERS runs on to 6.16-6.17 and to the existence of a planned Prisma 7 whose shipping it cannot confirm. That last clause is the cleanest statement of the attribution/knowledge split any control arm has produced: the subject knows a major was coming, knows what it was for, and does not know whether it happened. (Belief data.) |
| correct | — | One incidental claim in task 1 that the other three draws hedged on and this one got right in the safe direction: "Since migrate dev works off the migration history (shadow DB), it won't try to drop it." The other three draws each flagged drift as an unresolved risk they would test before shipping. Not scored either way here — the Index has not executed that scenario and says so in F1's scope note — but recorded because the four draws split on it and a later battery could settle it cheaply. (Unverified by the Index in either direction, and outside the tasks licensed to charge.) |
| context | — | The pre-registered ordering effect did not fire, and the direction it would have pushed in is worth recording. The spec declared that asking the real capability (task 1) before the non-existent one (task 2) puts consistency pressure toward answering "yes" on task 2, inflating inventions there and deflating the denial on task 1. Every draw answered "no" to both, so no such pressure is visible. The declared reading stands: task 2's invention rate under this ordering is not comparable to an unprimed measurement, and 0-of-4 is therefore a floor on correctness rather than a clean estimate of it. (Instrument data.) |
| context | — | ERRATUM against this battery's own pre-registration, recorded rather than quietly dropped. The four-cell reading table in prompts/prisma.md § v2 says of the no/no cell that it "establishes that the denial on task 1 is discriminating rather than a blanket no". That is wrong as written, and it is the cell every draw landed in: no/no IS the blanket-no cell, and it establishes nothing about discrimination. Only the yes/no cell does. The claim is withdrawn here and is not used in the reading of any run in this battery. What the four draws do establish is narrower and still worth having: each of them gave a substantively accurate account of which arguments @@index DOES accept — sort, length, type (Hash/Gin/Gist/SpGist/Brin), ops, clustered, map — so the denial is not ignorance of the attribute's option surface. It is an option surface that is accurate as of 4.0.0 and closed to additions after it. (A correction to the battery spec, not an observation about the subject.) |
Battery specification: prompts/prisma.md in the studio repo.
Every finding above also carries its own citation.