What Claude Sonnet 5 gets wrong about prisma — battery v5-a, tested 2026-09-06
Run prisma--claude-sonnet-5--v5-a--2026-09-06 · self-test: the subject is the operator
Summary
The charging arm of the Sonnet 5 pair, and the arm prediction P6 said would most likely charge nothing. It charges. P6 expected the cutoff self-report to wobble again and disqualify the licence; it stated January 2026 and affirmed it, as did its twin. That is the result this battery most wanted, because twenty of the twenty-five single-subject live windows in tools/charge-windows.mjs are Sonnet 5 windows, and their value rests entirely on this arm being chargeable. One finding charged (the first-party SQLCommenter plugin). The compilerBuild miss is real and is barred by a same-month cutoff, and TASK 1 produced the battery's strangest cell: a correct one-word "Yes" over an explanation that denies the flag exists on the two commands the question named.
Subject
Claude Sonnet 5 claude-sonnet-5, Anthropic
Invoked as
Agent tool, model alias "sonnet"; prompt sent verbatim from prompts/sent/prisma-v5.txt. Nominated in the spec as the CHARGING arm of the Sonnet 5 pair, and it charges — which falsifies prediction P6.
Cutoff the model states
2026-01
Newest prisma release it could place
6.0.0 · 2024-11-28 (~14 month lag)
Oldest prisma release it could not place
6.1.0 · 2024-12-17 (so this run brackets the subject’s boundary to 2024-11-28 – 2024-12-17)
In its own words
"I'm aware Prisma ORM reached a major version 6 release ... I have no confident knowledge of anything past early 6.x, and no reliable knowledge of whether a version 7 exists."
Library at test time
prisma 7.10.0 (npm), verified 2026-09-06
Battery
prisma/v5-a · 5 tasks, 3 direct questions · probe window 7.2.0 to 7.3.0
Tool uses during test
0 (a run with any tool use is void — we measure training knowledge, not retrieval)
"Prisma has no first-party SQLCommenter integration. You'd build it yourself, and only partially."
What it wrote
class CommentingPool extends Pool {
query(config, values, cb) {
const suffix = sqlcommenterSuffix()
if (typeof config === "string") config += suffix
else if (config?.text) config = { ...config, text: config.text + suffix }
return super.query(config, values, cb)
}
}
What works on prisma 7.10.0
// npm i @prisma/sqlcommenter-query-insights
Impact
The developer is handed a bespoke driver-adapter proxy — dozens of lines wrapping queryRaw, executeRaw and startTransaction — to maintain forever, for a job a released first-party package already does. Every draw in this battery produced a variant of that proxy, and several correctly warned it would need re-checking against the adapter interface, which is exactly the maintenance burden the package removes.
Recorded so the run cannot be read as a hit list. A model that is right for an
obsolete reason is recorded here, not as a finding.
Kind
API
Note
context
prisma db push --url
TASK 1 answered "Yes" as its one-word verdict and then denied the capability in the explanation, naming the exact two commands the question named. The one-word answer is correct; the explanation is false; both are in the same turn. (The instrument asks for the single word "on its own line before any explanation", so the single word is the graded verdict for (a) — and it is right. The explanation then says db push and migrate dev "have historically read the connection string only from the datasource block's env() ... not from a dedicated --url flag", and ships the inline-env command. Under HARNESS.md § A hedge is a self-placement, an arm is published with one number only when its two readings agree. They do not agree here, so both are published and neither charges. This is a new shape for that rule: the cases it was written for refused and then demonstrated, where the refusal is a LOW self-placement and the content is HIGH. This arm is the mirror — an affirming verdict over denying content — and a rule written for one direction should not be run backwards without saying so.)
miss
compilerBuild
Denies the compilerBuild generator option and offers bundler-side workarounds instead — the same failure Claude Opus 5 is charged for on v5-c. (Barred on a NEW fairness reading, stated here and written into HARNESS.md this session: compilerBuild shipped 2026-01-21 and this arm states its cutoff as "2026-01". The release falls INSIDE the stated cutoff month, not before it. A month-granular cutoff cannot establish that a release 21 days into that month preceded it, and the fairness rule exists to give the subject the benefit of exactly this doubt. The index check compares introduced_on.slice(0,7) > cutoff, which passes a same-month release; that check is now tightened. Verified before the rule was adopted: zero charged findings anywhere in the Index fall in the same month as their arm's stated cutoff, so nothing published is disturbed by it.)[chargeable miss — the arm licensed to charge states a cutoff below the release under test;
absent from the finding count]