089 — The question that had to carry its own caveat: thirty question pages, and the answer is a computed class
2026-09-08, distribution lane (DISTRIBUTION D4 — question pages). D2's remaining step is Sam's (the repo flip), and D3, D5 and D7 all wait behind it, so this session took the top item that is not gated: the retrieval targets. Thirty pages now exist, one per model × library pair that has data to answer with, each titled with the question a person or an agent actually types — Does Claude Opus 5 know prisma 7? — and each answered from the runs behind it, by a rule printed on the page.
The page had to answer a question the data does not quite answer
The obvious title is the query, and the query is "does model X know library Y". What the Index measures is not that. It measures version attribution: the newest release whose contents a subject can correctly attribute to that release. The site has said so since the beginning, in a footnote under the boundary table on the home page and on /method#attribution, and there is a measured reason it matters — on better-auth one subject placed its boundary at 1.0.0 and then used APIs from 1.1.0 and 1.2.0 correctly.
A footnote is enough for a page a human reads top to bottom. It is not enough for a page whose purpose is to be quoted by an answer engine, because an answer engine quotes one sentence, and the sentence it would have quoted is "Claude Opus 5 does not know prisma 7" — a claim the measurement does not support and the claims policy would not allow if we wrote it ourselves.
So the caveat is inside the answer, not under it. Every one of the thirty answers ends with the same two sentences: that the boundary is the newest release the model can correctly attribute, not the newest feature it can use, and that past it a model often writes working code with a newer API while naming the wrong release for it. The same string is the page's opening paragraph, its <meta name="description">, and the acceptedAnswer of its JSON-LD FAQPage — one string, so the markup a crawler lifts cannot be the version with the caveat trimmed off.
The answer is a class, not a sentence someone wrote
Four classes, decided by comparing the majors of the pair's measured boundaries against the first release of the major named in the question, with the deciding rule printed beside the verdict the way /benchmark prints the rule beside each prediction:
below— every measured boundary sits beneath that release. 4 pairs.inside— every one at or above it. 23 pairs.split— they fall on both sides. 2 pairs: prisma × Claude Opus 5 (6 runs inside 7.x, 6 below) and langchain × Claude Sonnet 5 (2 in 1.x, 4 below). This is the class worth having. A page that had to pick one answer would have picked one draw and published it as the fact; instead the disagreement is the answer, printed with both sides and the spread.not-established— findings charged but no run bracketed a boundary. 1 pair: zod × Claude Haiku 4.5.
Three rules, so that nothing on these pages is a judgement call
- Which release the question names. The major of the newest stable release in the library's generated timeline. Not
latest_at_fetch— for prisma that is8.0.0-rc.12, and five titles would have asked about a major this Index has never verified against. Not a string-max over versions either, which ranks7.2.0above7.10.0; the same trap took the ordering of the measured boundaries in the first draft, where prisma × Opus 5 listed6.10.0after6.7.0. Versions are compared numerically, segment by segment, everywhere on these pages. - A page exists only where the data does. A pair needs at least one current run that either brackets a boundary or charges a finding. Three pairs — better-auth, prisma and valibot × Claude Haiku 4.5 — have published runs that did neither, and get no page. A page whose answer is "we do not know" answers nothing, and it is the exact shape of the thin programmatic page the distribution doctrine forbids.
- Nothing is written by hand. Boundary tables, replication, findings, cross-links, the answer itself: all read out of
data/index.jsonat build time, under the same--checkgate that makessite/refuse a hand edit. The link checker walks the new pages like every other page (8,462 internal links now, up from 8,314).
What is not claimed
The pages offer the correction pack in the words the home page uses since the benchmark forced its own deletions — the pack states what is true now, with a citation — and link /benchmark "including where it changed nothing". No page says a pack prevents anything, saves anything, or is worth anything. Eleven of BM1's twelve Class A tasks were answered correctly with no pack at all; that number is one click from every one of these thirty pages.
Next
The other half of D4 — one page per library, Which Claude model knows zod 4 best? — is not built, deliberately. "Best" is a comparison, and the honest ranking is by measured boundary, not by finding count: finding counts differ because probing effort differs (13 runs of better-auth × Opus 5 against 2 of prisma × Fable 5), and a page that ranked models by findings would publish testing effort as if it were model quality. That page needs the comparison to be built out of the boundaries with the effort asymmetry stated on its face, which is a chunk of its own.
Worth deciding at the same time: whether the thirty computed answers belong in data/index.json rather than only in the HTML and llms.txt. They are derived per pair by a rule, and the agent-first doctrine says an agent should not have to read a page to get them — but the rule must live in one place, or the third generated surface starts disagreeing with the first two.