117 — The two artifacts that were one measurement

2026-09-10 · distribution lane · DISTRIBUTION D1, BM2 build-order step 1

BM2's second acceptance assertion was authored, and the gate refused the task. The refusal is correct, the fact behind it is a published S1, and the gate could not say why until this session added an eighth amendment to §9 — because the defect is a property of the pair of artifacts, and every refusal §9 had was about one artifact at a time.

The task

A2's seat was drawn for LF14: "The client engines are gone: LibraryEngine (engineType = "library"), BinaryEngine (engineType = "binary"), DataProxyEngine and ReactNativeEngine, and with them the PRISMA_CLIENT_ENGINE_TYPE, PRISMA_QUERY_ENGINE_BINARY, PRISMA_QUERY_ENGINE_LIBRARY and PRISMA_CLI_QUERY_ENGINE_TYPE environment variables. Prisma 7's query compiler is a WebAssembly module inside the client." S1, dated 7.0.0, bisected across twenty-three rungs, and charged against this subject by a published battery.

The prompt is the ordinary version of that: a distroless image built with no network access, nothing downloaded at generation time, no Rust query engine binary in the image. Graded by generation — prisma generate must succeed and the tree it writes must contain no file named for a query engine. The correct artifact is a schema that says nothing about engines, because on prisma 7 there is nothing to configure. The stale artifact is the same schema with engineType = "library", which is the prisma 6 answer to "do not ship a standalone engine executable": it selects the Node-API engine, a libquery_engine-*.node loaded in-process instead of a query-engine-* binary spawned beside it — and on 6.x that is the wrong answer to the question asked, because both are Rust query engines and both are written into the generated output.

The two artifacts differ by one line. The generator provider is held at prisma-client-js on both, deliberately: the prisma-client provider LF6 prescribes does not exist below 6.6.0, and an artifact using it would carry a second version dependence at a rung that is not this fact's.

The measurement

REFUSE A2   A  LF14  rising    @7.0.0
       correct  ............+++++++++++
       stale    ............+++++++++++

The rows are identical. Both artifacts fail on all twelve 6.x rungs — the engine is there, which is what the prompt asked to avoid — and both pass on all eleven 7.x rungs. The cause was already written down in the fact's own probe and had never been read as a fact about benchmarkability: prisma 7 ignores engineType rather than rejecting it. Checked directly at 7.0.0 before any artifact was written — a schema carrying engineType = "binary" generates successfully and writes query_compiler_bg.wasm, query_compiler_bg.js and no engine, which is byte for byte what a schema carrying nothing writes.

So LF14 is a published S1 fact whose staleness has no executable consequence at the graded release. A subject that still believes in selectable client engines and a subject that does not both pass this task on prisma 7.10.0. That is a fact about the library, it is worth having, and it means the honest task set cannot contain LF14.

What the gate said, and what it would have admitted

A2 sits at a major floor, so the seventh amendment's second guard fired first and refused it for the dialect reason: a rising task cannot tell "the API did not exist below" from "the artifact is written in this major's dialect". True of this task, and not its defect.

Away from a major floor there was no guard at all. Run against the committed HEAD rather than reasoned about, the previous gate admitted an identical pair in both shapes — rising and falling — each with a boundary at the fact's own release, and refused only a pair that passes everywhere, by the neither artifact flips sentence, which is about the correct solution not needing the fact and not about the pair. An admitted task of that shape goes to the arms, and its BARE cells measure nothing: a subject that knows the fact and a subject that does not produce the same verdict, and the number that comes out is indistinguishable from a real one.

§9's eighth amendment

A task whose two artifacts have the same row is refused. A Class A task exists to separate a subject that holds the stale belief from one that does not, and that separation is the difference between the rows — which no per-artifact check can see. profileOf now carries the verdicts it read, for this one purpose; nothing serialises them, because the flip_test block is assembled field by field, so they cannot drift into a results document. The refusal prints the shared row so the reason is visible rather than asserted.

Selftest 39 → 42 checks, one per case measured against HEAD: identical rising, identical falling, identical flat-passing. BM1 does not move — re-gated on its own fifteen-rung ladder under the amendment, all eighteen tasks admitted and every flip_test block byte-identical to the one published in data/benchmark/zod--claude-opus-5--bm1--2026-09-07.json.

Two smaller things this needed

A generate instrument the artifact can steer. subject.generateOf memoises on a tag the caller supplies and walks go-<tag>, so the caller must know where the generator will write — fine for a probe whose schema the probe file wrote, wrong for a benchmark artifact that writes its own generator block, because fixing the output path in the prompt puts a harness detail into a task. subject.generateProject gives the schema a project directory of its own and walks the whole of it, so whatever relative path the generator names is found. It is memoised on the schema text and not on a tag, which matters more here than anywhere: the two artifacts of one task differ by one line, and a tag they shared would have handed the second one the first one's generated tree — which would have produced identical rows for a reason that was the harness rather than the library.

Two open seats and one reserve. A2 is refused and A7 (LF21) is already recorded as due for refusal. Consuming the reserve in the order the refusals were discovered would let the sequence an operator happened to author things in decide which fact enters the set — the property §7 exists to remove. So the rule, fixed in prompts/benchmark-bm2-draw.md while both seats were open and before either substitute was authored, is task order: A2 takes S1 #10 (LF16), A7 takes S1 #11 (LF1).

State

BM2: A1 admitted, A2 refused and its seat reassigned, ten Class A assertions left plus the two substitutes, then the six Class N tasks, BM2's predictions, then the arms. The refused task keeps its prompt and both artifacts — A2.lf14.reference.mjs, A2.lf14.stale.mjs, prompts/sent/benchmark-prisma-a2-lf14-refused.txt, A2_LF14 in tools/benchmark/tasks/prisma.mjs — so the refusal re-runs, which is the convention BM1's A2/LF4 set.