Run valibot--claude-opus-5--v3-d--2026-09-05 · self-test: the subject is the operator
Blind twin, charges nothing. Agrees with v3-c on all five tasks and on the boundary. Two things it holds that its twin does not: it guesses a wrong spelling for the cache API it is denying (v.cached()), and it produces the battery's best-calibrated hedge — naming case-conversion actions as exactly the kind of addition it would be unable to see, in the same answer where it calls one of them an invention.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus"; prompt sent verbatim from prompts/sent/valibot-v3.txt, byte-identical to the file sent to `v3-c` and drawn blind and concurrently with it. Blind twin of `v3-c`, born duplicated; charges nothing. |
| Cutoff the model states | 2026-05 |
| Newest valibot release it could place | 1.1.0 · 2025-05-06 (~12 month lag) |
| Oldest valibot release it could not place | 1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24) |
| In its own words | "The most recent release whose contents I can actually describe is v1.1.0 (~April 2025) ... I have a vague sense that the 1.x line continued past that (a 1.2.x, plausibly), but I cannot tell you what changed in it, so I won't pretend to." |
| Library at test time | valibot 1.4.2 (npm), verified 2026-09-05 |
| Battery | valibot/v3-d · 5 tasks, 3 direct questions · probe window 1.3.0 to 1.4.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-05 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | guard |
Task 1, the guard probe. "no", with the same correct subsidiary analysis as its twin: "v.check() is typed CheckAction<TInput, ...> — output type equals input type, so a v.pipe(v.unknown(), v.check(isPluginConfig)) still gives you unknown." Verified: TS18046. Its v.custom<PluginConfig> substitute compiles and narrows. (Blind twin of v3-c; a -b-position draw never charges. Charged on the twin as F1.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on valibot--claude-opus-5--v3-c--2026-09-05] |
| miss | cache |
Task 4, the cache probe. "no": "Valibot has no memoization layer — no v.cached(), no cache option on parse." Both halves of that are wrong at 1.3.0+, though the spelling it guesses at (v.cached()) is not the real one (v.cache()). The LRU-bounded memo parser it wrote is correct. (Blind twin; charged on v3-c as F2.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
charged as a finding on valibot--claude-opus-5--v3-c--2026-09-05] |
| miss | toCamelCase / toKebabCase / toPascalCase / toSnakeCase |
Task 2, the case-conversion probe, offer direction. "no" — "nothing for kebab/camel/snake/title conventions" — with the hand-rolled slugify. Three of those four conventions shipped at 1.4.0. (Barred twice over: blind twin, and 1.4.0 is the same month as this subject's stated cutoff.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test; absent from the finding count] |
| miss | toKebabCase |
Task 3, the recognition direction. "It does not compile. Both actions are inventions." toKebabCase is real at 1.4.0+. But this arm produced the best-calibrated hedge in the battery and it is worth quoting: "I'm confident about these two specifically; if some very recent release added case-convention actions I'd be unaware of it", and in its direct answers, "case-conversion actions are exactly the kind of small, popular addition that could have landed in a release I can't see." It named the failure mode it was in, correctly, while committing to the wrong verdict. (Barred twice over, as above. HARNESS.md § A hedge is a self-placement, not a grade on the content: the hedge does not soften the verdict the pull request received, and the artefact is still a rejected-correct PR.) [chargeable miss — the arm licensed to charge states a cutoff below the release under test;
absent from the finding count] |
| correct | parseJson / stringifyJson |
Task 5, the attribution anchor. parseJson/stringifyJson at 1.1.0, "around April 2025" — correct to the minor, and it added the correct negative that they were not in 1.0.0. Read for attribution. |
| correct | toTitleCase |
The poison rung. Refused toTitleCase. P4 holds on this arm. |
| context | — | The boundary: 1.1.0 describable, 1.2.x the first release known only as a number — identical to its twin and to both v2 Opus draws. Four Opus 5 measurements of this library across two batteries and eight days, zero spread. |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.