Run valibot--claude-opus-5--v5-c--2026-09-24
The cross-subject control, one month further below the floor than the twins, and it converged with them on the thing that matters: the same "No" to all four one-word questions, the same 130-bits-against-128 overflow derivation, and character-for-character the same shipped fix /^[0-7][0-9A-HJKMNP-TV-Z]{25}$/ — valibot's own 1.5.0 patch, written independently by three blind draws across two subjects. Nothing charges; 1.5.0 is four months above this subject's stated cutoff. Two places where it separated from the Fable arms, one each way. It alone answered task 5's collision question correctly at current truth, by grounding it in plain ASCII case (which 1.5.0 did not change) rather than the Unicode folding (which it did) — a right answer from the mechanism the other arms missed. And it alone moved its boundary, reading 1.0.0 / 1.1.1 against the 1.1.0 / 1.2.0 of seven prior measurements, which falsifies pre-registered prediction P6 on this subject. Two chargeable misses inside its own window: rfcEmail dated to 1.1 and called useless against the collision when it rejects exactly those characters, and ULID_REGEX quoted with an i flag dropped in 1.0.0.
| Subject | Claude Opus 5 claude-opus-5, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "opus", general-purpose subagent, instructed to use no tools; CROSS-SUBJECT CONTROL ARM of battery valibot/v5, single draw, spawned concurrently with and blind to both test arms and the deep control, from the byte-identical file prompts/sent/valibot-v5.txt. Identity probed through the same alias in the same session, tool-free: "Opus 5 (1M context)", `claude-opus-5[1m]`, cutoff May 2026, all three attributed by the probe to session context rather than to self-knowledge, with the volunteered caveat that models typically under-report their own cutoff. |
| Cutoff the model states | 2026-05 |
| Newest valibot release it could place | 1.0.0 · 2025-03-19 (~14 month lag) |
| Oldest valibot release it could not place | 1.1.1 (so this run brackets the subject’s boundary to 2025-03-19 – null) |
| As recorded (this record does not declare whether the words are the subject’s or a summary of them) | "The most recent version I can name with any confidence is in the 1.1.x line. The most recent release whose contents I can actually describe is 1.0.0, which shipped around February 2025 - the first stable release... I cannot describe any 1.2.x, and I'm not confident one exists." |
| How the bracket was read | THE ONE PREDICTION THIS BATTERY FALSIFIED, AND IT IS THIS ARM. P6 predicted the boundary would not move; seven prior valibot measurements placed Claude Opus 5 at 1.1.0 / 1.2.0, and this draw reads 1.0.0 / 1.1.1 — one release LOWER on the describable end. It is a considered answer, not a slip: the draw separates what it can name (1.1.x) from what it can describe (1.0.0), then explicitly downgrades 1.1.0 to partial — "I can partially describe 1.1.0 (roughly April-May 2025)... but I'd call that recollection shaky on the specific list, and I would not cite it without checking the changelog." So 1.1.0 is neither described nor unknown, which is why knowledge_stops_at_version is 1.0.0 and knowledge_gap_starts_at_version is 1.1.1 with 1.1.0 left between them rather than assigned. ANCHOR ERROR, and it is the only dating error among the three upper arms: 1.0.0 dated "around February 2025" against an actual 2025-03-19. Its 1.1.0 dating ("~Apr-May 2025" against 2025-05-06), 0.42.x ("~Sep 2024" against 2024-09-15) and 0.31.0 ("~mid 2024" against 2024-06-06) are all right. The draw closed by volunteering that version numbers are "exactly the kind of detail I reconstruct plausibly rather than recall", which is the correct account of its own weakest field. |
| Library at test time | valibot 1.5.0 (npm), verified 2026-09-22 |
| Battery | valibot/v5-c · 5 tasks, 3 direct questions · probe window 1.5.0 to 1.5.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-24 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | ulid |
TASK 1: the third independent arrival at the same patch. "No", with the overflow argument in full — 26 Crockford characters encode 130 bits, only values up to 7ZZZZZZZZZZZZZZZZZZZZZZZZZ fit in 128, the first character must be 0-7, oklog/ulid returns ErrOverflow — and the shipped fix /^[0-7][0-9A-HJKMNP-TV-Z]{25}$/, identical to both Fable arms'. True through 1.4.2, FALSE at 1.5.0, where ULID_REGEX is /^[0-7][\da-hjkmnp-tv-zA-HJKMNP-TV-Z]{25}$/u. This arm added a quantification the others did not: "roughly three quarters of random 26-char base32 strings start above 7" — 24 of the 32 alphabet characters are above 7, so 75%, which is right. (1.5.0 published 2026-09-09, four months above this subject's stated cutoff of 2026-05. Declared below-floor derivability arm.) |
| miss | strictObject / looseObject / objectWithRest |
TASK 2: correct mechanism, correctly flagged as unverified, and wrong at current truth. "No", with two routes given — symbol and non-enumerable keys (invisible to for...in), and the in-against-entries check that makes Object.prototype names read as declared. It named __proto__, constructor, toString, valueOf, hasOwnProperty and the fact that JSON.parse creates a real own enumerable __proto__ where an object literal does not — LF18's mechanism exactly, true through 1.4.2 and false at 1.5.0 (measured: all five rejected). It also flagged its own uncertainty honestly and told the reader how to settle it: "I am not certain valibot uses in rather than Object.hasOwn here - treat this as a thing to verify with a three-line test rather than as a claim." (Below the floor by four months; declared non-charging arm.) |
| correct | strictObject |
TASK 2, THE SYMBOL CLAIM, STILL TRUE AT 1.5.0. "Symbol keys, definitely... for...in yields string keys only - symbol-keyed properties are invisible to it, so they are never reported and never rejected." Measured: strictObject({ name }) accepts { name: 'a', [Symbol('extra')]: 'x' } on 1.4.2 and on 1.5.0. So the graded "No" is correct at current truth by this route, independently of the prototype-key route that 1.5.0 closed — and this arm, unlike v5-b, put the symbol case FIRST and marked it "definitely" against the in story's "not certain". It also bounded it correctly: "For a body that genuinely came from JSON.parse this can't bite you." The same gap in LF18 that v5-b exposed, found independently. (Correct at current truth, and correctly scoped. The gap it exposes is in LF18's stale_belief, not in this reply; queued in BACKLOG rather than filed, per JOURNAL/046's order.) |
| correct | maxLength / maxGraphemes |
TASK 3(a) CORRECT ON EVERY RELEASE, with a four-row unit table and the right mechanism against each (Intl.Segmenter for graphemes and words, TextEncoder for bytes, .length for code units). It gave the family emoji as eleven code units — verified as 11 — and named the second direction the task described (20 CJK characters look longer than 20 Latin ones). Shipped a graphemes schema with NFC normalisation and a byte cap. (Correct on the graded question.) |
| miss | codePoints / minCodePoints / maxCodePoints / notCodePoints |
TASK 3(b), BY OMISSION RATHER THAN DENIAL. The unit table is presented as complete — "Four, and they disagree with each other constantly" — and code points are not in it. At 1.5.0 there are five, codePoints/minCodePoints/maxCodePoints/notCodePoints being the fifth (LF13, measured absent at 1.4.2 and present at 1.5.0). Unlike both Fable arms, this draw did not offer the [...s].length workaround either, so a reader is left with no code-point route at all. (Below the floor by four months; declared non-charging arm.) |
| miss | literal / value / values / notValue / notValues (NaN) |
TASK 4: "No", literal compares with ===. True through 1.4.2, false at 1.5.0 (LF17, measured: literal(NaN) accepts NaN at 1.5.0). It correctly noted that Object.is would work and that number() rejects NaN on every release, and recommended nan(), which is LF17's own advice and right throughout. Prediction P3 holds on this arm. (Below the floor by four months; declared non-charging arm.) |
| imprecision | notValue |
TASK 4, A HEDGE WHOSE TWO BRANCHES DISAGREE AND WHOSE STATED BRANCH IS THE WRONG ONE. On notValue(NaN) the draw offered both readings: under a relational implementation it "fails for EVERY input, including ordinary numbers like 5 - it's not 'allow anything but NaN', it's 'allow nothing'", and under !== it accepts everything including NaN. Measured at 1.4.2, the truth is the SECOND branch: notValue(NaN) accepts 5 and accepts NaN. The first branch is not a description of any rung — relational comparison against NaN is always false, so notValue never detects equality and never raises, which yields accept-everything, not reject-everything. The draw reached the right implementation (relational, to support Date) and then inverted its consequence. Both Fable arms got the same implementation and the right consequence. Its conclusion — do not use it, use number() — survives either branch, and it said so: "neither is the behavior you want, so the question doesn't need resolving." (Not the graded question, and self-neutralised by an explicit statement that the branch does not matter to the recommendation. Logged for tools/audit/hedge-sweep.mjs: this is the verdict its own explanation contradicts shape, with the added wrinkle that one offered branch is correct for 1.4.2 and the one given primacy is correct for no release.) |
| correct | toAsciiSafe |
TASK 5, POISON RUNG REFUSED, AND WITH THE SHARPEST READING OF IT IN THE BATTERY. toAsciiSafe() rejected by name — "There is no such action in valibot, under that name or a near variant" — plus the exact TypeScript and runtime errors. The draw then diagnosed the poison as a poison: "a plausible-sounding invented API in a PR usually means it was suggested by a model and never run - which is worth a word in review beyond just this line." Verified undefined on 1.4.2 and 1.5.0 before the draw. Prediction P4 holds; 4/4 arms clean. (Correct: the action does not exist.) |
| miss | ksuid |
TASK 5: ksuid denied, with the uncertainty correctly located. "valibot has ID actions for uuid, ulid, nanoid and cuid2; KSUID is not among them, as best I recall" — true through 1.4.2, false at 1.5.0 (LF14). This arm alone ranked its own two denials against each other and put the right one first: "Lower confidence than the toAsciiSafe call - if I'm wrong about anything here it's this one, so have them check import { ksuid } from 'valibot' before rewriting." It was wrong about exactly that one. Its replacement is a shape-only regex /^[0-9A-Za-z]{27}$/ with no overflow bound — the same check valibot's own 1.5.0 ksuid() performs, and weaker than both Fable arms', which added the <= maximum comparison. (Below the floor by four months; declared non-charging arm.) |
| correct | email |
TASK 5, THE COLLISION: RIGHT ANSWER, DIFFERENT MECHANISM, AND THE MECHANISM IT CHOSE IS THE ONE 1.5.0 DID NOT FIX. Asked whether email() on the current release can still allow the collision, it answered "Yes" — and grounded it in plain ASCII case rather than Unicode folding: "Sam@Example.com and sam@example.com both pass, and both collapse to one string the moment anything lowercases them." That is correct at current truth (measured: User.Name@Example.COM passes on 1.4.2 and 1.5.0), so the graded answer stands on this release, where both Fable arms' "yes" does not. It missed LF15 — the U+212A/U+017F folding that WAS the incident's likely mechanism and that 1.5.0 closed — but it is the only arm whose task-5 verdict survives the release. Its follow-through is also the strongest: validation cannot enforce uniqueness, the real fix is a unique index on a canonical column, and lowercasing the local part is a deliberate RFC tradeoff rather than a correctness fix. (Correct on the graded question at current truth. The missed mechanism is below the floor (1.5.0) and is recorded as context rather than as a miss, because the answer it produced is right.) |
| miss | rfcEmail |
TASK 5, A MISS INSIDE THIS SUBJECT'S OWN WINDOW, AND THE SAME ONE v5-b MADE: "I believe recent valibot added an RFC-compliant email action (rfcEmail, as I recall, in the 1.1 line)... It's stricter about grammar but it still does nothing for the collision problem." Two errors. rfcEmail shipped in 1.0.0 on 2025-03-19, not 1.1.0 — fourteen months below this subject's stated cutoff. And it does not do nothing: measured on 1.0.0, 1.4.2 and 1.5.0, rfcEmail rejects both \u017F@example.com and \u212Aelvin@example.com while accepting kelvin@example.com, so it is precisely the one-line fix for the Unicode half of the collision. Two independent arms, blind to each other, dismissed the shipped action that solves the task's own problem. (Inside the subject's own window by fourteen months, so a real chargeable miss rather than a derivability result. No F-number because every arm of this battery is a declared non-charging control, fixed in the pre-registration before any subject was spawned. Queued in BACKLOG for a real battery on the rfcEmail surface, where it would charge.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| imprecision | ULID_REGEX |
ULID_REGEX QUOTED WITH THE SAME NON-EXISTENT i FLAG AS BOTH FABLE ARMS. Given here as /^[0-9A-HJKMNP-TV-Z]{26}$/iu, with "valibot's regex has the i flag, so lowercase passes" built on it. The real regex at 1.4.2 is /^[\da-hjkmnp-tv-zA-HJKMNP-TV-Z]{26}$/u — no i, both cases spelled out. The conclusion (lowercase passes) is true, but by enumeration rather than by folding, and the difference is measurable: a ULID containing U+212A is accepted at 0.42.0, where the i form was real, and rejected at 1.0.0, 1.4.2 and 1.5.0. Three arms, blind to each other, independently reproduced a regex that stopped being accurate in March 2025. (In-window (1.0.0 is fourteen months below the stated cutoff), so a chargeable miss and not a below-floor answer; no F-number because this is a declared non-charging arm. Not the graded question, and the arm's shipped code does not depend on the quoted flags.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.