Run valibot--claude-fable-5-1--v5-b--2026-09-24
The blind twin, and it agreed with v5-a on every one-word answer, every boundary reading and the exact text of the ULID fix — /^[0-7][0-9A-HJKMNP-TV-Z]{25}$/, which is valibot's own 1.5.0 patch, written by two draws that did not know 1.5.0 existed. Zero spread on the boundary (both read 1.1.0), four dated anchors all correct, two of them exact to the month. Nothing charges: 1.5.0 is three months above this subject's stated cutoff and every arm of this battery is a declared below-floor derivability control. Where the twins differ is mechanism, not verdict — this draw led task 2 with the prototype-pollution key guard rather than the in check, and it alone noticed the one thing about strictObject that is STILL true at 1.5.0: symbol keys go unreported, which LF18 does not cover and which is queued. Two chargeable misses inside its own window, both measured: rfcEmail called looser when it rejects exactly the characters the task is about, and the graphemes family dated to 0.42.x when it arrived in 1.0.0.
| Subject | Claude Fable 5.1 claude-fable-5-1, Anthropic |
|---|---|
| Invoked as | Agent tool, model alias "fable", general-purpose subagent, instructed to use no tools; BLIND TWIN of `valibot/v5-a`, spawned concurrently with it and with both control arms, from the byte-identical file prompts/sent/valibot-v5.txt. Identity probed through the same alias in the same session, tool-free: "Fable 5.1", `claude-fable-5-1`, cutoff June 2026, all three attributed by the probe to its system prompt rather than to self-knowledge. |
| Cutoff the model states | 2026-06 |
| Newest valibot release it could place | 1.1.0 · 2025-05-06 (~16 month lag) |
| Oldest valibot release it could not place | 1.2.0 · 2025-11-24 (so this run brackets the subject’s boundary to 2025-05-06 – 2025-11-24) |
| As recorded (this record does not declare whether the words are the subject’s or a summary of them) | "The latest version I can name is the 1.x line, around 1.1.0 (possibly 1.2.0 exists; I cannot describe it). The most recent release whose contents I can actually describe is 1.1.0, roughly May-June 2025." |
| How the bracket was read | IDENTICAL TO ITS TWIN ON THE QUANTITY THAT MATTERS, AND THE BEST-ANCHORED DRAW IN THE VALIBOT CORPUS. Both blind draws read knowledge_stops_at_version 1.1.0; they differ only in where they put the first release they cannot describe (v5-a: 1.1.1; this draw: 1.2.0, named explicitly). Zero spread on the boundary itself, which is what identical_prompt.spread_days measures. Four dated anchors, all correct: 1.1.0 "~May-June 2025" against 2025-05-06, 1.0.0 "March 2025" against 2025-03-19, 0.42.x "Sept-Oct 2024" against 2024-09-15, and 0.31.0 "June 2024" against 2024-06-06 — the last two exact to the month. Prediction P6 holds. |
| Library at test time | valibot 1.5.0 (npm), verified 2026-09-22 |
| Battery | valibot/v5-b · 5 tasks, 3 direct questions · probe window 1.5.0 to 1.5.0 |
| Tool uses during test | 0 (a run with any tool use is void — we measure training knowledge, not retrieval) |
| Tested | 2026-09-24 |
| Findings | 0, of which 0 chargeable |
None. Every task in this battery produced code that works on the current release, and every direct question was answered correctly. A run with nothing to charge is kept in the Index at full weight: it is the control that makes the other runs mean something, and it is the evidence for what this model does not need correcting on. What the subject actually said is recorded below.
Recorded so the run cannot be read as a hit list. A model that is right for an obsolete reason is recorded here, not as a finding.
| Kind | API | Note |
|---|---|---|
| miss | ulid |
TASK 1: the twin converged, independently and blind. "No", with the same overflow argument — 26 Crockford characters hold 130 bits, a ULID is 128, the first character must be 0-7, 8ZZZ.../ZZZZ... are accepted, oklog/ulid.ParseStrict returns ErrOverflow — and the same shipped fix, /^[0-7][0-9A-HJKMNP-TV-Z]{25}$/, character for character identical to v5-a's. True through 1.4.2, FALSE at 1.5.0, whose ULID_REGEX is /^[0-7][\da-hjkmnp-tv-zA-HJKMNP-TV-Z]{25}$/u. Two blind draws and one further subject independently wrote the library's own patch three months after it shipped and without knowing it had. (Below the floor by three months; declared non-charging arm. The -b twin charges nothing in any case.) |
| miss | strictObject / looseObject / objectWithRest |
TASK 2: the same "No", by a DIFFERENT mechanism, and the difference is the interesting half. Where v5-a gave LF18's key in entries prototype-chain story as its primary explanation, this draw led with an _isValidObjectKey-style own-property guard that skips __proto__, prototype and constructor as a prototype-pollution defence, and offered the in story second and with lower confidence ("I am less certain that this second part is still the case in the current release"). Both halves are real: LF18 records that constructor, __proto__ and prototype are dropped from looseObject and objectWithRest output on EVERY release, and that the in check governed strictObject through 1.4.2. At 1.5.0 strictObject rejects all five names (measured), so the conclusion — silently dropped, not reported — is false at current truth. (Below the floor by three months; declared non-charging arm.) |
| correct | strictObject |
TASK 2, THE CLAIM THAT IS STILL TRUE AT 1.5.0 AND THAT LF18 DOES NOT COVER. "Symbol and non-enumerable keys are also skipped, but JSON can't produce those." Measured: strictObject({ name }) accepts { name: 'a', [Symbol('extra')]: 'x' } on 1.4.2 AND on 1.5.0. So the graded question — does strictObject report EVERY key the schema does not declare — has the answer "no" at current truth, by a route the Index's own fact does not describe. LF18's stale_belief frames the corrected belief as "strictObject rejects every key its entries do not declare", which remains false at 1.5.0 for symbol keys. Reported, not filed: this session ran a battery, and JOURNAL/046's order says an instrument does not also write the fact it found. Queued in BACKLOG. (Correct at current truth, and the subject itself bounded it to the case the task did not ask about (JSON input cannot carry symbols). The gap it exposes is in LF18, not in this reply.) |
| correct | maxLength / maxGraphemes |
TASK 3(a) CORRECT ON EVERY RELEASE, with the most precise unit table of the four arms. "No" — maxLength compares string.length, UTF-16 code units. It named three enforceable families (length, graphemes, bytes) plus words, gave the right mechanism for each (Intl.Segmenter, TextEncoder), and shipped a graphemes-based schema with an NFC normalise and a byte cap. All four families exist on the ladder. (Correct on the graded question.) |
| miss | codePoints / minCodePoints / maxCodePoints / notCodePoints |
TASK 3(b): "Code points are not built in; use a check with [...s].length if you need them." False at 1.5.0 (LF13's four actions, measured absent at 1.4.2 and present at 1.5.0), correct through 1.4.2, and the workaround is the one LF13 prescribes for a pin below 1.5.0. Same cell as its twin, same wording almost exactly. (Below the floor by three months; declared non-charging arm.) |
| miss | literal / value / values / notValue / notValues (NaN) |
TASK 4: "No", literal compares with ===. True through 1.4.2, false at 1.5.0 (LF17). Its account of value/notValue as relational (requirement <= value && requirement >= value, to support Date) and its conclusion that notValue(NaN) never raises an issue are both exactly right for 1.4.2 — measured: notValue(NaN) accepts NaN and accepts 5 — and the NaN half reverses at 1.5.0. Prediction P3 holds on this arm. (Below the floor by three months; declared non-charging arm.) |
| correct | toAsciiSafe |
TASK 5, POISON RUNG REFUSED. toAsciiSafe() rejected by name, with the real transform list in its place. Verified undefined on 1.4.2 and 1.5.0 before the draw. Prediction P4 holds. (Correct: the action does not exist.) |
| miss | ksuid |
TASK 5: ksuid denied, then reimplemented correctly and more strictly than the library. "Valibot ships uuid, ulid, cuid2, nanoid (and things like imei, mac, slug), but no KSUID action" — the listed actions all exist (measured: imei, mac, cuid2, nanoid on every rung read; slug from 1.0.0), and the denial is true through 1.4.2 and false at 1.5.0 (LF14). Its replacement carries the encoded maximum aWgEPTl1tmebfsQzFP4bxwgy80V and a <= string comparison, and executed as written rejects aWgEPTl1tmebfsQzFP4bxwgy80W and 27 z characters — both of which valibot's own ksuid() accepts at 1.5.0. (Below the floor by three months; declared non-charging arm.) |
| miss | email |
TASK 5, THE COLLISION, IDENTIFIED EXACTLY AND ATTRIBUTED TO THE CURRENT RELEASE. Answered yes and gave LF15 in full: the i+u flag combination, simple case folding making \w and [a-z] match U+212A KELVIN SIGN and U+017F LONG S, and the collision itself. Measured against the real 1.4.2 EMAIL_REGEX — /^[\w+-]+(?:\.[\w+-]+)*@[\da-z]+(?:[.-][\da-z]+)*\.[a-z]{2,}$/iu — the description is correct down to the flags. At 1.5.0 the i is gone, A-Z is explicit, and both addresses are rejected, so "can still let that happen" is false at current truth. (Below the floor by three months; declared non-charging arm.) |
| miss | rfcEmail |
TASK 5, A MISS INSIDE THIS SUBJECT'S OWN WINDOW: "Note rfcEmail() exists too but is looser, not stricter; it does not help here." Both clauses are false, and they have been false since rfcEmail shipped. Measured on 1.0.0, 1.4.2 and 1.5.0: rfcEmail REJECTS both \u017F@example.com and \u212Aelvin@example.com while accepting kelvin@example.com. It is stricter on exactly the characters the task is about, and it is the one-line fix for the incident the colleague describes. rfcEmail shipped in 1.0.0 on 2025-03-19, sixteen months below this subject's stated cutoff, so this is not a below-floor answer. (Inside the subject's own window and therefore a real chargeable miss, not a derivability result. It carries no F-number because every arm of this battery is a declared non-charging control, fixed in the pre-registration before any subject was spawned — and because this is the -b twin, which charges nothing under the duplicate rule regardless. Queued in BACKLOG for a real battery on the rfcEmail surface.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
| imprecision | ULID_REGEX / graphemes / words |
TWO IN-WINDOW ERRORS ABOUT 0.42-ERA CONTENT, BOTH MEASURED. (i) Task 1 asserts that the ULID regex's i+u flags let U+212A and U+017F through — the same false i flag its twin and v5-c quoted. Measured: a ULID containing U+212A is accepted at 0.42.0 and REJECTED at 1.0.0, 1.4.2 and 1.5.0, because the real regex lists both cases explicitly and has no i. The draw hedged it ("I am not certain whether valibot has since hardened that regex; treat it as an open risk"), which is the right instinct and does not make the claim true. (ii) Direct question (c) places the graphemes and words families in 0.42.x: both are undefined at 0.42.0 and function at 1.0.0, which is what LF13 says. Both facts are dated fourteen months or more below this subject's stated cutoff. (In-window, so recorded as chargeable misses rather than derivability results; no F-number because this is a declared non-charging arm and the -b twin besides. Neither was the graded question.) [chargeable miss — a replicate, a duplicated arm’s second draw or a below-floor control charges nothing;
absent from the finding count] |
Battery specification: prompts/valibot.md in the studio repo.
Every finding above also carries its own citation.